diff --git a/.gitignore b/.gitignore
index 5b57e2dd..23f4970f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -33,4 +33,5 @@ notes/
docs/public
docs/resources/_gen/
docs/.hugo_build.lock
+test/integration/clab-*
test/integration/**/clab-*
diff --git a/docs/content/docs/Apis/DefaultApi.md b/docs/content/docs/Apis/DefaultApi.md
new file mode 100644
index 00000000..513f6da6
--- /dev/null
+++ b/docs/content/docs/Apis/DefaultApi.md
@@ -0,0 +1,2 @@
+# DefaultApi
+
diff --git a/docs/content/docs/advanced/openapi-generator-config.yaml b/docs/content/docs/advanced/openapi-generator-config.yaml
new file mode 100644
index 00000000..75e95512
--- /dev/null
+++ b/docs/content/docs/advanced/openapi-generator-config.yaml
@@ -0,0 +1,9 @@
+# docker run --rm -v ${PWD}:/local openapitools/openapi-generator-cli generate -c /local/docs/content/docs/advanced/openapi-generator-config.yaml
+
+generatorName: markdown
+inputSpec: /local/internal/apiserver/openapi.yaml
+outputDir: /local/docs/content/docs/advanced/rest-api-documentation
+templateDir: /local/docs/content/docs/advanced/openapi-templates
+files:
+ README.mustache:
+ destinationFilename: _index.md
\ No newline at end of file
diff --git a/docs/content/docs/advanced/openapi-templates/_index.mustache b/docs/content/docs/advanced/openapi-templates/_index.mustache
new file mode 100644
index 00000000..7b3df486
--- /dev/null
+++ b/docs/content/docs/advanced/openapi-templates/_index.mustache
@@ -0,0 +1,69 @@
+---
+title: "REST API interface"
+linkTitle: "REST API interface"
+weight: 3
+description: >
+ This document describes the REST API exposed by the gNMIc Operator, including the available endpoints, request formats, and usage examples.
+---
+
+{{#generateApiDocs}}
+
+## Documentation for API Endpoints
+
+All URIs are relative to *{{{basePath}}}:8082*
+
+| Class | Method | HTTP request | Description |
+|------------ | ------------- | ------------- | -------------|
+{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{#lambda.lowercase}}{{classname}}{{/lambda.lowercase}}* | [**{{operationId}}**](/docs/advanced/rest-api-documentation/apis/{{#lambda.lowercase}}{{classname}}{{/lambda.lowercase}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} |
+{{/operation}}{{/operations}}{{/apis}}{{/apiInfo}}
+{{/generateApiDocs}}
+
+{{#generateModelDocs}}
+
+## Documentation for Models
+
+{{#modelPackage}}
+{{#models}}{{#model}} - [{{#lambda.lowercase}}{{{classname}}}{{/lambda.lowercase}}](/docs/advanced/rest-api-documentation/models/{{#lambda.lowercase}}{{{classFilename}}}{{/lambda.lowercase}}/)
+{{/model}}{{/models}}
+{{/modelPackage}}
+{{^modelPackage}}
+No model defined in this package
+{{/modelPackage}}
+{{/generateModelDocs}}
+
+{{! TODO: optional documentation for authorization? }}
+## Documentation for Authorization
+
+For a detailed explanation on how to configure the required secrets within the gNMIc Operator, refer to [TargetSource > Push mode](/docs/user-guide/targetsource/push/).
+
+{{^authMethods}}
+All endpoints do not require authorization.
+{{/authMethods}}
+{{#authMethods}}
+{{#last}}
+ Authentication schemes defined for the API:
+{{/last}}
+{{/authMethods}}
+{{#authMethods}}
+
+### {{name}}
+
+{{#isApiKey}}- **Type**: API key
+- **API key parameter name**: {{keyParamName}}
+- **Location**: {{#isKeyInQuery}}URL query string{{/isKeyInQuery}}{{#isKeyInHeader}}HTTP header{{/isKeyInHeader}}
+{{/isApiKey}}
+{{#isBasicBasic}}- **Type**: HTTP basic authentication
+{{/isBasicBasic}}
+{{#isBasicBearer}}- **Type**: HTTP Bearer Token authentication{{#bearerFormat}} ({{{.}}}){{/bearerFormat}}
+{{/isBasicBearer}}
+{{#isHttpSignature}}- **Type**: HTTP signature authentication
+{{/isHttpSignature}}
+{{#isOAuth}}- **Type**: OAuth
+- **Flow**: {{flow}}
+- **Authorization URL**: {{authorizationUrl}}
+- **Scopes**: {{^scopes}}N/A{{/scopes}}
+{{#scopes}} - {{scope}}: {{description}}
+{{/scopes}}
+{{/isOAuth}}
+
+{{/authMethods}}
\ No newline at end of file
diff --git a/docs/content/docs/advanced/openapi-templates/apis.mustache b/docs/content/docs/advanced/openapi-templates/apis.mustache
new file mode 100644
index 00000000..8a75a8a9
--- /dev/null
+++ b/docs/content/docs/advanced/openapi-templates/apis.mustache
@@ -0,0 +1,50 @@
+---
+title: "Routes"
+linkTitle: "Routes"
+weight: 4
+description: >
+ Available HTTP routes on the gNMIc Operator API interface.
+---
+
+# {{#lambda.lowercase}}{{classname}}{{/lambda.lowercase}}{{#description}}
+ {{.}}{{/description}}
+
+All URIs are relative to *{{basePath}}:8082*
+
+| Method | HTTP request | Description |
+|------------- | ------------- | -------------|
+{{#operations}}{{#operation}}| **{{operationId}}** | **{{httpMethod}}** {{path}} | {{summary}} |
+{{/operation}}{{/operations}}
+
+{{#operations}}
+{{#operation}}
+
+# **{{operationId}}**
+> {{#returnType}}{{.}} {{/returnType}}{{operationId}}({{#allParams}}{{{paramName}}}{{^-last}}, {{/-last}}{{/allParams}})
+
+{{summary}}{{#notes}}
+
+ {{.}}{{/notes}}
+
+### Parameters
+{{^allParams}}This endpoint does not need any parameter.{{/allParams}}{{#allParams}}{{#-last}}
+|Name | Type | Description | Notes |
+|------------- | ------------- | ------------- | -------------|{{/-last}}{{/allParams}}
+{{#allParams}}| **{{paramName}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}{{#isFile}}**{{dataType}}**{{/isFile}}{{^isFile}}{{#generateModelDocs}}[**{{dataType}}**](/docs/advanced/rest-api-documentation/models/{{#lambda.lowercase}}{{baseType}}{{/lambda.lowercase}}/){{/generateModelDocs}}{{^generateModelDocs}}**{{dataType}}**{{/generateModelDocs}}{{/isFile}}{{/isPrimitiveType}}| {{description}} |{{^required}} [optional]{{/required}}{{#defaultValue}} [default to {{.}}]{{/defaultValue}}{{#allowableValues}} [enum: {{#values}}{{{.}}}{{^-last}}, {{/-last}}{{/values}}]{{/allowableValues}} |
+{{/allParams}}
+
+### Return type
+
+{{#returnType}}{{#returnTypeIsPrimitive}}**{{returnType}}**{{/returnTypeIsPrimitive}}{{^returnTypeIsPrimitive}}{{#generateModelDocs}}[**{{returnType}}**](/docs/advanced/rest-api-documentation/models/{{#lambda.lowercase}}{{returnBaseType}}{{/lambda.lowercase}}/){{/generateModelDocs}}{{^generateModelDocs}}**{{returnType}}**{{/generateModelDocs}}{{/returnTypeIsPrimitive}}{{/returnType}}{{^returnType}}null (empty response body){{/returnType}}
+
+### Authorization
+
+{{^authMethods}}No authorization required{{/authMethods}}{{#authMethods}}[{{name}}](/docs/advanced/rest-api-documentation/#{{name}}){{^-last}}, {{/-last}}{{/authMethods}}
+
+### HTTP request headers
+
+- **Content-Type**: {{#consumes}}{{{mediaType}}}{{^-last}}, {{/-last}}{{/consumes}}{{^consumes}}Not defined{{/consumes}}
+- **Accept**: {{#produces}}{{{mediaType}}}{{^-last}}, {{/-last}}{{/produces}}{{^produces}}Not defined{{/produces}}
+
+{{/operation}}
+{{/operations}}
\ No newline at end of file
diff --git a/docs/content/docs/advanced/openapi-templates/models.mustache b/docs/content/docs/advanced/openapi-templates/models.mustache
new file mode 100644
index 00000000..a110b934
--- /dev/null
+++ b/docs/content/docs/advanced/openapi-templates/models.mustache
@@ -0,0 +1,29 @@
+---
+title: "Model"
+linkTitle: "Model"
+weight: 4
+description: >
+ Documentation for OpenAPI models and their schema-defined properties.
+---
+
+{{#models}}
+{{#model}}
+# {{#lambda}}{{{classname}}}{{/lambda}}
+{{#description}}
+{{{description}}}
+{{/description}}
+
+## Properties
+
+| Name | Type | Description | Notes |
+|------------ | ------------- | ------------- | -------------|
+{{#parent}}
+{{#parentVars}}
+| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} |
+{{/parentVars}}
+{{/parent}}
+{{#vars}}| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} |
+{{/vars}}
+
+ {{/model}}
+{{/models}}
\ No newline at end of file
diff --git a/docs/content/docs/advanced/rest-api-documentation.md b/docs/content/docs/advanced/rest-api-documentation.md
new file mode 100644
index 00000000..edba35f0
--- /dev/null
+++ b/docs/content/docs/advanced/rest-api-documentation.md
@@ -0,0 +1,33 @@
+---
+title: "Documentation for gNMIc Operator REST API"
+linkTitle: "REST API interface"
+weight: 3
+description: >
+ Documentation of REST API interface according to openAPI standard.
+---
+
+
+## Documentation for API Endpoints
+
+All URIs are relative to *http://localhost*
+
+| Class | Method | HTTP request | Description |
+|------------ | ------------- | ------------- | -------------|
+| *DefaultApi* | [**applyTargets**](../Apis/DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. |
+*DefaultApi* | [**getClusterPlan**](../Apis/DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. |
+
+
+
+## Documentation for Models
+
+ - [Target](../Models/Target.md)
+
+
+
+## Documentation for Authorization
+
+
+### bearerAuth
+
+- **Type**: HTTP Bearer Token authentication
+
diff --git a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md
new file mode 100644
index 00000000..a5f0f3d4
--- /dev/null
+++ b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md
@@ -0,0 +1,57 @@
+# DefaultApi
+
+All URIs are relative to *http://localhost*
+
+| Method | HTTP request | Description |
+|------------- | ------------- | -------------|
+| [**applyTargets**](DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. |
+| [**getClusterPlan**](DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. |
+
+
+
+# **applyTargets**
+> List applyTargets(Target)
+
+Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator.
+
+### Parameters
+
+|Name | Type | Description | Notes |
+|------------- | ------------- | ------------- | -------------|
+| **Target** | [**List**](../Models/Target.md)| Target must be passed as a list, multiple targets possible. | |
+
+### Return type
+
+[**List**](../Models/Target.md)
+
+### Authorization
+
+[signature](../README.md#signature), [bearerAuth](../README.md#bearerAuth)
+
+### HTTP request headers
+
+- **Content-Type**: application/json
+- **Accept**: application/json
+
+
+# **getClusterPlan**
+> getClusterPlan()
+
+Get cluster plan.
+
+### Parameters
+This endpoint does not need any parameter.
+
+### Return type
+
+null (empty response body)
+
+### Authorization
+
+No authorization required
+
+### HTTP request headers
+
+- **Content-Type**: Not defined
+- **Accept**: Not defined
+
diff --git a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md
new file mode 100644
index 00000000..18f7031c
--- /dev/null
+++ b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md
@@ -0,0 +1,14 @@
+# Target
+## Properties
+
+| Name | Type | Description | Notes |
+|------------ | ------------- | ------------- | -------------|
+| **name** | **String** | Name of device to be monitored. | [default to null] |
+| **address** | **String** | IPv4/IPv6 address or hostname. | [default to null] |
+| **port** | **Integer** | gNMIc port. | [optional] [default to null] |
+| **targetProfile** | **String** | TargetProfile applied to apply to this router. | [optional] [default to null] |
+| **labels** | [**List**](map.md) | Labels must be map[string]string. For example vendor:nokia. | [optional] [default to null] |
+| **operation** | **String** | Either `created`, `updated` or `deleted`. `created` and `updated` are identical and both apply the target. | [default to null] |
+
+[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
+
diff --git a/docs/content/docs/advanced/rest-api-documentation/_index.md b/docs/content/docs/advanced/rest-api-documentation/_index.md
new file mode 100644
index 00000000..a552b56a
--- /dev/null
+++ b/docs/content/docs/advanced/rest-api-documentation/_index.md
@@ -0,0 +1,34 @@
+# Documentation for gNMIc Operator REST API
+
+
+## Documentation for API Endpoints
+
+All URIs are relative to *http://localhost*
+
+| Class | Method | HTTP request | Description |
+|------------ | ------------- | ------------- | -------------|
+| *DefaultApi* | [**applyTargets**](Apis/DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. |
+*DefaultApi* | [**getClusterPlan**](Apis/DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. |
+
+
+
+## Documentation for Models
+
+ - [Target](./Models/Target.md)
+
+
+
+## Documentation for Authorization
+
+
+### bearerAuth
+
+- **Type**: HTTP Bearer Token authentication
+
+
+### signature
+
+- **Type**: API key
+- **API key parameter name**: X-Hook-Signature
+- **Location**: HTTP header
+
diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md
new file mode 100644
index 00000000..fd481841
--- /dev/null
+++ b/docs/content/docs/examples/NetBox/Export Template/_index.md
@@ -0,0 +1,399 @@
+---
+title: "Pull with Export Template"
+linkTitle: "Pull with Export Template"
+weight: 1
+description: >
+ Discover targets from NetBox using HTTP provider with NetBox Export Template
+---
+
+This guide shows how to use **NetBox Export Templates** with the HTTP provider to discover and sync targets.
+
+Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing the load on the operator.
+
+## Overview
+
+An **Export Template** is a Jinja2 template that:
+
+1. **Queries** NetBox's internal database (devices, interfaces, etc.)
+2. **Filters** results based on custom criteria
+3. **Transforms** data into your desired output format
+4. **Returns** the formatted output via REST API endpoint
+
+When used with gNMIc's HTTP provider, the operator fetches the rendered JSON template and parses the result with no further transformation needed by the gNMIc Operator.
+
+---
+
+## Prerequisites
+
+- A running Kubernetes cluster with gNMIc Operator installed
+- `kubectl` access to your cluster
+- A reachable NetBox instance with permissions to create Export Templates
+- A NetBox API token
+- Familiarity with Jinja2 templates
+
+---
+
+## Step 1: Create a NetBox API Token and Store It Securely
+
+### Step 1a: Create the API Token in NetBox
+
+Create a dedicated API token in NetBox for gNMIc Operator access.
+
+1. Log in to NetBox.
+2. Open your user profile or go to **User > API Tokens**.
+3. Click **Add** or **Add token**.
+4. Enter a descriptive name such as `gNMIc Operator`.
+5. Grant the minimum permissions required for read-only device discovery.
+6. Copy the token value and store it safely; NetBox will not show it again.
+
+### Step 1b: Store the Token in a Kubernetes Secret
+
+Create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing the token.
+
+```bash
+# Substitute YOUR_NETBOX_API_TOKEN with your actual token
+# Bearer Token Format (v2): nbt_.
+kubectl create secret generic netbox-api-token \
+ --from-literal=token=YOUR_NETBOX_API_TOKEN \
+ -n gnmic-system
+```
+
+Verify the Secret was created:
+
+```bash
+kubectl get secret netbox-api-token -n gnmic-system -o yaml
+```
+
+---
+
+## Step 2: Create an Export Template in NetBox
+
+Log in to your NetBox instance and navigate to **Customization > Export Templates**.
+
+### Step 2a: Create a New Template
+
+Click **Add Export Template** and fill in the details:
+
+| Field | Value | Notes |
+|-------|-------|-------|
+| **Name** | `gNMIc Device Export` | Descriptive name for your template |
+| **Content Type** | `dcim > device` | Export template applies to Device objects |
+| **Template Code** | (see below) | Jinja2 template |
+| **File Extension** | `json` | Output format |
+| **Mime Type** | `application/json` | Correct MIME type for JSON |
+
+### Step 2b: Template Code Example
+
+The following Export Templates only work for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox data model details, see the [NetBox Devices Data Model](https://netboxlabs.com/docs/netbox/models/dcim/device/) documentation.
+
+See the HTTP provider's "Default Response Format" section for the expected JSON structure: [HTTP provider docs](../../user-guide/targetsource/providers/http.md)
+
+#### Basic Template (All Devices)
+
+```jinja2
+[
+ {% for device in queryset %}
+ {
+ "name": "{{ device.name }}",
+ "address": "{{ device.primary_ip4.address.ip }}",
+ "labels": {
+ "site": "{{ device.site.name }}",
+ "role": "{{ device.role.name }}",
+ "region": "{{ device.site.region.name }}",
+ "type": "{{ device.device_type.model }}"
+ }
+ }{{ "," if not loop.last }}
+ {% endfor %}
+]
+```
+
+#### Advanced Template (Filtered by Status and Role)
+
+```jinja2
+[
+ {% for device in queryset.filter(status='active', role__name__in=['leaf', 'spine']) %}
+ {
+ "name": "{{ device.name }}",
+ "address": "{{ device.primary_ip4.address.ip }}",
+ "labels": {
+ "site": "{{ device.site.name }}",
+ "role": "{{ device.role.name }}",
+ "region": "{{ device.site.region.name }}",
+ "model": "{{ device.device_type.model }}",
+ "serial": "{{ device.serial }}",
+ "asset_tag": "{{ device.asset_tag }}"
+ }
+ }{{ "," if not loop.last }}
+ {% endfor %}
+]
+```
+
+**Key template elements:**
+
+- `queryset`: The filtered set of devices (all unless you add `.filter()`)
+- `device.name`: Device hostname
+- `device.primary_ip4.address.ip`: Primary IPv4 address
+- `device.site.name`, `device.device_role.name`: NetBox relationships (site, role, etc.)
+- `loop.last`: Jinja2 loop variable to avoid trailing comma on last item
+
+### Step 2c: Save and Access the Template
+
+Once saved, NetBox exposes the template via:
+
+```
+http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc+Device+Export
+```
+
+Or fetch it directly:
+
+```bash
+# Replace with your NetBox URL and template name
+# Substitute YOUR_NETBOX_API_TOKEN with your actual token
+# Bearer Token Format (v2): nbt_.
+curl -H "Authorization: Bearer YOUR_NETBOX_API_TOKEN" \
+ "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export"
+```
+
+The response should be a JSON array of targets ready for the gNMIc Operator.
+
+Sample JSON output produced by the basic export template:
+
+```json
+[
+
+ {
+ "name": "edge-rtr-01.dc1.example.com",
+ "address": "203.0.113.1",
+ "labels": {
+ "site": "DC1",
+ "role": "edge",
+ "region": "eu-central-1",
+ "type": "router"
+ }
+ },
+
+]
+```
+
+> Ensure the response is valid JSON and contains no hidden or invalid characters, otherwise the gNMIc Operator will fail to parse it.
+
+> If you instead return a JSON object with a nested array, add a mapping section such as `targetsField: "self.targets"` to the TargetSource CR.
+
+---
+
+## Step 3: Create a TargetProfile
+
+Define how discovered targets should be configured. The `TargetProfile` points to a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing device credentials, such as username/password or client certificates.
+
+Create a credentials Secret first, then reference it from the TargetProfile.
+
+```yaml
+# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password
+apiVersion: v1
+kind: Secret
+metadata:
+ name: device-credentials
+ namespace: gnmic-system
+type: Opaque
+stringData:
+ username: YOUR_DEVICE_USERNAME
+ password: YOUR_DEVICE_PASSWORD
+```
+
+```yaml
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetProfile
+metadata:
+ name: netbox-device
+ namespace: gnmic-system
+spec:
+ credentialsRef: device-credentials
+ timeout: 10s
+```
+
+For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`.
+
+---
+
+## Step 4: Create a TargetSource Using Export Template
+
+Create a `TargetSource` that references your NetBox export template endpoint:
+
+```yaml
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetSource
+metadata:
+ name: netbox-export-source
+ namespace: gnmic-system
+spec:
+ targetPort: 57400
+ targetProfile: netbox-device
+ targetLabels:
+ inventory: netbox
+ sync-source: export-template
+ provider:
+ http:
+ url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export"
+ method: GET
+ interval: 30m
+ timeout: 30s
+ authentication:
+ token:
+ scheme: Token
+ tokenSecretRef:
+ name: netbox-api-token
+ key: token
+```
+
+---
+
+## Step 5: Verify Target Discovery
+
+Once the `TargetSource` is deployed, verify that targets are being discovered:
+
+```bash
+# List discovered targets
+kubectl get targets -n gnmic-system
+
+# Check TargetSource status and sync details
+kubectl describe targetsource netbox-export-source -n gnmic-system
+```
+
+Successful sync shows:
+
+- `status.status`: "success" (or similar)
+- `status.targetsCount`: number of devices
+- `status.lastSync`: recent timestamp
+
+---
+
+## Example: Complete Setup
+
+Here's a full example combining all components:
+
+```yaml
+---
+# Secret for NetBox API token
+apiVersion: v1
+kind: Secret
+metadata:
+ name: netbox-api-token
+ namespace: gnmic-system
+type: Opaque
+data:
+ # base64-encoded token (echo -n "YOUR_TOKEN" | base64)
+ token: YOUR_BASE64_ENCODED_TOKEN
+
+---
+# Secret for Target Credential
+apiVersion: v1
+kind: Secret
+metadata:
+ name: device-credentials
+ namespace: gnmic-system
+type: Opaque
+stringData:
+ username: YOUR_DEVICE_USERNAME
+ password: YOUR_DEVICE_PASSWORD
+
+---
+# TargetProfile
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetProfile
+metadata:
+ name: netbox-device
+ namespace: gnmic-system
+spec:
+ credentialsRef: device-credentials
+ timeout: 10s
+
+
+---
+# TargetSource using Export Template
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetSource
+metadata:
+ name: netbox-export-source
+ namespace: gnmic-system
+spec:
+ targetPort: 57400
+ targetProfile: netbox-device
+ targetLabels:
+ inventory: netbox
+ sync-source: export-template
+ provider:
+ http:
+ url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export"
+ method: GET
+ interval: 30m
+ timeout: 30s
+ authentication:
+ token:
+ scheme: Token
+ tokenSecretRef:
+ name: netbox-api-token
+ key: token
+```
+
+---
+
+## Advantages of Export Templates
+
+- **Powerful Filtering**: Filter devices by site, status, role, tags, etc. directly in NetBox
+- **Reduced Operator Load**: NetBox handles data transformation; operator just fetches JSON
+- **Reusability**: One template can serve multiple consumers
+- **Maintainability**: Update discovery logic in NetBox without changing Kubernetes manifests
+- **Performance**: Avoids REST API pagination for large inventories
+
+---
+
+## Limitations & Considerations
+
+### 1. Reverse Proxy and URL Path Rewriting
+
+If NetBox is behind a reverse proxy with URL path rewriting:
+
+- **Issue**: The export template endpoint uses query parameters that may not survive proxy transformation.
+- **Solution**:
+ - Ensure the proxy preserves query strings exactly.
+ - Test the export URL directly:
+ ```bash
+ curl -H "Authorization: Token YOUR_TOKEN" \
+ "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export"
+ ```
+ - If the proxy blocks or modifies parameters, consider using a direct NetBox endpoint without proxying.
+
+### 2. Large Inventory Rendering
+
+- Very large device counts can cause NetBox to take time rendering the template.
+- **Solution**:
+ - Use `.filter()` in your template to limit results.
+ - Create separate export templates for different device groups (e.g., by site or role).
+
+### 3. Complex Jinja2 Logic
+
+- NetBox's Jinja2 sandbox restricts some Python functions for security.
+- **Solution**: Keep templates simple and use NetBox's built-in filters and objects. Test the URL with curl or similar before deploying.
+
+---
+
+## Template Troubleshooting
+
+### Missing Targets in Kubernetes
+
+- **Check**: Are all required fields populated in NetBox? (e.g., `primary_ip4` may be `None` if not set)
+- **Solution**: Add conditional checks:
+ ```jinja2
+ {% if device.primary_ip4 %}
+ "address": "{{ device.primary_ip4.address.ip }}"
+ {% endif %}
+ ```
+
+### Authorization Fails
+
+If you get a 403 error:
+
+- Verify the token is valid and not expired.
+- Ensure the API token is enabled.
+
+---
diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md
new file mode 100644
index 00000000..fcbd990a
--- /dev/null
+++ b/docs/content/docs/examples/NetBox/REST API/_index.md
@@ -0,0 +1,319 @@
+---
+title: "Pull with REST API"
+linkTitle: "Pull with REST API"
+weight: 2
+description: >
+ Discover targets from NetBox using the HTTP provider and NetBox REST API
+---
+
+This guide shows how to configure the HTTP provider to discover targets from NetBox using its REST API.
+
+The REST API approach is direct and straightforward — query NetBox's standard API endpoints to retrieve devices that match your criteria.
+
+## Prerequisites
+
+- A running Kubernetes cluster with gNMIc Operator installed
+- `kubectl` access to your cluster
+- A reachable NetBox instance (inside or outside the cluster)
+- A NetBox API token
+
+## Overview
+
+The HTTP `TargetSource` loader performs these steps:
+
+1. **Fetch** JSON device data from a NetBox REST API endpoint (`/api/dcim/devices/`)
+2. **Transform** each device record into a gNMIc target using CEL expressions
+3. **Create** or **update** `Target` resources in Kubernetes with the extracted data
+
+---
+
+## Step 1: Create a NetBox API Token and Store It Securely
+
+### Step 1a: Create the API Token in NetBox
+
+Create a dedicated API token in NetBox for gNMIc Operator access.
+
+1. Log in to NetBox.
+2. Open your user profile or go to **User > API Tokens**.
+3. Click **Add** or **Add token**.
+4. Enter a descriptive name such as `gNMIc Operator`.
+5. Grant the minimum permissions required for read-only device discovery.
+6. Copy the token value and store it safely; NetBox will not show it again.
+
+### Step 1b: Store the Token in a Kubernetes Secret
+
+Create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing the token so it is not embedded in manifests.
+
+```bash
+# Substitute YOUR_NETBOX_API_TOKEN with your actual token
+# Bearer Token Format (v2): nbt_.
+kubectl create secret generic netbox-api-token \
+ --from-literal=token=YOUR_NETBOX_API_TOKEN \
+ -n gnmic-system
+```
+
+Verify the Secret was created:
+
+```bash
+kubectl get secret netbox-api-token -n gnmic-system -o yaml
+```
+
+---
+
+## Step 2: Create a TargetProfile
+
+Define how discovered targets should be configured. The `TargetProfile` points to a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing device credentials, such as username/password or client certificates.
+
+Create a credentials Secret first, then reference it from the profile.
+
+```yaml
+# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password
+apiVersion: v1
+kind: Secret
+metadata:
+ name: device-credentials
+ namespace: gnmic-system
+type: Opaque
+stringData:
+ username: YOUR_DEVICE_USERNAME
+ password: YOUR_DEVICE_PASSWORD
+```
+
+```yaml
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetProfile
+metadata:
+ name: netbox-device
+ namespace: gnmic-system
+spec:
+ credentialsRef: device-credentials
+ timeout: 10s
+```
+
+For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`.
+
+---
+
+## Step 3: Create a TargetSource Using REST API
+
+The following `TargetSource` queries NetBox's REST API to discover devices:
+
+```yaml
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetSource
+metadata:
+ name: netbox-rest-source
+ namespace: gnmic-system
+spec:
+ targetPort: 57400
+ targetProfile: netbox-device
+ targetLabels:
+ inventory: netbox
+ sync-source: rest-api
+ provider:
+ http:
+ url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000"
+ method: GET
+ interval: 5m
+ timeout: 30s
+ authentication:
+ token:
+ scheme: Bearer
+ tokenSecretRef:
+ name: netbox-api-token
+ key: token
+ pagination:
+ nextField: "next"
+ mapping:
+ targetsField: "self.results"
+ address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''"
+ labels: |
+ {
+ "site": item.site.name,
+ "role": item.device_role.name,
+ "model": item.device_type.model,
+ "status": item.status.value
+ }
+```
+
+> This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation.
+
+The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the HTTP Provider docs "Response Mapping via CEL" section for more details: [HTTP provider docs](../../user-guide/targetsource/providers/http.md)
+
+Use `self` for the full response and `item` for each candidate object.
+
+---
+
+## Step 4: Apply and Verify Target Discovery
+
+Deploy the `TargetSource` and check that targets are being discovered and synced:
+
+```bash
+# List discovered targets
+kubectl apply -f /path/to/targetsource.yaml -n gnmic-system
+
+# List discovered targets
+kubectl get targets -n gnmic-system
+
+# Check TargetSource status
+kubectl describe targetsource netbox-rest-source -n gnmic-system
+```
+
+Look for:
+- `status.status`: "success" (or similar)
+- `status.targetsCount`: number of discovered devices
+- `status.lastSync`: recent timestamp
+
+---
+
+## Example: Complete Setup
+
+Here's a complete example combining all resources:
+
+```yaml
+---
+# Secret for NetBox API token
+apiVersion: v1
+kind: Secret
+metadata:
+ name: netbox-api-token
+ namespace: gnmic-system
+type: Opaque
+data:
+ # base64-encoded token (echo -n "YOUR_TOKEN" | base64)
+ token: YOUR_BASE64_ENCODED_TOKEN
+
+---
+# Secret for Target Credential
+apiVersion: v1
+kind: Secret
+metadata:
+ name: device-credentials
+ namespace: gnmic-system
+type: Opaque
+stringData:
+ username: YOUR_DEVICE_USERNAME
+ password: YOUR_DEVICE_PASSWORD
+
+---
+# TargetProfile
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetProfile
+metadata:
+ name: netbox-device
+ namespace: gnmic-system
+spec:
+ credentialsRef: device-credentials
+ timeout: 10s
+
+---
+# TargetSource with REST API
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetSource
+metadata:
+ name: netbox-rest-source
+ namespace: gnmic-system
+spec:
+ targetPort: 57400
+ targetProfile: netbox-device
+ targetLabels:
+ inventory: netbox
+ sync-source: rest-api
+ provider:
+ http:
+ url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000"
+ method: GET
+ interval: 5m
+ timeout: 30s
+ authentication:
+ token:
+ scheme: Bearer
+ tokenSecretRef:
+ name: netbox-api-token
+ key: token
+ pagination:
+ nextField: "next"
+ mapping:
+ targetsField: "self.results"
+ address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''"
+ labels: |
+ {
+ "site": item.site.name,
+ "role": item.device_role.name,
+ "model": item.device_type.model,
+ "status": item.status.value
+ }
+```
+
+---
+
+## Performance Considerations & Limitations
+
+### REST API Query Limits
+
+- **Query Size**: The example uses `limit=1000`. Adjust based on your NetBox instance's pagination settings and response size limits.
+- **Response Timeout**: Large device lists can take time. Set appropriate timeouts in your `TargetSource`.
+
+### Reverse Proxy Considerations
+
+If NetBox is behind a reverse proxy:
+
+- **Base URL**: Ensure the reverse proxy correctly handles the `/api/dcim/devices/` path.
+- **Authentication**: Some proxies may require additional headers; verify with your proxy and NetBox admin.
+- **HTTPS**: If using HTTPS, ensure certificates are trusted by the operator or else use the `tls` setting.
+
+### Large Inventories
+
+For inventories with thousands of devices:
+
+- Consider using **Export Templates** (see [NetBox Export Templates]({{< relref "../Export Template" >}})) for better filtering and performance.
+- Implement filtering in the REST API URL (e.g., `?site=us-west&status=active`).
+
+---
+
+## Security Considerations
+
+### Token and Credentials
+
+- **Never** embed plaintext tokens or credentials in manifests or YAML files.
+- Always store tokens in Kubernetes Secrets.
+- Restrict RBAC permissions on the Secret to only necessary service accounts.
+
+### HTTPS and Certificates
+
+If connecting to NetBox via HTTPS:
+
+- Ensure cluster DNS resolves the hostname correctly.
+- Mount CA certificates if using self-signed certificates.
+- Verify the operator's HTTP client configuration for certificate validation.
+
+---
+
+## Troubleshooting
+
+### Show TargetSource Errors
+
+```bash
+kubectl describe targetsource netbox-rest-source -n gnmic-system
+```
+
+### Targets Not Appearing
+
+- Check that the `TargetProfile` exists and is correctly referenced.
+- Verify labels and addresses are being extracted correctly from the NetBox response.
+- Review operator logs for parsing errors:
+ ```bash
+ kubectl logs -l app=gnmic-operator -n gnmic-operator-system
+ ```
+
+### Rate Limiting or Timeouts
+
+Increase the sync interval in your `TargetSource` or adjust timeouts:
+
+```yaml
+spec:
+ provider:
+ http:
+ interval: 1h
+ timeout: 1m
+```
diff --git a/docs/content/docs/examples/NetBox/_index.md b/docs/content/docs/examples/NetBox/_index.md
new file mode 100644
index 00000000..4885d60e
--- /dev/null
+++ b/docs/content/docs/examples/NetBox/_index.md
@@ -0,0 +1,8 @@
+---
+title: "NetBox"
+linkTitle: "NetBox"
+weight: 6
+# draft: true
+description: >
+ Discover targets from NetBox using the HTTP provider
+---
diff --git a/docs/content/docs/examples/NetBox/webhook/_index.md b/docs/content/docs/examples/NetBox/webhook/_index.md
new file mode 100644
index 00000000..5cc0d316
--- /dev/null
+++ b/docs/content/docs/examples/NetBox/webhook/_index.md
@@ -0,0 +1,236 @@
+---
+title: "Push Mode with Webhook"
+linkTitle: "Push Mode with Webhook"
+weight: 2
+description: >
+ Configure a webhook in NetBox to update targets in the gNMIc Operator in real time.
+---
+
+## Netbox Webhook Configuration
+
+This example walks through configuring a webhook in NetBox to push real-time target updates to the gNMIc Operator. It covers the configuration in the gNMIc Operator (Step 1-3), and the configuration within Netbox (step 4).
+
+1. Create Targetprofile
+2. Create Kubernetes Secrets
+3. Apply TargetSource
+4. Netbox setup
+ a: Configure Webhook
+ b: Create Event Rule
+5. Verification
+
+At the end, the logs will show the incoming POST requests and the targets updates can be verified with `kubectl get targets`.
+
+## Prerequisites
+
+- Kubernetes cluster with gNMIc Operator installed
+- `kubectl` access to your cluster
+- Running NetBox instance
+- Network connectivity from NetBox to the gNMIc Operator API endpoint
+
+---
+
+### 1. Create TargetProfile
+
+Define how discovered targets should be configured. The `TargetProfile` contains device credentials, such as username/password or client certificates. These are either defined inline strings or stored in a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/).
+
+```yaml
+# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password
+apiVersion: v1
+kind: Secret
+metadata:
+ name: device-credentials
+ namespace: gnmic-system
+type: Opaque
+stringData:
+ username: YOUR_DEVICE_USERNAME
+ password: YOUR_DEVICE_PASSWORD
+```
+
+When using a secret, create a credentials Secret first, then reference it from the profile.
+
+```yaml
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetProfile
+metadata:
+ name: netbox-device
+ namespace: gnmic-system
+spec:
+ credentialsRef: device-credentials
+ timeout: 10s
+```
+
+For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`.
+
+---
+
+### 2. Create Kubernetes Secrets
+
+Bearer authentication and signature verification both require Kubernetes secrets. Ensure that the secrets:
+
+- Are created in the same namespace as the TargetSource (`gnmic-system` in this example).
+- Use `name` and `key` values that match the TargetSource spec.
+
+```bash
+kubectl create secret generic gnmic-api-auth --from-literal=bearer-token=YOUR_SECRET_TOKEN -n gnmic-system
+kubectl create secret generic gnmic-signature --from-literal=signature=YOUR_SECRET_SIGNATURE -n gnmic-system
+```
+
+---
+
+### 3. Apply TargetSource
+
+The TargetSource has the following settings configured:
+
+- `spec.provider.http.push.enabled` must be set to `true`, otherwise updates are rejected.
+- Bearer authentication and signature verification are enabled, referencing to the secrets created in step 2.
+
+```yaml
+# netbox.yaml
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetSource
+metadata:
+ name: netbox
+ namespace: gnmic-system
+spec:
+ targetPort: 57400
+ targetProfile: netbox-device
+ targetLabels:
+ inventory: netbox
+ sync-source: rest-api
+ provider:
+ http:
+ push:
+ enabled: true
+ auth:
+ bearer:
+ tokenSecretRef:
+ name: gnmic-api-auth
+ key: bearer-token
+ signature:
+ secretRef:
+ name: gnmic-signature
+ key: signature
+```
+
+> Namespace is `gnmic-system`, the name of the TargetSource is `netbox`. These values will be in the URL in step 4.
+
+---
+
+### 4. Netbox Setup
+
+Next, configure a webhook in NetBox. The webhook is triggered by device events (for example, updates) and sends an HTTP POST request to the gNMIc Operator.
+
+#### Configure Webhook
+
+In NetBox, go to `Operations > Webhooks` and create a webhook with the following settings:
+
+- *Name*: gNMIc Operator push
+- *URL*: `http://gnmic-controller-manager-api.gnmic-system.svc.cluster.local:8082/api/v1/gnmic-system/target-source/netbox/applyTargets`
+ - URL contains the namespace `gnmic-system` and TargetSource name `netbox`. See section address in [Push Mode](/docs/user-guide/targetsource/push/) for more details on URL construction.
+ - `gnmic-controller-manager-api.gnmic-system.svc.cluster.local` is only reachable if Netbox is inside the cluster.
+ - The address may instead be `http://localhost:8082/` or `http://servername:8082/`.
+- *HTTP method*: POST
+- *HTTP content type*: application/json
+- *Additional headers:* `Authorization: Bearer YOUR_SECRET_TOKEN`
+- *Body Template*:
+
+ ```json
+ [
+ {
+ "name": "{{ data.name }}",
+ "address": "{{ data.primary_ip4.address.split('/')[0] }}",
+ "operation": "{{ event }}",
+ "targetProfile": "{{ data.custom_fields.target_profile | default('', true) }}",
+ "port": {{ data.custom_fields.gnmic_port | default(57400, true) }},
+ "labels": [
+ {"vendor":"{{ data.device_type.manufacturer.name }}"}
+ ]
+ }
+ ]
+ ```
+
+- *Secret*: `YOUR_SECRET_SIGNATURE`
+- *SSL Verification*: true
+
+#### Create Event Rule
+
+The webhook requires a trigger, configured as an event rule under `Operations > Event Rules`.
+
+- *Name*: gNMIc Operator push target change
+- *Object types*: `DCIM > Device`
+- *Event types*: `Object Created`, `Object Updated` and `Object Deleted`
+- *Action type*: Webhook
+- *Webhook*: gNMIc Operator push
+
+---
+
+### 5. Verification
+
+Updating a device in NetBox should now trigger the webhook. Verify this with the following commands:
+
+```bash
+kubectl get targets
+kubectl get targets -o yaml
+
+# Check logs of incoming POST requests:
+kubectl logs -n gnmic-system deploy/gnmic-controller-manager -f
+```
+
+Every incoming POST request is logged, including rejected requests. If no POST requests appear in the logs, the webhook request is not reaching the gNMIc Operator.
+
+---
+
+## Example: Complete Setup
+
+Here's a complete example combining all resources:
+
+ ```yaml
+---
+# Secret for Target Credential
+apiVersion: v1
+kind: Secret
+metadata:
+ name: device-credentials
+ namespace: gnmic-system
+type: Opaque
+stringData:
+ username: YOUR_DEVICE_USERNAME
+ password: YOUR_DEVICE_PASSWORD
+
+---
+# TargetProfile
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetProfile
+metadata:
+ name: netbox-device
+ namespace: gnmic-system
+spec:
+ credentialsRef: device-credentials
+ timeout: 10s
+---
+# Apply Targetsource
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetSource
+metadata:
+ name: netbox
+ namespace: gnmic-system
+spec:
+ targetPort: 57400
+ targetProfile: netbox-device
+ targetLabels:
+ inventory: netbox
+ sync-source: rest-api
+ provider:
+ http:
+ push:
+ enabled: true
+ auth:
+ bearer:
+ tokenSecretRef:
+ name: gnmic-api-auth
+ key: bearer-token
+ signature:
+ secretRef:
+ name: gnmic-signature
+ key: signature
+```
diff --git a/docs/content/docs/reference/api.md b/docs/content/docs/reference/api.md
index 1cfe4fb0..d20cb3e5 100644
--- a/docs/content/docs/reference/api.md
+++ b/docs/content/docs/reference/api.md
@@ -153,30 +153,111 @@ description: >
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
-| `http` | HTTPConfig | No | - | HTTP endpoint for target discovery |
-| `consul` | ConsulConfig | No | - | Consul service discovery config |
-| `configMap` | string | No | - | ConfigMap name containing targets |
-| `podSelector` | LabelSelector | No | - | Select Kubernetes Pods as targets |
-| `serviceSelector` | LabelSelector | No | - | Select Kubernetes Services as targets |
-| `labels` | map[string]string | No | - | Labels to apply to discovered targets |
+| `provider` | ProviderSpec | Yes | - | Provider-specific discovery configuration |
+| `targetPort` | int32 | No | - | Default port used when the discovered target does not provide a port |
+| `targetProfile` | string | Yes | - | Reference to `TargetProfile` applied to discovered targets |
+| `targetLabels` | map[string]string | No | - | Labels added to all discovered targets |
+
+### ProviderSpec
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `http` | HTTPConfig | No | HTTP provider configuration |
### HTTPConfig
+| Field | Type | Required | Default | Description |
+|-------|------|----------|---------|-------------|
+| `url` | string | No | - | HTTP endpoint used to pull targets. Required unless push is enabled |
+| `method` | string | No | GET | HTTP request method |
+| `headers` | map[string]string | No | - | HTTP headers to include in requests |
+| `body` | string | No | - | Request body for POST requests |
+| `authentication` | AuthenticationSpec | No | - | Authentication configuration for the HTTP endpoint |
+| `interval` | duration | No | 6h | Polling interval used to refresh targets |
+| `timeout` | duration | No | 10s | Timeout for HTTP requests |
+| `tls` | ClientTLSConfig | No | - | Client TLS configuration for HTTPS endpoints |
+| `pagination` | PaginationSpec | No | - | Pagination settings for parsing responses |
+| `mapping` | ResponseMappingSpec | No | - | Response mapping configuration for JSON responses |
+| `push` | PushSpec | No | - | Push-based update configuration |
+
+### ClientTLSConfig
+
+| Field | Type | Required | Default | Description |
+|-------|------|----------|---------|-------------|
+| `insecureSkipVerify` | bool | No | false | Skip verification of the server certificate |
+| `caBundleRef` | ConfigMapKeySelector | No | - | Reference to a ConfigMap containing a PEM CA bundle |
+
+### AuthenticationSpec
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `basic` | BasicAuthSpec | No | Basic authentication configuration |
+| `token` | TokenAuthSpec | No | Token authentication configuration |
+
+### BasicAuthSpec
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `credentialsSecretRef` | SecretKeySelector | Yes | Reference to a Secret containing username/password keys |
+
+### TokenAuthSpec
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `scheme` | string | Yes | Token scheme, e.g. Bearer |
+| `tokenSecretRef` | SecretKeySelector | Yes | Reference to a Secret containing the token |
+
+### PaginationSpec
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `nextField` | string | No | JSON field containing the next page reference or pagination token |
+
+### ResponseMappingSpec
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `targetsField` | string | No | CEL expression selecting the list of targets from the response |
+| `name` | string | No | CEL expression for the target name |
+| `address` | string | No | CEL expression for the target address |
+| `port` | string | No | CEL expression for the target port |
+| `labels` | string | No | CEL expression returning a map of labels |
+| `targetProfile` | string | No | CEL expression for the target profile |
+
+### PushSpec
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `enabled` | bool | No | Enable push updates |
+| `auth` | PushAuthSpec | No | Push authentication configuration |
+
+### PushAuthSpec
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `bearer` | PushBearerAuthSpec | No | Bearer token authentication configuration |
+| `signature` | PushSignatureAuthSpec | No | Signature authentication configuration |
+
+### PushBearerAuthSpec
+
| Field | Type | Required | Description |
|-------|------|----------|-------------|
-| `url` | string | Yes | URL of the HTTP endpoint |
+| `tokenSecretRef` | SecretKeySelector | Yes | Reference to a Secret containing the bearer token |
-### ConsulConfig
+### PushSignatureAuthSpec
| Field | Type | Required | Description |
|-------|------|----------|-------------|
-| `url` | string | Yes | Consul server URL |
+| `secretRef` | SecretKeySelector | Yes | Reference to a Secret used to verify request signatures |
+| `header` | string | Yes | Header containing the signature |
+| `algorithm` | string | No | Signature algorithm |
### TargetSourceStatus
| Field | Type | Description |
|-------|------|-------------|
| `status` | string | Sync status (Synced, Error, Pending) |
+| `observedGeneration` | int64 | Observed generation of the spec |
| `targetsCount` | int32 | Number of discovered targets |
| `lastSync` | Time | Last successful sync timestamp |
diff --git a/docs/content/docs/user-guide/targetsource.md b/docs/content/docs/user-guide/targetsource.md
deleted file mode 100644
index d1fee0c4..00000000
--- a/docs/content/docs/user-guide/targetsource.md
+++ /dev/null
@@ -1,242 +0,0 @@
----
-title: "TargetSource"
-linkTitle: "TargetSource"
-weight: 4
-description: >
- Dynamic target discovery from external sources
----
-
-The `TargetSource` resource enables dynamic discovery of network devices from external sources. The operator automatically creates, updates, and deletes `Target` resources based on discovered devices.
-
-## Discovery Sources
-
-TargetSource supports multiple discovery backends:
-
-| Source | Description |
-|--------|-------------|
-| `http` | Fetch targets from an HTTP endpoint |
-| `consul` | Discover targets from Consul service registry |
-| `configMap` | Read targets from a Kubernetes ConfigMap |
-| `podSelector` | Create targets from Kubernetes Pods |
-| `serviceSelector` | Create targets from Kubernetes Services |
-
-## HTTP Discovery
-
-Discover targets from an HTTP endpoint that returns a JSON list of targets:
-
-```yaml
-apiVersion: operator.gnmic.dev/v1alpha1
-kind: TargetSource
-metadata:
- name: http-discovery
-spec:
- http:
- url: http://inventory-service:8080/targets
- labels:
- source: inventory
-```
-
-The HTTP endpoint should return a JSON array of target objects.
-
-## Consul Discovery
-
-Discover targets from Consul service registry:
-
-```yaml
-apiVersion: operator.gnmic.dev/v1alpha1
-kind: TargetSource
-metadata:
- name: consul-discovery
-spec:
- consul:
- url: http://consul:8500
- labels:
- source: consul
- datacenter: dc1
-```
-
-## ConfigMap Discovery
-
-Read targets from a Kubernetes ConfigMap:
-
-```yaml
-apiVersion: operator.gnmic.dev/v1alpha1
-kind: TargetSource
-metadata:
- name: configmap-targets
-spec:
- configMap: network-devices
- labels:
- source: configmap
-```
-
-The ConfigMap should contain target definitions in a structured format.
-
-## Kubernetes Pod Discovery
-
-Create targets from Kubernetes Pods matching a label selector:
-
-```yaml
-apiVersion: operator.gnmic.dev/v1alpha1
-kind: TargetSource
-metadata:
- name: pod-discovery
-spec:
- podSelector:
- matchLabels:
- app: network-simulator
- gnmi: enabled
- labels:
- source: kubernetes
- type: simulator
-```
-
-This is useful for:
-- Containerized network simulators
-- Virtual network functions (VNFs)
-- Development/testing environments
-
-## Kubernetes Service Discovery
-
-Create targets from Kubernetes Services matching a label selector:
-
-```yaml
-apiVersion: operator.gnmic.dev/v1alpha1
-kind: TargetSource
-metadata:
- name: service-discovery
-spec:
- serviceSelector:
- matchLabels:
- protocol: gnmi
- labels:
- source: kubernetes
-```
-
-## Label Inheritance
-
-Labels defined in the `TargetSource.spec.labels` field are applied to all discovered targets:
-
-```yaml
-apiVersion: operator.gnmic.dev/v1alpha1
-kind: TargetSource
-metadata:
- name: datacenter-a
-spec:
- consul:
- url: http://consul-dc-a:8500
- labels:
- datacenter: dc-a
- environment: production
- source: consul
-```
-
-All targets discovered from this source will have:
-- `datacenter: dc-a`
-- `environment: production`
-- `source: consul`
-
-This enables using label selectors in Pipelines to select targets by their discovery source.
-
-## Status
-
-The TargetSource status shows discovery state:
-
-```yaml
-status:
- status: Synced
- targetsCount: 42
- lastSync: "2024-01-15T10:30:00Z"
-```
-
-| Field | Description |
-|-------|-------------|
-| `status` | Current sync status (Synced, Error, Pending) |
-| `targetsCount` | Number of targets discovered |
-| `lastSync` | Timestamp of last successful sync |
-
-## Example: Multi-Source Discovery
-
-Combine multiple TargetSources for different environments:
-
-```yaml
-# Production devices from Consul
-apiVersion: operator.gnmic.dev/v1alpha1
-kind: TargetSource
-metadata:
- name: prod-consul
-spec:
- consul:
- url: http://consul-prod:8500
- labels:
- environment: production
- source: consul
----
-# Lab devices from ConfigMap
-apiVersion: operator.gnmic.dev/v1alpha1
-kind: TargetSource
-metadata:
- name: lab-devices
-spec:
- configMap: lab-network-devices
- labels:
- environment: lab
- source: configmap
----
-# Simulator pods
-apiVersion: operator.gnmic.dev/v1alpha1
-kind: TargetSource
-metadata:
- name: simulators
-spec:
- podSelector:
- matchLabels:
- app: srlinux
- labels:
- environment: dev
- source: kubernetes
-```
-
-Then use label selectors in your Pipeline:
-
-```yaml
-apiVersion: operator.gnmic.dev/v1alpha1
-kind: Pipeline
-metadata:
- name: production-telemetry
-spec:
- clusterRef: prod-cluster
- enabled: true
- targetSelectors:
- - matchLabels:
- environment: production
- # ... subscriptions, outputs
-```
-
-## Lifecycle
-
-### Target Creation
-
-When a TargetSource discovers a new device:
-1. A new `Target` resource is created
-2. Labels from `spec.labels` are applied
-3. Owner reference is set to the TargetSource
-
-### Target Updates
-
-When a discovered device's properties change:
-1. The corresponding `Target` is updated
-2. Clusters using that target are reconciled
-
-### Target Deletion
-
-When a device is no longer discovered:
-1. The `Target` resource is deleted
-2. Clusters stop collecting from that target
-
-### TargetSource Deletion
-
-When a TargetSource is deleted:
-1. All Targets owned by it are deleted (via owner references)
-2. Clusters are reconciled to remove those targets
-
diff --git a/docs/content/docs/user-guide/targetsource/_index.md b/docs/content/docs/user-guide/targetsource/_index.md
new file mode 100644
index 00000000..f9897a71
--- /dev/null
+++ b/docs/content/docs/user-guide/targetsource/_index.md
@@ -0,0 +1,214 @@
+---
+title: "TargetSource"
+linkTitle: "TargetSource"
+weight: 4
+description: >
+ Dynamic target discovery from external sources
+---
+
+The `TargetSource` resource enables dynamic discovery of network devices from external sources. The operator automatically creates, updates, and deletes `Target` resources based on discovered devices.
+
+## Basic Configuration
+
+```yaml
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetSource
+metadata:
+ name: targetsource-1
+spec:
+ provider:
+ # Configure one of the supported providers
+ targetPort: 57400
+ targetProfile: default
+ targetLabels:
+ source: inventory
+```
+
+The supported TargetSource providers are documented on the [TargetSource Provider](./providers/) page.
+
+## Spec Fields
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `provider` | object | Yes | Provider-specific discovery configuration. Exactly one provider must be configured |
+| `targetPort` | int32 | No | Default port used when the discovered target does not provide a port |
+| `targetProfile` | string | No | Reference to default `TargetProfile` applied to all discovered targets if no profile was discovered |
+| `targetLabels` | map[string]string | No | Labels added to all discovered targets |
+
+
+
+
+
+
+
+## Label Inheritance
+
+Each generated `Target` receives an ownership label identifying the originating `TargetSource`:
+```yaml
+operator.gnmic.dev/targetsource: targetsource-1
+```
+
+This label is automatically managed by the operator and is used to:
+- Identify targets owned by a specific `TargetSource`
+- Determine which targets should be updated or deleted during reconciliation
+
+The `operator.gnmic.dev/targetsource` label is reserved and always takes precedence over any provider-supplied labels.
+
+### TargetSource Labels
+
+Additional labels can be applied to all generated targets using `spec.targetLabels`:
+
+```yaml
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetSource
+metadata:
+ name: targetsource-1
+spec:
+ provider:
+ http:
+ url: http://targetsource-1:8080/targets
+ targetLabels:
+ datacenter: dc-a
+ environment: production
+```
+
+All targets discovered from this source will have:
+- `datacenter: dc-a`
+- `environment: production`
+
+This enables Pipelines to select targets using label selectors.
+
+### Labels from Discovery Providers
+
+Discovery providers may return additional labels for each target. These labels are applied directly to the generated `Target` resource.
+
+The `gnmic_operator_` label prefix is reserved for operator-specific behavior. Labels using this prefix are interpreted by the operator and are not applied directly to the generated `Target` resource.
+
+Supported operator labels:
+
+| Label | Description |
+|--------|-------------|
+| `gnmic_operator_target_profile` | Overrides the `TargetProfile` configured in the `TargetSource` |
+
+### Label Precedence
+
+If the same label key is defined in multiple places, labels are applied in the following order (highest precedence first):
+
+1. `TargetSource` ownership label (`operator.gnmic.dev/targetsource`)
+2. Labels from `TargetSource.spec.targetLabels`
+3. Labels returned by the discovery provider
+
+## Status
+
+The `TargetSource` status shows discovery state:
+
+```yaml
+status:
+ status: Synced
+ observedGeneration: 1
+ targetsCount: 42
+ lastSync: "2024-01-15T10:30:00Z"
+```
+
+| Field | Description |
+|-------|-------------|
+| `status` | Current sync status (Synced, Error, Pending) |
+| `observedGeneration` | Generation of the spec last processed by the controller |
+| `targetsCount` | Number of targets discovered |
+| `lastSync` | Timestamp of last successful sync |
+
+## Lifecycle
+
+### Target Creation
+
+When a `TargetSource` discovers a new device:
+
+1. A new `Target` resource is created
+2. The `TargetProfile` referenced in `spec.targetProfile` is assigned
+3. Labels from `spec.targetLabels` are applied
+4. The `TargetSource` is set as the owner reference
+
+### Target Updates
+
+On each discovery cycle, existing `Target` resources are reconciled with the latest discovered state:
+
+1. The corresponding `Target` resource is updated and overwritten
+2. Clusters consuming the target are reconciled automatically
+
+> Manual changes to `Target` resources managed by a `TargetSource` are overwritten on every reconciliation cycle.
+
+### Target Deletion
+
+When a device is no longer returned by the discovery provider:
+
+1. The corresponding `Target` resource is deleted
+2. Clusters automatically stop using the target
+
+### TargetSource Deletion
+
+When a `TargetSource` is deleted:
+
+1. All `Target` resources owned by it are deleted via owner references
+2. Clusters are reconciled and remove the deleted targets
+
diff --git a/docs/content/docs/user-guide/targetsource/providers/_index.md b/docs/content/docs/user-guide/targetsource/providers/_index.md
new file mode 100644
index 00000000..29284d7f
--- /dev/null
+++ b/docs/content/docs/user-guide/targetsource/providers/_index.md
@@ -0,0 +1,7 @@
+---
+title: "TargetSource Provider"
+linkTitle: "TargetSourceProvider"
+weight: 1
+description: >
+ Configuring TargetSource discovery providers
+---
diff --git a/docs/content/docs/user-guide/targetsource/providers/http.md b/docs/content/docs/user-guide/targetsource/providers/http.md
new file mode 100644
index 00000000..64b64262
--- /dev/null
+++ b/docs/content/docs/user-guide/targetsource/providers/http.md
@@ -0,0 +1,629 @@
+---
+title: "HTTP Provider"
+linkTitle: "HTTP"
+weight: 2
+description: >
+ The HTTP provider discovers targets from an HTTP endpoint returning JSON, or receives webhook-based updates when push mode is enabled.
+---
+
+## Basic Configuration
+
+```yaml
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetSource
+metadata:
+ name: targetsource-1
+spec:
+ provider:
+ http:
+ url: http://inventory-service:8080/targets
+ authentication:
+ token:
+ scheme: Bearer
+ tokenSecretRef:
+ name: inventory-token
+ key: token
+ # Enable push mode
+ push:
+ enabled: true
+ targetPort: 57400
+ targetProfile: default
+ targetLabels:
+ source: inventory
+```
+
+## HTTP Spec Fields
+
+| Field | Type | Required | Default | Description |
+|-------|------|----------|---------|-------------|
+| `url` | string | No | - | HTTP endpoint used to pull targets. Required unless `push.enabled` is enabled |
+| `method` | string | No | GET | HTTP method used for requests |
+| `headers` | map[string]string | No | - | HTTP headers to include in requests |
+| `body` | string | No | - | Request body for POST requests |
+| `authentication` | object | No | - | Authentication configuration for the HTTP endpoint |
+| `interval` | duration | No | 30m | Polling interval used to refresh targets |
+| `timeout` | duration | No | 30s | Timeout for HTTP requests |
+| `tls` | object | No | - | Client TLS configuration for HTTPS endpoints |
+| `pagination` | object | No | - | Pagination configuration for parsing HTTP responses |
+| `mapping` | object | No | - | Response mapping configuration for JSON responses |
+| `push` | object | No | - | Push-based update configuration |
+
+## Pull Mode
+
+The HTTP provider supports pull-based target discovery by periodically querying a remote HTTP endpoint that returns target data in JSON format.
+
+```yaml
+spec:
+ provider:
+ http:
+ url: http://inventory-service:8080/targets
+```
+
+In pull mode, the operator sends HTTP requests to the configured url at a fixed interval and updates targets based on the response. The `push.enabled` field is optional when pull mode is enabled, but can still be used for accepting incoming webhook notifications.
+
+*How Pull Mode Works*
+1. The operator sends an HTTP request to the configured url
+2. The response is parsed (either directly or via mapping)
+3. Targets are created, updated, or removed based on the returned data
+4. This process repeats according to the configured interval
+
+
+### Authentication
+
+The HTTP provider supports authenticated requests to the inventory endpoint.
+
+Exactly one authentication method can be configured.
+
+#### Basic Authentication
+
+Credentials are referenced from a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/).
+
+```yaml
+spec:
+ provider:
+ http:
+ url: https://inventory.example.com/targets
+ authentication:
+ basic:
+ credentialSecretRef:
+ name: inventory-credentials
+ key: username
+```
+
+#### Token Authentication
+
+Token authentication is configured using a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) reference.
+
+```yaml
+spec:
+ provider:
+ http:
+ url: https://inventory.example.com/targets
+ authentication:
+ token:
+ scheme: Bearer
+ tokenSecretRef:
+ name: inventory-token
+ key: token
+```
+
+### TLS
+
+TLS settings can be configured for HTTPS endpoints.
+
+```yaml
+spec:
+ provider:
+ http:
+ url: https://inventory.example.com/targets
+ tls:
+ insecureSkipVerify: false
+ caBundleRef:
+ name: inventory-ca
+ key: ca.crt
+```
+
+#### TLS Fields
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `insecureSkipVerify` | bool | No | Skip verification of the server certificate. Defaults to `false` |
+| `caBundleRef` | object | No | Reference to a [Kubernetes ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) containing a PEM-encoded CA bundle |
+
+### Pagination
+
+Pagination enables the operator to retrieve complete result sets from APIs that return data in multiple pages. The operator automatically follows pagination until no further pages are available.
+
+```yaml
+spec:
+ provider:
+ http:
+ url: https://inventory.example.com/devices
+ pagination:
+ nextField: "self.next"
+```
+
+#### Pagination Fields
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `nextField` | string | No | CEL expression used to extract the next page reference from the response |
+| `requestParam` | string | No | Query parameter used when the extracted value is a token |
+
+The `nextField` value may either contain:
+- A full URL for the next request
+- A pagination token appended as a query parameter to the original URL
+
+#### How Pagination Works
+
+The operator handles the following pagination patterns:
+
+##### 1. Link Header Pagination
+If the API provides a Link response header with `rel="next"`, the operator will automatically follow it.
+
+Example response header:
+```
+Link: ; rel="next"
+```
+
+Behavior:
+```
+Request 1: GET /devices?page=1
+Request 2: GET /devices?page=2
+Request 3: GET /devices?page=3
+...
+```
+
+##### 2. URL-Based Pagination
+If the response contains a full URL in the body (e.g. `"next": "https://..."`), it will be used directly.
+
+Example response:
+```json
+{
+ "devices": [...],
+ "next": "https://inventory.example.com/devices?offset=50"
+}
+```
+
+##### 3. Token-Based Pagination
+If the response contains a pagination token, the operator appends it as a query parameter.
+
+Example:
+```yaml
+pagination:
+ nextField: "self.next_token"
+ requestParam: "page_token"
+```
+
+Example:
+```
+GET /devices
+-> "next_token": "abc123"
+GET /devices?page_token=abc123
+```
+
+##### CEL-Based Extraction
+The nextField is evaluated as a CEL expression using:
+- `self` -> entire JSON response
+
+Example:
+```yaml
+pagination:
+ nextField: "self['@odata.nextLink']"
+```
+
+This allows extracting values from nested or special keys.
+
+### Response Processing
+
+The HTTP provider supports two response processing modes:
+
+- **Default response format**: The endpoint returns a JSON array of target objects.
+- **Response mapping**: Custom JSON structures are mapped to target fields using CEL expressions.
+
+If `mapping` is configured, the custom mapping rules are used. Otherwise, the response itself must be a JSON array.
+
+#### Default Response Format
+
+If `mapping` is not configured, the endpoint must return a JSON array of objects with the following structure:
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `name` | string | Yes | Name of the generated `Target` resource |
+| `address` | string | Yes | Device address (FQDN or IP address) |
+| `port` | int32 | No | Port used for gNMI connections. If omitted, `spec.targetPort` is used |
+| `labels` | map[string]string | No | Labels added to the generated `Target` resource |
+| `targetProfile` | string | No | Reference to a `TargetProfile`. If omitted, `spec.targetProfile` is used |
+
+Example response:
+
+```json
+[
+ {
+ "name": "spine1",
+ "address": "spine1.local",
+ "port": 57400,
+ "labels": {
+ "role": "spine"
+ },
+ "targetProfile": "spine-profile"
+ },
+ {
+ "name": "leaf1",
+ "address": "leaf1.local",
+ "port": 57400,
+ "labels": {
+ "role": "leaf"
+ }
+ },
+ {
+ "name": "leaf2",
+ "address": "leaf2.local",
+ "port": 57400,
+ "labels": {
+ "role": "leaf"
+ }
+ }
+]
+```
+
+#### Response Mapping via CEL
+
+When your inventory API's JSON structure differs from the default format, use CEL (Common Expression Language) mapping to extract target fields.
+
+```yaml
+spec:
+ provider:
+ http:
+ url: https://inventory.example.com/devices
+ mapping:
+ targetsField: "self.results"
+ name: "item.hostname"
+ address: "item.management.ip"
+ port: "item.gnmi.port"
+ targetProfile: "item.profile"
+ labels: "{'role': item.metadata.role, 'site': item.metadata.site}"
+```
+
+##### Understanding `targetsField`
+
+The `targetsField` expression tells the operator where to find the list of target objects in your API response. It's particularly important when your API wraps the target list in a data structure.
+
+**When to use `targetsField`:**
+- Your API returns `{"results": [...]}` -> use `"self.results"`
+- Your API returns `{"data": {"devices": [...]}}` -> use `"self.data.devices"`
+- Your API returns a plain array `[...]` -> omit `targetsField` (default behavior)
+
+**Example scenarios:**
+
+*Custom API response example 1:*
+```json
+{
+ "count": 42,
+ "next": "https://...",
+ "results": [
+ {"id": 1, "name": "device1", "primary_ip": "10.0.0.1"},
+ {"id": 2, "name": "device2", "primary_ip": "10.0.0.2"}
+ ]
+}
+```
+Usage: `targetsField: "self.results"`
+
+*Custom API response example 2:*
+```json
+{
+ "status": "success",
+ "data": {
+ "timestamp": "2024-01-01T00:00:00Z",
+ "devices": [
+ {"name": "router1", "mgmt_ip": "192.168.1.1"},
+ {"name": "router2", "mgmt_ip": "192.168.1.2"}
+ ]
+ }
+}
+```
+Usage: `targetsField: "self.data.devices"`
+
+##### Mapping Fields
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `targetsField` | string | No | CEL expression selecting the target list from the response. If omitted, assumes response is a direct JSON array |
+| `name` | string | No | CEL expression for the target name |
+| `address` | string | No | CEL expression for the target address |
+| `port` | string | No | CEL expression for the target port |
+| `labels` | string | No | CEL expression returning a map of labels |
+| `targetProfile` | string | No | CEL expression for the target profile |
+
+##### CEL Variables
+
+The mapping expressions support the following variables:
+- `item`: the current target object being processed
+- `self`: the complete unprocessed response from the HTTP endpoint
+
+#### Performance: CEL vs Direct Mapping
+
+Understanding the performance implications helps optimize your configurations:
+
+**Direct Mapping (No CEL)** - *Fastest*
+- Used when your API response matches the default structure exactly
+- No expression compilation or evaluation overhead
+- Suitable for high-frequency polling (e.g., every minute)
+- Example: API returns `[{"name": "...", "address": "..."}]`
+
+**CEL Mapping** - *Slight overhead*
+- CEL expressions are compiled once at startup (not per request)
+- Evaluation is performed per target object during each poll cycle
+- At high scale (10,000+ targets), consider the `interval` between polls
+
+**Best practices:**
+- Use direct mapping if your API already returns the correct structure
+- For large result sets, increase the interval
+- Combine CEL and direct mapping for efficiency (see hybrid mapping below)
+- Use CEL extensions (see reference table below) to reduce complexity and improve readability
+
+#### CEL Extensions
+
+The operator includes a set of standard CEL extensions from the official [CEL Go library](https://github.com/google/cel-go) to enable more advanced expressions.
+
+These [extensions](https://pkg.go.dev/github.com/google/cel-go/ext) expand CEL with additional capabilities commonly needed when transforming API responses:
+
+| Extension | Purpose |
+|----------|----------|
+| **Strings** | String manipulation such as splitting values, case conversion, and extracting parts of text (e.g. parsing hostnames or IPs) |
+| **Math** | Numeric operations and comparisons (e.g. calculations, min/max, type conversions) |
+| **Lists** | Working with arrays (e.g. indexing, filtering, joining values) |
+| **Sets** | Set-style operations such as membership checks and comparisons |
+| **Regex** | Pattern matching and validation using regular expressions |
+| **Bindings** | Defining intermediate variables to simplify complex expressions |
+
+**Examples:**
+
+```yaml
+mapping:
+ # Extract site from hostname
+ labels: |
+ {
+ 'site': item.name.split('-')[0]
+ }
+
+ # Conditional profile
+ targetProfile: "item.type == 'edge' ? 'edge' : 'core'"
+
+ # Pattern-based classification
+ labels: |
+ {
+ 'role': item.name.matches('^spine') ? 'spine' : 'leaf'
+ }
+```
+
+#### Combining CEL and Direct Mapping (Hybrid Approach)
+
+You don't need to map all fields with CEL. The operator supports mixing CEL expressions and direct field lookups for maximum efficiency:
+
+| Scenario | Behavior | Use Case |
+|----------|----------|----------|
+| `name`, `address` use CEL; others omitted | Extracts mapped fields via CEL; looks for `port`, `labels`, `targetProfile` directly in item JSON | Simple API where only some fields need transformation |
+| Only `labels` uses CEL | Other fields use direct mapping; labels constructed from CEL expression | API returns correct `name`, `address`, `port` but custom labels need extraction |
+| Only `address` uses CEL | Direct mapping for other fields; only address requires transformation | Most fields match API exactly except address requires CIDR parsing or format conversion |
+| All fields use CEL | Complete transformation via expressions | API structure completely different from expected format |
+
+This hybrid approach optimizes performance by only compiling and evaluating CEL where needed.
+
+**Example - Partial CEL mapping (only transform what needs transforming):**
+```yaml
+mapping:
+ # Use CEL only when you need to transform a field
+ name: "item.hostname"
+ address: "item.primary_ip4 != null ? item.primary_ip4.split('/')[0] : item.primary_ip6.split('/')[0]" # CEL: parse CIDR
+
+ # Fields that already exist should be omitted
+ # Port already exists as "port" field in item
+ # port: item.port <- omit this
+
+ # Use CEL for structured or derived values
+ labels: |
+ {
+ "site": item.site.name,
+ "role": item.device_role.name
+ }
+
+ # targetProfile can also be omitted if already present or not needed
+```
+
+In this example, only `address` and `labels` use CEL expressions; `name`, `port`, and `targetProfile` use direct field lookups for efficiency.
+
+#### Using YAML `|` for Complex CEL Expressions
+
+When writing more complex CEL expressions, it is recommended to use YAML’s pipe (`|`) literal block instead of inline strings.
+
+This is especially useful for expressions that span multiple lines or contain nested logic.
+
+**Labels example:**
+
+```yaml
+mapping:
+ labels: |
+ {
+ "site": item.site.name,
+ "rack": item.rack != null ? item.rack.name : "",
+ "role": item.role != null ? item.role : "unknown",
+ "tags": item.tags.join(',')
+ }
+```
+
+**Why use `|` instead of quoted strings:**
+- **Readability**: Multi-line expressions are easier to understand
+- **Maintainability**: Complex CEL expressions don't require escaping
+- **YAML best practice**: Literal blocks handle special characters naturally
+
+## Push Mode
+
+The HTTP provider supports webhook-based target updates via `spec.provider.http.push`.
+
+```yaml
+spec:
+ provider:
+ http:
+ push:
+ enabled: true
+```
+
+When `push.enabled` is true, the operator accepts incoming webhook notifications and can update targets without polling a remote endpoint. The `url` field is optional when push mode is enabled, but can still be used for polling and fallback behavior.
+
+See [Push mode](/docs/user-guide/targetsource/push/) for more details.
+
+## Recommended Production Settings
+
+When deploying HTTP TargetSource providers in production networks, follow these guidelines to ensure reliable and efficient target discovery:
+
+### Polling Configuration
+| Scenario | Setting | Rationale |
+|----------|---------|-----------|
+| **Small environment** (< 100 targets) | `interval: 5m` | Frequent updates without excessive load |
+| **Medium environment** (100-500 targets) | `interval: 10m` | Balance between freshness and API load |
+| **Large environment** (500-2000 targets) | `interval: 15m` | Reduce API polling overhead |
+| **Very large environment** (2000+ targets) | `interval: 30m` | Minimize impact on inventory system |
+| **High-frequency changes** | Use `push` mode with `interval` | Enables updates via push while periodic polling ensures completeness and consistency |
+
+**Timeout Configuration:**
+```yaml
+timeout: 30s # Allows for network latency
+```
+
+If timeouts consistently occur, increase `interval` instead of timeout (don't poll faster)
+
+### Authentication & Security
+
+**Always use TLS in production:**
+```yaml
+tls:
+ insecureSkipVerify: false # Never skip verification in production
+ caBundleRef:
+ name: inventory-ca-bundle
+ key: ca.crt
+```
+
+**For authenticated APIs:**
+- Store credentials in [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/)
+- Rotate credentials periodically
+- Use token-based auth when possible (simpler secret rotation)
+
+Example:
+```yaml
+authentication:
+ token:
+ scheme: Bearer
+ tokenSecretRef:
+ name: inventory-api-token
+ key: token
+```
+
+### Pagination & Large Result Sets
+
+**Configuration for APIs returning large result sets:**
+```yaml
+pagination:
+ nextField: next # Always configure pagination if your API supports it
+
+interval: 30m # Increase interval for large datasets (reduces cumulative API load)
+timeout: 60s # Increase only if individual requests are slow or responses are large
+```
+
+Pagination splits large datasets into multiple smaller HTTP requests. This improves reliability and reduces the likelihood of timeouts compared to fetching a single large response.
+
+**Optimization strategies:**
+- Request API filtering (if supported) to reduce result set size (e.g. ?limit=1000 or ?status=active)
+- If the API does not support pagination or filtering increase the timeout
+- Consider webhook push mode for frequently-changing inventories (if API supports it)
+
+### Mapping Optimization
+
+**Use hybrid CEL and direct mapping for performance:**
+```yaml
+# EFFICIENT - Only CEL-transform what needs it
+mapping:
+ #
+ name: "item.hostname" # CEL expression
+ # port: (OMITTED) # Direct: exists as "port" in item
+
+ # Only these need transformation -> use CEL
+ address: "item.primary_ip.split('/')[0]" # CEL: parse CIDR
+ labels: | # CEL: construct from nested fields
+ {'site': item.site.name}
+```
+
+**Avoid unnecessary CEL complexity:**
+```yaml
+# GOOD - Simple expressions
+mapping:
+ address: "item.management_ip"
+ port: "int(item.gnmi_port)"
+
+# AVOID - Nested ternary logic (hard to debug)
+mapping:
+ name: "item.has_override ? item.override_name : (item.hostname != '' ? item.hostname : 'default-' + string(item.id))"
+```
+
+**CEL expression best practices:**
+- Compile expressions once at startup (not per request), so complexity is paid only once
+- Use `ext.Bindings` for repeated expressions to avoid redundant evaluation
+- Test CEL expressions thoroughly; they're compiled but errors only appear during evaluation
+- Keep expressions under 200 characters for maintainability
+
+### Example Production Configuration
+
+```yaml
+apiVersion: gnmic.openconfig.net/v1alpha1
+kind: TargetSource
+metadata:
+ name: production-inventory
+spec:
+ provider:
+ http:
+ # Security
+ url: https://inventory.prod.example.com/api/dcim/devices/?limit=100
+ tls:
+ insecureSkipVerify: false
+ caBundleRef:
+ name: netbox-ca
+ key: ca.crt
+
+ # Authentication
+ authentication:
+ token:
+ scheme: Bearer
+ tokenSecretRef:
+ name: api-token
+ key: token
+
+ # Timing
+ interval: 15m # Balanced update frequency
+ timeout: 30s # Allow for network latency
+
+ # Pagination
+ pagination:
+ nextField: next
+
+ # Mapping for fields
+ mapping:
+ targetsField: "self.results"
+ #name: "item.name" -> already handled with fallback direct mapping
+ address: "item.primary_ip4 != null ? item.primary_ip4.split('/')[0] : item.primary_ip6.split('/')[0]"
+ port: "item.custom_fields.gnmi_port"
+ labels: "{\n 'site': item.site.name,\n 'role': item.device_role.name,\n 'status': item.status.value\n }"
+ targetProfile: "item.custom_fields.gnmi_profile"
+
+ # Global settings
+ targetPort: 9339
+ targetProfile: default-profile
+```
+
+This configuration ensures:
+
+- Secure HTTPS communication with certificate validation
+- API authentication with token-based credentials
+- Balanced polling interval for stable environments
+- Proper pagination handling for large device inventories
+- Rich label extraction from custom fields
+- Fallback to defaults when fields are missing
diff --git a/docs/content/docs/user-guide/targetsource/push.md b/docs/content/docs/user-guide/targetsource/push.md
new file mode 100644
index 00000000..ef8d77d3
--- /dev/null
+++ b/docs/content/docs/user-guide/targetsource/push.md
@@ -0,0 +1,136 @@
+---
+title: "Push Mode"
+linkTitle: "Push Mode"
+weight: 4
+description: >
+ Enables REST API interface that accepts real-time target updates.
+---
+
+## Basic configuration
+
+This CR enables the push interface with no authentication.
+
+```yaml
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetSource
+metadata:
+ name: targetsource-1
+spec:
+ provider:
+ http: # can be changed to a differnet TargetSourceProvider
+ push:
+ enabled: true
+```
+
+> `http` is currently the only TargetSourceProvider implemented, once others are added they can be used instead. Push mode is not coupled to a specific TargetSourceProvider implementation.
+
+---
+
+## Spec Fields
+
+| Field | Type | Required | Default | Description |
+|-------|------|----------|---------|-------------|
+| `push` | object | No | - | Push interface config |
+| `enabled` | bool | Yes | False | Whether the push interface is active |
+| `auth` | object | No | - | Bearer token authentication |
+| `signature` | object | No | - | HTTP body verification using HMAC |
+| `algorithm` | string | No | sha512 | Algorithm for signature verification(`sha256`or`sha512`) |
+
+---
+
+## Address
+
+The REST API endpoint runs on `http://cluster-address:8082/api/v1/:namespace/target-source/:name/applyTargets`.
+
+- `cluster-address`: Address of your cluster.
+- `:namespace`: Namespace the TargetSource is created in.
+- `:name`: Name of the TargetSource.
+
+See [Push mode with webhook](/docs/examples/netbox/webhook) for an example on how to configure the URL.
+
+### Cluster Address
+
+The cluster address depends on where the API is accessed from.
+
+- Use `http://:8082/` when accessing the API from outside the cluster.
+- Use `http://localhost:8082/` for local development (requires port-forwarding).
+- Use `gnmic-controller-manager-api.gnmic-system.svc.cluster.local` when NetBox (or another source of truth) runs in the same cluster.
+- If you use a reverse proxy, run `kubectl get service -n ` and use the returned service address and port in your proxy configuration.
+
+---
+
+## REST API
+
+Refer to the [REST API documentation](/docs/advanced/rest-api-documentation/) for the expected request schema and payload format. Any system or script capable of sending HTTP POST requests can integrate with this interface.
+
+---
+
+## Security
+
+The API supports Bearer Token authentication and X-Hook-Signature, both are optional and **turned off by default**. They are enabled by adding them to the specification. They can also be used in combination.
+
+An example configuration of both is documented in the [Netbox webhook](/docs/examples/netbox/webhook) example.
+
+---
+
+### Bearer Authentication
+
+Bearer authentication compares a token stored in Kubernetes with the one sent in the HTTP header. The Kubernetes secret is referenced as `tokenSecretRef`.
+
+```yaml
+apiVersion: operator.gnmic.dev/v1alpha1
+kind: TargetSource
+metadata:
+ name: targetsource-1
+spec:
+ provider:
+ http:
+ push:
+ enabled: true
+ auth:
+ bearer:
+ tokenSecretRef:
+ name: gnmic-api-auth # secret name
+ key: bearer-token # secret key
+```
+
+This requires the [creation](https://kubernetes.ltd/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic/) of an Opaque Kuberentes secret:
+
+- Must be in the same namespace the gNMIc controller runs in.
+- `name`: refers to the secret name
+- `key`: key of the secret
+- Example: `kubectl create secret generic gnmic-api-auth --from-literal=bearer-token=YOUR_SECRET_TOKEN`
+
+#### Authorization Header
+
+HTTP request must contain the Bearer token in the header in the format:
+
+```yaml
+Authorization: Bearer YOUR_SECRET_TOKEN
+```
+
+---
+
+### Signature
+
+Signature verification requires an Opaque Kubernetes secret that stores the shared key (see Bearer Authentication). For each request, the HMAC generated from the request body and shared key must be provided in the `X-Hook-Signature` header.
+
+```yaml
+spec:
+ provider:
+ http:
+ push:
+ enabled: true
+ auth:
+ signature:
+ algorithm: sha512
+ secretRef:
+ name: gnmic-signature
+ key: signature
+```
+
+---
+
+#### Reverse Proxy
+
+In order to have a secure setup, the HTTP post requests must be sent using TLS. The REST API interface does not support HTTPS, at least not directly. It is recommended to terminate the TLS connection at the reverse proxy and forward a HTTP request to the gNMIc Operator.