diff --git a/docs/GatewayAPI.md b/docs/GatewayAPI.md new file mode 100644 index 000000000..6c8f224f2 --- /dev/null +++ b/docs/GatewayAPI.md @@ -0,0 +1,295 @@ +--- +title: Gateway API +parent: Deploy & Configure +nav_order: 5 +--- + +# Configuring Ingress Using Gateway API + +This guide explains how to expose Splunk services managed by Splunk Operator using the Kubernetes [Gateway API](https://gateway-api.sigs.k8s.io/). + +## Who this guide is for + +Use this document if you are: + +1. Running Splunk Enterprise on Kubernetes with Splunk Operator. +2. Receiving data from an external forwarding tier. +3. Planning to use Gateway API instead of legacy Ingress patterns. + +## What problem this solves + +Splunk data ingestion commonly needs two externally reachable data paths: + +1. `8088` for HEC (HTTP Event Collector). +2. `9997` for Splunk-to-Splunk (S2S) forwarding. + +Gateway API gives you a clear model to expose these paths safely: + +1. `GatewayClass` defines which controller implements Gateway API. +2. `Gateway` defines listeners and exposure policy. +3. `TCPRoute`/`HTTPRoute`/`TLSRoute` define backend routing behavior. +4. `ReferenceGrant` controls cross-namespace backend references. + +## Before you start + +### Required cluster state + +1. Splunk Operator deployment is healthy. +2. Splunk Enterprise custom resources are in `Ready` phase. +3. Splunk services exist for the target deployment. +4. A Gateway API implementation is installed. + +Check services in your Splunk namespace: + +```bash +kubectl get svc -n +``` + +For indexer ingest, verify service ports include: + +1. `8088` +2. `9997` + +### Required tools + +1. `kubectl` +2. `helm` +3. `curl` +4. `nc` (netcat) + +### Validation baseline for this guide + +This flow was validated on AKS on March 10, 2026 with: + +1. Splunk Operator `3.0.0` +2. Splunk Enterprise image `10.2.0` +3. Envoy Gateway `v1.7.0` + +Observed results: + +1. HEC health on `8088` returned `HEC is healthy`. +2. HEC event post on `8088` returned `{"text":"Success","code":0}`. +3. S2S TCP check on `9997` succeeded. +4. Cross-namespace policy checks produced expected `NotAllowedByListeners`, `RefNotPermitted`, and `UnsupportedValue` (multiple TCP routes on a single listener). + +## Quick implementation flow + +This guide follows four phases: + +1. Install Gateway API implementation. +2. Create Gateway resources for Splunk services. +3. Validate object status and endpoint reachability. +4. Validate HEC and S2S traffic. + +## Phase 1: Install Gateway API implementation + +Example: Envoy Gateway + +```bash +helm upgrade --install eg oci://docker.io/envoyproxy/gateway-helm \ + --version 1.7.0 \ + --namespace envoy-gateway-system \ + --create-namespace +``` + +Note: Gateway API resources are portable, but controller behavior can vary by implementation. Validate behavior with your selected controller. + +## Phase 2: Create same-namespace Gateway and routes (recommended starting point) + +Keep `Gateway` and routes in the same namespace as Splunk services first. This minimizes policy complexity. + +```yaml +apiVersion: gateway.networking.k8s.io/v1 +kind: GatewayClass +metadata: + name: eg +spec: + controllerName: gateway.envoyproxy.io/gatewayclass-controller +--- +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: splunk-edge-gw + namespace: +spec: + gatewayClassName: eg + listeners: + - name: hec-tcp + protocol: TCP + port: 8088 + allowedRoutes: + namespaces: + from: Same + - name: s2s-tcp + protocol: TCP + port: 9997 + allowedRoutes: + namespaces: + from: Same +--- +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: TCPRoute +metadata: + name: hec-tcp-route + namespace: +spec: + parentRefs: + - name: splunk-edge-gw + sectionName: hec-tcp + rules: + - backendRefs: + - name: splunk--indexer-service + port: 8088 +--- +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: TCPRoute +metadata: + name: s2s-tcp-route + namespace: +spec: + parentRefs: + - name: splunk-edge-gw + sectionName: s2s-tcp + rules: + - backendRefs: + - name: splunk--indexer-service + port: 9997 +``` + +## Phase 3: Validate Gateway and route status + +```bash +kubectl -n get gateway splunk-edge-gw -o yaml +kubectl -n get tcproute hec-tcp-route s2s-tcp-route -o yaml +``` + +Expected status: + +1. `Gateway`: `Accepted=True`, `Programmed=True` +2. `TCPRoute`: `Accepted=True`, `ResolvedRefs=True` + +## Phase 4: Validate external traffic paths + +Get gateway address: + +```bash +GW_IP=$(kubectl -n get gateway splunk-edge-gw -o jsonpath='{.status.addresses[0].value}') +``` + +### Validate HEC on `8088` + +```bash +HEC_TOKEN=$(kubectl -n get secret splunk--secret -o jsonpath='{.data.hec_token}' | base64 --decode) + +curl -sk "https://$GW_IP:8088/services/collector/health" +curl -sk "https://$GW_IP:8088/services/collector/event" \ + -H "Authorization: Splunk $HEC_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"event":"gateway-api-hec-test","source":"gateway-api","sourcetype":"gwtest"}' +``` + +Expected: + +1. Health endpoint returns `HEC is healthy`. +2. Event endpoint returns `{"text":"Success","code":0}`. + +### Validate S2S on `9997` + +```bash +nc -vz -w 5 "$GW_IP" 9997 +``` + +Expected: TCP connection succeeds. + +## External forwarding tier in a separate namespace + +If your forwarding tier route objects are created outside the Splunk namespace (for example in `fw-tier`), two controls must both allow traffic: + +1. Listener attachment policy: `allowedRoutes`. +2. Cross-namespace backend reference permission: `ReferenceGrant`. + +### Default secure-deny behavior + +Cross-namespace route attachments can fail with one or both of the following, depending on listener policy and backend reference policy: + +1. `Accepted=False` + `NotAllowedByListeners`. +2. `ResolvedRefs=False` + `RefNotPermitted`. + +### Explicit allow-list behavior + +To allow external forwarding tier routes safely: + +1. Update listener policy to permit route namespace attachment. +2. Add `ReferenceGrant` in the backend namespace. + +Example `ReferenceGrant`: + +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: allow-fw-tier-to-splunk-idxc + namespace: +spec: + from: + - group: gateway.networking.k8s.io + kind: TCPRoute + namespace: fw-tier + to: + - group: "" + kind: Service + name: splunk--indexer-service +``` + +Implementation note: some controllers effectively allow one active `TCPRoute` per listener. If multiple routes target the same listener, you may see `UnsupportedValue`. + +## Architecture and flow diagrams + +These diagrams support the operational flow above. + +### C4 context view + +![C4 Context - Gateway API with Splunk Operator](pictures/gateway-api/png/c4-context-gateway-api-splunk.png) + +PlantUML source: [`pictures/gateway-api/src/c4-context-gateway-api-splunk.puml`](pictures/gateway-api/src/c4-context-gateway-api-splunk.puml) + +### C4 container view + +![C4 Container - Gateway, Routes, and Splunk Services](pictures/gateway-api/png/c4-container-gateway-api-splunk.png) + +PlantUML source: [`pictures/gateway-api/src/c4-container-gateway-api-splunk.puml`](pictures/gateway-api/src/c4-container-gateway-api-splunk.puml) + +### Flowchart: HEC and S2S validation path + +![Flow - HEC and S2S test path](pictures/gateway-api/png/flow-hec-s2s-test-path.png) + +PlantUML source: [`pictures/gateway-api/src/flow-hec-s2s-test-path.puml`](pictures/gateway-api/src/flow-hec-s2s-test-path.puml) + +### Flowchart: cross-namespace policy behavior + +![Flow - Cross namespace route policy behavior](pictures/gateway-api/png/flow-cross-namespace-policy.png) + +PlantUML source: [`pictures/gateway-api/src/flow-cross-namespace-policy.puml`](pictures/gateway-api/src/flow-cross-namespace-policy.puml) + +Regenerate PNG files: + +```bash +plantuml -tpng -o ../png ./docs/pictures/gateway-api/src/*.puml +``` + +## Troubleshooting checklist + +1. Gateway has no external address: + - Check Gateway controller pods and generated LoadBalancer service. +2. Route rejected (`Accepted=False`): + - Verify `parentRefs`, `sectionName`, and listener `allowedRoutes` policy. +3. Route unresolved (`ResolvedRefs=False`): + - Verify backend service exists and cross-namespace access is allowed via `ReferenceGrant`. +4. HEC returns auth errors: + - Verify token from `splunk--secret` and endpoint path `/services/collector/event`. + +## See also + +1. [Ingress](Ingress.md) +2. [Custom Resources](CustomResources.md) +3. [Install](Install.md) diff --git a/docs/Ingress.md b/docs/Ingress.md index b7b695f1d..5ccb2992e 100644 --- a/docs/Ingress.md +++ b/docs/Ingress.md @@ -26,10 +26,10 @@ service/splunk-standalone-standalone-headless service/splunk-standalone-standalone-service ``` -We provide some examples below for configuring a few of the most popular Ingress controllers: [Istio](https://istio.io/) , [Nginx-inc](https://docs.nginx.com/nginx-ingress-controller/overview/) and [Ingress Nginx](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration). We hope these will serve as a useful starting point to configuring ingress in your environment. +We provide some examples below for configuring supported ingress approaches: [Gateway API](GatewayAPI.md) and [Istio](https://istio.io/). We hope these will serve as a useful starting point to configuring ingress in your environment. +* [Configuring Ingress Using Gateway API](GatewayAPI.md) * [Configuring Ingress Using Istio](#Configuring-Ingress-Using-Istio) -* [Configuring Ingress Using NGINX](#Configuring-Ingress-Using-NGINX) * [Using Let's Encrypt to manage TLS certificates](#using-lets-encrypt-to-manage-tls-certificates) @@ -52,7 +52,7 @@ For Ingress we recommend using separate ports for encrypted and non-encrypted tr Indexer Discovery is not supported on a Kubernetes cluster. Instead, the Ingress controllers will be responsible to connect forwarders to peer nodes in Indexer clusters. #### Sticky Sessions -When configuring the ingress configuration, it is important to ensure that the session is sticky to the Splunk specific instance. This is required for Splunk to work properly since otherwise, a blank page might be experienced when trying to access the Splunk instances. The examples below show how to configure this using Istio and Nginx Ingress controllers. +When configuring the ingress configuration, it is important to ensure that the session is sticky to the Splunk specific instance. This is required for Splunk to work properly since otherwise, a blank page might be experienced when trying to access the Splunk instances. The examples below show how to configure this using Istio. ## Configuring Ingress Using Istio @@ -468,496 +468,13 @@ Follow [Istio Sticky Sessions Documentation](https://istio.io/latest/docs/refere Istio is a popular choice for its Service Mesh capabilities. However, Service Mesh for Splunk instances are only supported on Istio v1.8 and above, along with Kubernetes v1.19 and above. At the time of this documentation, neither Amazon AWS nor Google Cloud have updated their stack to these versions. -## Configuring Ingress Using NGINX - -**NOTE**: There are at least 3 flavors of the Nginx Ingress controller. - -- Kubernetes Ingress Nginx (open source) -- Nginx Ingress Open Source (F5's open source version) -- Nginx Ingress Plus (F5's paid version) - -[Nginx Comparison Chart](https://github.com/nginxinc/kubernetes-ingress/blob/main/docs/content/intro/nginx-ingress-controllers.md). - -It is important to confirm which Nginx Ingress controller you intended to implement, as they have *very* different annotations and configurations. For these examples, we are using the Kubernetes Ingress Nginx (option 1) and the Nginx Ingress Open Source (option 2). - -## Configuring Ingress Using Kubernetes Ingress Nginx - -For instructions on how to install and configure the NGINX Ingress Controller, see the -[NGINX Ingress Controller GitHub repository](https://github.com/kubernetes/ingress-nginx) and the [Installation Guide](https://kubernetes.github.io/ingress-nginx/deploy/). - -This Ingress Controller uses a ConfigMap to enable Ingress access to the cluster. Currently there is no support for TCP gateway termination. Requests for this feature are available in [Request 3087](https://github.com/kubernetes/ingress-nginx/issues/3087), and [Ticket 636](https://github.com/kubernetes/ingress-nginx/issues/636) but at this time only HTTPS is supported for gateway termination. - -For Splunk Forwarder communications over TCP, the only configuration available is End-to-End TLS termination. The details for creating and managing certificates, as well as the Forwarder and Indexer's configurations are the same as the example for Istio End-to-End TLS above. - -For all configurations below, we started with the standard yaml provided in the Installation Guide for AWS as a template: -[Ingress NGINX AWS Sample Deployment](https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.43.0/deploy/static/provider/aws/deploy.yaml). Then we will add or update those components based on each scenario. - -### Configuring Ingress for Splunk Web - -You can configure Nginx to provide direct access to Splunk Web. - -Example to create the Ingress configuration for a standalone: - -```yaml -apiVersion: networking.k8s.io/v1beta1 -kind: Ingress -metadata: - name: ingress-standalone - annotations: - # use the shared ingress-nginx - kubernetes.io/ingress.class: "nginx" - nginx.ingress.kubernetes.io/default-backend: splunk-standalone-standalone-service - nginx.ingress.kubernetes.io/proxy-body-size: "0" - nginx.ingress.kubernetes.io/proxy-read-timeout: "600" - nginx.ingress.kubernetes.io/proxy-send-timeout: "600" -spec: - rules: - - host: splunk.example.com - http: - paths: - - path: / - backend: - serviceName: splunk-standalone-standalone-service - servicePort: 8000 -``` - - - -Example to create an Ingress configuration for multiple hosts: - -```yaml -apiVersion: networking.k8s.io/v1beta1 -kind: Ingress -metadata: - name: ingress-standalone - annotations: - # use the shared ingress-nginx - kubernetes.io/ingress.class: "nginx" - nginx.ingress.kubernetes.io/default-backend: splunk-standalone-standalone-service - nginx.ingress.kubernetes.io/proxy-body-size: "0" - nginx.ingress.kubernetes.io/proxy-read-timeout: "600" - nginx.ingress.kubernetes.io/proxy-send-timeout: "600" -spec: - rules: - - host: splunk.example.com - http: - paths: - - path: / - backend: - serviceName: splunk-example-search-head-service - servicePort: 8000 - - path: /services/collector - backend: - serviceName: splunk-example-indexer-service - servicePort: 8088 - - host: deployer.splunk.example.com - http: - paths: - - backend: - serviceName: splunk-example-deployer-service - servicePort: 8000 - - host: cluster-manager.splunk.example.com - http: - paths: - - backend: - serviceName: splunk-example-cluster-manager-service - servicePort: 8000 -``` - -Example to create a TLS enabled Ingress configuration - -There are a few important configurations to be aware of in the TLS configuration: - -* The `nginx.ingress.kubernetes.io/backend-protocol:` annotation requires `"HTTPS"` when TLS is configured on the backend services. -* The secretName must reference a [valid TLS secret](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets). - -Note: This example assumes that https is enabled for Splunk Web. - -``` -apiVersion: networking.k8s.io/v1beta1 -kind: Ingress -metadata: - annotations: - kubernetes.io/ingress.class: "nginx" - nginx.ingress.kubernetes.io/affinity: "cookie" - nginx.ingress.kubernetes.io/affinity-mode: "persistent" - nginx.ingress.kubernetes.io/session-cookie-name: "route" - nginx.ingress.kubernetes.io/session-cookie-expires: "172800" - nginx.ingress.kubernetes.io/session-cookie-max-age: "172800" - nginx.ingress.kubernetes.io/client-body-buffer-size: 100M - nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" - nginx.ingress.kubernetes.io/session-cookie-samesite: "true" - nginx.ingress.kubernetes.io/session-cookie-path: "/en-US" - cert-manager.io/cluster-issuer: selfsigned - name: splunk-ingress - namespace: default -spec: - rules: - - host: shc.example.com - http: - paths: - - path: /en-US - pathType: Prefix - backend: - serviceName: splunk-shc-search-head-service - servicePort: 8000 - - host: hec.example.com - http: - paths: - - path: /services/collector - pathType: Prefix - backend: - serviceName: splunk-idc-indexer-service - servicePort: 8088 - tls: - - hosts: - - shc.example.com - - hec.example.com - secretName: operator-tls -``` - - -### Configuring Ingress NGINX for Splunk Forwarders with End-to-End TLS - - -Note: In this example we used port 9997 for non-encrypted communication, and 9998 for encrypted. - -Update the default Ingress NGINX configuration to add the ConfigMap and Service ports: - -1. Create a configMap to define the port-to-service routing - -```yaml -apiVersion: v1 -kind: ConfigMap -metadata: - name: tcp-services - namespace: ingress-nginx -data: - 9997: "default/splunk-standalone-standalone-service:9997" - 9998: "default/splunk-standalone-standalone-service:9998" -``` - -2. Add the two ports into the Service used to configure the Load Balancer - -```yaml -apiVersion: v1 -kind: Service -metadata: - annotations: - service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp - service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true' - service.beta.kubernetes.io/aws-load-balancer-type: nlb - labels: - helm.sh/chart: ingress-nginx-3.10.1 - app.kubernetes.io/name: ingress-nginx - app.kubernetes.io/instance: ingress-nginx - app.kubernetes.io/version: 0.41.2 - app.kubernetes.io/managed-by: Helm - app.kubernetes.io/component: controller - name: ingress-nginx-controller - namespace: ingress-nginx -spec: - type: LoadBalancer - externalTrafficPolicy: Local - ports: - - name: http - port: 80 - protocol: TCP - targetPort: http - - name: https - port: 443 - protocol: TCP - targetPort: https - - name: tcp-s2s - port: 9997 - protocol: TCP - targetPort: 9997 - - name: tls-s2s - port: 9998 - protocol: TCP - targetPort: 9998 -``` - -##### Documentation tested on Ingress Nginx v1.19.4 and Kubernetes v1.17 - -### Sticky Sessions -Follow [Ingress Nginx Sticky Sessions Documentation](https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/) to learn how to configure session stickiness for the Ingress Nginx controller. - -## Configuring Ingress Using NGINX Ingress Controller (Nginxinc) - -The Nginx Ingress Controller is an open source version of the F5 product. Please review their documentation below for more details. - -[NGINX Ingress Controller Github Repo](https://github.com/nginxinc/kubernetes-ingress) - -[NGINX Ingress Controller Docs Home](https://docs.nginx.com/nginx-ingress-controller/overview/) - -[NGINX Ingress Controller Annotations Page](https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/advanced-configuration-with-annotations/) - - -### Install the Nginx Helm Chart -We followed the product's Helm Chart installation guide. It requires a cluster with internet access. -[NGINX Ingress Controller Helm Installation]( https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-helm/) - -#### Setup Helm - -```shell -# clone the repo and check out the current production branch -$ git clone https://github.com/nginxinc/kubernetes-ingress/ -$ cd kubernetes-ingress/deployments/helm-chart -$ git checkout v1.9.0 - -# add the helm chart -$ helm repo add nginx-stable https://helm.nginx.com/stable -$ helm repo update - -# install custom resource definitions -$ kubectl create -f crds/ -``` - -#### Install Ingress -```shell -cd deployments/helm-chart - -# Edit and make changes to values.yaml as needed -helm install splunk-nginx nginx-stable/nginx-ingress - -#list the helms installed -helm list - -NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION -splunk-nginx default 5 2020-10-29 15:03:47.6 EDT deployed nginx-ingress-0.7.0 1.9.0 - -#if needed to update any configs for ingress, update the values.yaml and run upgrade -helm upgrade splunk-nginx nginx-stable/nginx-ingress -``` - -#### Configure Ingress services - -##### Configure Ingress for Splunk Web and HEC - -The following ingress example yaml configures Splunk Web as well as HEC as an operator installed service. HEC is exposed via ssl and Splunk Web is non-ssl. - -* TLS to the ingress controller is configured in the `tls:` section of the yaml and `secretName` references a valid [TLS secret](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets). -* For any backend service that has TLS enabled, a corresponding `nginx.org/ssl-services annotation` is required. - -Create Ingress -```yaml -apiVersion: extensions/v1beta1 -kind: Ingress -metadata: - annotations: - certmanager.k8s.io/cluster-issuer: letsencrypt-prod - nginx.org/client-body-buffer-size: 100M - nginx.org/client-max-body-size: "0" - nginx.org/server-snippets: | - client_body_buffer_size 100m; - nginx.org/ssl-services: splunk-standalone-standalone-headless - name: splunk-ingress - namespace: default -spec: - ingressClassName: nginx - rules: - - host: splunk.example.com - http: - paths: - - backend: - serviceName: splunk-standalone-standalone-service - servicePort: 8000 - path: /en-US - pathType: Prefix - - backend: - serviceName: splunk-standalone-standalone-headless - servicePort: 8088 - path: /services/collector - pathType: Prefix - - backend: - serviceName: splunk-standalone-standalone-headless - servicePort: 8089 - path: /.well-known - pathType: Prefix - tls: - - hosts: - - splunk.example.com - secretName: operator-tls -status: - loadBalancer: {} -``` - -##### Ingress Service for Splunk Forwarders -Enable the global configuration to setup a listener and transport server - -1. Create the GlobalConfiguration: - -```yaml -apiVersion: k8s.nginx.org/v1alpha1 -kind: GlobalConfiguration -metadata: - name: nginx-configuration - namespace: default -spec: - listeners: - - name: s2s-tcp - port: 30403 - protocol: TCP -apiVersion: k8s.nginx.org/v1alpha1 -kind: TransportServer -metadata: - name: s2s-tcp -spec: - listener: - name: s2s-tcp - protocol: TCP - upstreams: - - name: s2s-app - service: splunk-standalone-standalone-service - port: 9997 - action: - pass: s2s-app -``` - -2. Edit the service to establish a node-port for the port being setup as the listener: - 1. List the service: - -```yaml -kubectl get svc -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -splunk-nginx-nginx-ingress LoadBalancer 172.20.195.54 aa725344587a4443b97c614c6c78419c-1675645062.us-east-2.elb.amazonaws.com 80:31452/TCP,443:30402/TCP,30403:30403/TCP 7d1h -``` - -​ 2. Edit the service and add the Splunk Forwarder ingress port: - -``` -kubectl edit service splunk-nginx-nginx-ingress -``` - -Example Service: -```yaml -apiVersion: v1 -kind: Service -metadata: - annotations: - meta.helm.sh/release-name: splunk-nginx - meta.helm.sh/release-namespace: default - creationTimestamp: "2020-10-23T17:05:08Z" - finalizers: - - service.kubernetes.io/load-balancer-cleanup - labels: - app.kubernetes.io/instance: splunk-nginx - app.kubernetes.io/managed-by: Helm - app.kubernetes.io/name: splunk-nginx-nginx-ingress - helm.sh/chart: nginx-ingress-0.7.0 - name: splunk-nginx-nginx-ingress - namespace: default - resourceVersion: "3295579" - selfLink: /api/v1/namespaces/default/services/splunk-nginx-nginx-ingress - uid: a7253445-87a4-443b-97c6-14c6c78419c9 -spec: - clusterIP: 172.20.195.54 - externalTrafficPolicy: Local - healthCheckNodePort: 32739 - ports: - - name: http - nodePort: 31452 - port: 80 - protocol: TCP - targetPort: 80 - - name: https - nodePort: 30402 - port: 443 - protocol: TCP - targetPort: 443 - - name: s2s - nodePort: 30403 - port: 30403 - protocol: TCP - targetPort: 30403 - selector: - app: splunk-nginx-nginx-ingress - sessionAffinity: None - type: LoadBalancer -``` - -##### Documentation tested on Nginx Ingress Controller v1.9.0 and Kubernetes v1.18 - -### Sticky Sessions -Follow [Nginx Ingress Controller Sticky Sessions Documentation](https://github.com/nginx/kubernetes-ingress/tree/main/examples/ingress-resources/session-persistence) to learn how to configure session stickiness for Nginx Ingress Controller. - - -## Using Let's Encrypt to manage TLS certificates +## Using Let's Encrypt to manage TLS certificates If you are using [cert-manager](https://docs.cert-manager.io/en/latest/getting-started/) with [Let’s Encrypt](https://letsencrypt.org/) to manage your TLS certificates in Kubernetes, this example Ingress object can be used to enable secure (TLS) access to all Splunk components from outside of your Kubernetes cluster: -### Example configuration for NGINX - -```yaml -apiVersion: extensions/v1beta1 -kind: Ingress -metadata: - name: splunk-ingress - annotations: - kubernetes.io/ingress.class: nginx - nginx.ingress.kubernetes.io/affinity: cookie - certmanager.k8s.io/cluster-issuer: "letsencrypt-prod" -spec: - rules: - - host: splunk.example.com - http: - paths: - - path: / - backend: - serviceName: splunk-example-search-head-service - servicePort: 8000 - - path: /services/collector - backend: - serviceName: splunk-example-indexer-service - servicePort: 8088 - - host: deployer.splunk.example.com - http: - paths: - - backend: - serviceName: splunk-example-deployer-service - servicePort: 8000 - - host: cluster-manager.splunk.example.com - http: - paths: - - backend: - serviceName: splunk-example-cluster-manager-service - servicePort: 8000 - - host: license-manager.splunk.example.com - http: - paths: - - backend: - serviceName: splunk-example-license-manager-service - servicePort: 8000 - tls: - - hosts: - - splunk.example.com - - deployer.splunk.example.com - - cluster-manager.splunk.example.com - - license-manager.splunk.example.com - secretName: splunk.example.com-tls -``` - -The `certmanager.k8s.io/cluster-issuer` annotation is optional, and is used to automatically create and manage certificates for you. You can change it to match your Issuer. - -If you are not using cert-manager, you should remove this annotation and update the `tls` section as appropriate. If you are manually importing your certificates into separate secrets for each hostname, you can reference these by using multiple `tls` objects in your Ingress: - -```yaml -tls: - - hosts: - - splunk.example.com - secretName: splunk.example.com-tls - - hosts: - - deployer.splunk.example.com - secretName: deployer.splunk.example.com-tls - - hosts: - - cluster-manager.splunk.example.com - secretName: cluster-manager.splunk.example.com-tls -… -``` - ### Example configuration for Istio + If you are using [cert-manager](https://docs.cert-manager.io/en/latest/getting-started/) with [Let’s Encrypt](https://letsencrypt.org/) to manage your TLS certificates in Kubernetes: 1. Create the Certificate object and populate a `splunk-example-com-tls` secret in the `istio-system` namespace. For example: diff --git a/docs/pictures/gateway-api/png/c4-container-gateway-api-splunk.png b/docs/pictures/gateway-api/png/c4-container-gateway-api-splunk.png new file mode 100644 index 000000000..b50c1e174 Binary files /dev/null and b/docs/pictures/gateway-api/png/c4-container-gateway-api-splunk.png differ diff --git a/docs/pictures/gateway-api/png/c4-context-gateway-api-splunk.png b/docs/pictures/gateway-api/png/c4-context-gateway-api-splunk.png new file mode 100644 index 000000000..6930c3d80 Binary files /dev/null and b/docs/pictures/gateway-api/png/c4-context-gateway-api-splunk.png differ diff --git a/docs/pictures/gateway-api/png/flow-cross-namespace-policy.png b/docs/pictures/gateway-api/png/flow-cross-namespace-policy.png new file mode 100644 index 000000000..acb21aba0 Binary files /dev/null and b/docs/pictures/gateway-api/png/flow-cross-namespace-policy.png differ diff --git a/docs/pictures/gateway-api/png/flow-hec-s2s-test-path.png b/docs/pictures/gateway-api/png/flow-hec-s2s-test-path.png new file mode 100644 index 000000000..3f78ca837 Binary files /dev/null and b/docs/pictures/gateway-api/png/flow-hec-s2s-test-path.png differ diff --git a/docs/pictures/gateway-api/src/c4-container-gateway-api-splunk.puml b/docs/pictures/gateway-api/src/c4-container-gateway-api-splunk.puml new file mode 100644 index 000000000..27efb4786 --- /dev/null +++ b/docs/pictures/gateway-api/src/c4-container-gateway-api-splunk.puml @@ -0,0 +1,49 @@ +@startuml +skinparam backgroundColor #FFFDF7 +skinparam shadowing true +skinparam defaultFontName "Avenir Next" +skinparam roundCorner 14 +skinparam rectangle { + BorderColor #7C2D12 + BackgroundColor #FFEDD5 + FontColor #431407 +} +skinparam arrow { + Color #C2410C + FontColor #431407 + Thickness 2 +} + +package "AKS Namespace: " #FFFBEB { + rectangle "Gateway\nlisteners: 8088, 9997" as GW #FED7AA + rectangle "TCPRoute hec-tcp-route\n-> Service:8088" as HECROUTE #FDBA74 + rectangle "TCPRoute s2s-tcp-route\n-> Service:9997" as S2SROUTE #FDBA74 + rectangle "splunk--indexer-service\nports: 8088,9997" as IDXSVC #FCD34D + rectangle "Indexer Pods" as IDXPODS #FDE68A +} + +package "AKS Namespace: fw-tier (optional)" #FEF3C7 { + rectangle "TCPRoute ext-fw-s2s-route" as EXTROUTE #FDE68A +} + +rectangle "Gateway API Policy Layer" as POLICY #DCFCE7 +rectangle "ReferenceGrant\n(namespace: )" as RG #BBF7D0 + +GW --> HECROUTE : listener attach +GW --> S2SROUTE : listener attach +HECROUTE --> IDXSVC : backendRef 8088 +S2SROUTE --> IDXSVC : backendRef 9997 +IDXSVC --> IDXPODS : kube-proxy/service routing + +EXTROUTE --> GW : parentRef cross-namespace +EXTROUTE --> IDXSVC : backendRef cross-namespace +POLICY --> EXTROUTE : allowedRoutes check +RG --> EXTROUTE : permits backendRef across namespaces + +note bottom of POLICY +Without policy allow: +- NotAllowedByListeners +- RefNotPermitted +end note + +@enduml diff --git a/docs/pictures/gateway-api/src/c4-context-gateway-api-splunk.puml b/docs/pictures/gateway-api/src/c4-context-gateway-api-splunk.puml new file mode 100644 index 000000000..666c9367d --- /dev/null +++ b/docs/pictures/gateway-api/src/c4-context-gateway-api-splunk.puml @@ -0,0 +1,49 @@ +@startuml +' Fancy C4 context-style diagram (custom colors) +skinparam backgroundColor #F8FAFF +skinparam shadowing true +skinparam defaultFontName "Avenir Next" +skinparam roundCorner 18 +skinparam rectangle { + BorderColor #1E3A8A + BackgroundColor #DBEAFE + FontColor #0F172A +} +skinparam package { + BorderColor #0E7490 + BackgroundColor #CFFAFE + FontColor #042F2E +} +skinparam arrow { + Color #0F766E + FontColor #0F172A + Thickness 2 +} + +rectangle "External Forwarding Tier\n(UF/HF, Edge Collectors)" as FORWARDERS #FDE68A +rectangle "Azure AKS Cluster" as AKS #E0E7FF { + rectangle "Gateway API\n(GatewayClass/Gateway/Routes)" as GAPI #BFDBFE + rectangle "Envoy Gateway Data Plane" as ENVOY #93C5FD + rectangle "Splunk Operator (SOK)" as SOK #A7F3D0 + rectangle "Splunk Enterprise on K8s\n(C3: CM/Indexer/SHC or Standalone)" as SPLUNK #86EFAC + rectangle "Kubernetes Services\n(8088 HEC, 9997 S2S)" as SVC #BBF7D0 +} +rectangle "Azure Key Vault" as AKV #FECACA +rectangle "External Secrets Operator (ESO)" as ESO #FCA5A5 + +FORWARDERS --> GAPI : Internet or private edge\nHEC(8088)/S2S(9997) +GAPI --> ENVOY : Gateway API config translation +ENVOY --> SVC : L4/L7 routing +SVC --> SPLUNK : Splunk ingest and web/API traffic +AKV --> ESO : Secret source of truth +ESO --> SOK : K8s namespace secret sync +SOK --> SPLUNK : Versioned CR secrets mounted to pods + +note right of GAPI +Gateway API enforces: +- listener policy +- namespace attachment rules +- route/backend references +end note + +@enduml diff --git a/docs/pictures/gateway-api/src/flow-cross-namespace-policy.puml b/docs/pictures/gateway-api/src/flow-cross-namespace-policy.puml new file mode 100644 index 000000000..b2194f450 --- /dev/null +++ b/docs/pictures/gateway-api/src/flow-cross-namespace-policy.puml @@ -0,0 +1,30 @@ +@startuml +skinparam backgroundColor #F0FDF4 +skinparam defaultFontName "Avenir Next" +skinparam shadowing true +skinparam ArrowColor #15803D +skinparam activity { + BackgroundColor #DCFCE7 + BorderColor #166534 + FontColor #052E16 +} +start +:Create TCPRoute in fw-tier namespace; +:Attach to Gateway in splunk namespace; +if (allowedRoutes from Same?) then (yes) + :Route status Accepted=False\nreason=NotAllowedByListeners; +endif +if (backendRef cross-namespace\nwithout ReferenceGrant?) then (yes) + :Route status ResolvedRefs=False\nreason=RefNotPermitted; +endif +:Update listener policy\nallowedRoutes.from=All; +:Add ReferenceGrant in backend namespace; +if (single active TCPRoute\nper listener?) then (yes) + :Route Accepted=True\nResolvedRefs=True; + :Traffic allowed; +else (no) + :UnsupportedValue (controller-specific); + :Consolidate to one route per listener; +endif +stop +@enduml diff --git a/docs/pictures/gateway-api/src/flow-hec-s2s-test-path.puml b/docs/pictures/gateway-api/src/flow-hec-s2s-test-path.puml new file mode 100644 index 000000000..6601b517c --- /dev/null +++ b/docs/pictures/gateway-api/src/flow-hec-s2s-test-path.puml @@ -0,0 +1,35 @@ +@startuml +skinparam backgroundColor #F5F3FF +skinparam defaultFontName "Avenir Next" +skinparam shadowing true +skinparam ArrowColor #6D28D9 +skinparam activity { + BackgroundColor #DDD6FE + BorderColor #5B21B6 + FontColor #2E1065 +} +start +:Deploy GatewayClass + Gateway; +:Create TCPRoute for 8088 (HEC); +:Create TCPRoute for 9997 (S2S); +:Check Gateway Accepted=True\nProgrammed=True; +if (HEC test over 8088?) then (yes) + :curl /services/collector/health; + :curl /services/collector/event; + if (code==0?) then (yes) + :HEC path validated; + else (no) + :Inspect token/TLS/backend service; + endif +endif +if (S2S test over 9997?) then (yes) + :nc -vz 9997; + if (tcp connect ok?) then (yes) + :S2S transport path validated; + else (no) + :Inspect listener/route/backend port; + endif +endif +:Document pass/fail evidence; +stop +@enduml