Add Gateway API (HTTPRoute) support to Console Helm chart and CRD#1329
Add Gateway API (HTTPRoute) support to Console Helm chart and CRD#1329david-yu wants to merge 1 commit into
Conversation
|
This PR is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
|
This PR is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
|
Moving to draft for now |
|
This PR is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
|
This PR is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
|
This PR is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
|
This PR is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
|
This PR is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
End-to-end test on EKS 1.34 in tandem with PR #1447TL;DR: Provisioned a fresh EKS 1.34 cluster, deployed both PRs on a single integration branch ( Stack
Console route — what got renderedEnabled the chart-side Console gateway integration with these values: console:
enabled: true
gateway:
enabled: true
parentRefs:
- name: redpanda-gateway
namespace: redpanda
sectionName: console-https
hostnames:
- console.example.com
path: /
pathType: PathPrefixThe chart rendered exactly one HTTPRoute ( k6 workload (mixed UI + API)
k6 summary
[*] The 50% non-2xx rate is the Per-bucket latency (k6 custom Trends)
The Tandem isolation checkBoth routes attached and served traffic concurrently throughout the run. Console k6 sustained 35.65 RPS with 0 5xx; concurrent 10 Mbps Kafka through the sibling TLSRoute did not degrade Console latency (p95 stayed at 7.99 ms for the full window). See the PR #1447 tandem comment for Kafka-side metrics from the same run. What this validates for PR #1329
Reproducible artifactsAll manifests + the k6 script are committed in
Raw artifacts (in test-harness repo)
🤖 Generated with Claude Code |
The earlier shape — registering only the chart's lightweight TLSRoute kind via AddKnownTypeWithName — left the v1alpha2 ListOptions / List kinds missing from the operator's scheme. controller-runtime's reflector calls List on every Watch, so every reconcile pass that included TLSRoute resources errored with: Failed to watch *redpanda.TLSRoute: no kind "ListOptions" is registered for version "gateway.networking.k8s.io/v1alpha2" Fix: switch the chart's Types() entry to the upstream `gatewayv1alpha2.TLSRoute` and register `gatewayv1alpha2.Install` on the operator's v2 scheme. The chart still renders the same wire bytes via the lightweight struct returned by TLSRoutes(); only the type the controller cache binds to changes. Surfaced during the tandem PR #1329 + #1447 e2e on EKS 1.34. See: #1447 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
d9c1cc4 to
2298c12
Compare
… and CRD Closes #1308. Squashed rebase of the original 14-commit branch onto current main; drops stale golden-file drift and a review-cycle import-reordering revert that had accumulated through iteration. Chart side (`charts/console/`): - New `gateway` values block alongside `ingress`. Mutual exclusion enforced in render: enabling both fails with a clear error. - `gateway.go` renders a single HTTPRoute attached to the supplied parent Gateway(s) via `parentRefs` + optional `sectionName`. - Notes template shows the Gateway URL when gateway is enabled (and the Ingress URL when not). - Tests: mutual-exclusion, gateway-only, gateway→ingress switch, gateway removal scenarios. Operator side (`operator/`): - `Console` CRD gains a `spec.gateway` field with the same shape as the chart's values; goverter conversion auto-generated. - V2 scheme registers `gatewayv1` so the Console reconciler can watch HTTPRoutes. - RBAC adds `gateway.networking.k8s.io/httproutes` perms. - Console controller's `SetupWithManager` skips the HTTPRoute watch if the Gateway API CRDs aren't installed in the cluster (graceful degradation; same pattern used for ServiceMonitor). Bumps `sigs.k8s.io/gateway-api` to v1.5.1 (workspace-wide). Validated end-to-end on EKS 1.34 in tandem with PR #1447's TLSRoute support; both routes coexisted cleanly on the same Envoy Gateway. See: #1329 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2298c12 to
daaca60
Compare
Closes #1361. Squashed rebase of the original 14-commit branch onto current main; consolidates the iterative CI/lint fixes and includes the v1alpha2 scheme registration fix surfaced during the tandem PR #1329 + #1447 e2e test on EKS 1.34. Design: - User brings their own Gateway (TLSRoute-capable, e.g. Envoy Gateway). The chart only manages TLSRoute + ClusterIP backend services. - Per-listener `gateway: true` opt-in enables gradual migration. Traditional NodePort/LoadBalancer listeners and TLSRoute listeners coexist on different ports. - SNI-based routing: each broker gets a unique hostname via `host` / `hostTemplate` per listener. - Bootstrap TLSRoute handles initial client connections; per-broker TLSRoutes handle direct broker connections after metadata discovery. Chart side (`charts/redpanda/`): - `external.gateway` block with `enabled`, `parentRefs`, `advertisedPort`. - Per-listener `gateway`, `host`, `hostTemplate` fields on `listeners.{kafka,http,admin,schemaRegistry}.external.*`. - `tlsroute.go` renders TLSRoute resources (bootstrap + per-broker) with proper SNI hostnames. - `service.gateway.go` renders ClusterIP backend services. - LoadBalancer / NodePort service rendering skips gateway-opted listeners so they coexist on different ports. - `secrets.go` constructs the per-listener gateway-aware advertised address. Operator side (`operator/`): - `Redpanda` CRD gains the `external.gateway` and per-listener fields; goverter conversion auto-generated. - V2 scheme registers `gatewayv1alpha2` (TLSRoute + TLSRouteList + ListOptions) so the controller-runtime cache can List/Watch the chart-rendered TLSRoute resources. The chart's lightweight TLSRoute struct stays for gotohelm rendering; the type the operator watches via `Types()` is the upstream `gatewayv1alpha2.TLSRoute`. - RBAC adds `gateway.networking.k8s.io/tlsroutes` perms. Validated end-to-end on EKS 1.34 with Envoy Gateway v1.2.6, TLS Passthrough mode, OMB at 10 Mbps + Console k6 in tandem: #1447 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… and CRD Closes #1308. Squashed rebase of the original 14-commit branch onto current main; drops stale golden-file drift and a review-cycle import-reordering revert that had accumulated through iteration. Chart side (`charts/console/`): - New `gateway` values block alongside `ingress`. Mutual exclusion enforced in render: enabling both fails with a clear error. - `gateway.go` renders a single HTTPRoute attached to the supplied parent Gateway(s) via `parentRefs` + optional `sectionName`. - Notes template shows the Gateway URL when gateway is enabled (and the Ingress URL when not). - Tests: mutual-exclusion, gateway-only, gateway→ingress switch, gateway removal scenarios. Operator side (`operator/`): - `Console` CRD gains a `spec.gateway` field with the same shape as the chart's values; goverter conversion auto-generated. - V2 scheme registers `gatewayv1` so the Console reconciler can watch HTTPRoutes. - RBAC adds `gateway.networking.k8s.io/httproutes` perms. - Console controller's `SetupWithManager` skips the HTTPRoute watch if the Gateway API CRDs aren't installed in the cluster (graceful degradation; same pattern used for ServiceMonitor). Bumps `sigs.k8s.io/gateway-api` to v1.5.1 (workspace-wide). Validated end-to-end on EKS 1.34 in tandem with PR #1447's TLSRoute support; both routes coexisted cleanly on the same Envoy Gateway. See: #1329 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
daaca60 to
54e6d17
Compare
Closes #1361. Squashed rebase of the original 14-commit branch onto current main; consolidates the iterative CI/lint fixes and includes the v1alpha2 scheme registration fix surfaced during the tandem PR #1329 + #1447 e2e test on EKS 1.34. Design: - User brings their own Gateway (TLSRoute-capable, e.g. Envoy Gateway). The chart only manages TLSRoute + ClusterIP backend services. - Per-listener `gateway: true` opt-in enables gradual migration. Traditional NodePort/LoadBalancer listeners and TLSRoute listeners coexist on different ports. - SNI-based routing: each broker gets a unique hostname via `host` / `hostTemplate` per listener. - Bootstrap TLSRoute handles initial client connections; per-broker TLSRoutes handle direct broker connections after metadata discovery. Chart side (`charts/redpanda/`): - `external.gateway` block with `enabled`, `parentRefs`, `advertisedPort`. - Per-listener `gateway`, `host`, `hostTemplate` fields on `listeners.{kafka,http,admin,schemaRegistry}.external.*`. - `tlsroute.go` renders TLSRoute resources (bootstrap + per-broker) with proper SNI hostnames. - `service.gateway.go` renders ClusterIP backend services. - LoadBalancer / NodePort service rendering skips gateway-opted listeners so they coexist on different ports. - `secrets.go` constructs the per-listener gateway-aware advertised address. Operator side (`operator/`): - `Redpanda` CRD gains the `external.gateway` and per-listener fields; goverter conversion auto-generated. - V2 scheme registers `gatewayv1alpha2` (TLSRoute + TLSRouteList + ListOptions) so the controller-runtime cache can List/Watch the chart-rendered TLSRoute resources. The chart's lightweight TLSRoute struct stays for gotohelm rendering; the type the operator watches via `Types()` is the upstream `gatewayv1alpha2.TLSRoute`. - RBAC adds `gateway.networking.k8s.io/tlsroutes` perms. Validated end-to-end on EKS 1.34 with Envoy Gateway v1.2.6, TLS Passthrough mode, OMB at 10 Mbps + Console k6 in tandem: #1447 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes #1361. Squashed rebase of the original 14-commit branch onto current main; consolidates the iterative CI/lint fixes and includes the v1alpha2 scheme registration fix surfaced during the tandem PR #1329 + #1447 e2e test on EKS 1.34. Design: - User brings their own Gateway (TLSRoute-capable, e.g. Envoy Gateway). The chart only manages TLSRoute + ClusterIP backend services. - Per-listener `gateway: true` opt-in enables gradual migration. Traditional NodePort/LoadBalancer listeners and TLSRoute listeners coexist on different ports. - SNI-based routing: each broker gets a unique hostname via `host` / `hostTemplate` per listener. - Bootstrap TLSRoute handles initial client connections; per-broker TLSRoutes handle direct broker connections after metadata discovery. Chart side (`charts/redpanda/`): - `external.gateway` block with `enabled`, `parentRefs`, `advertisedPort`. - Per-listener `gateway`, `host`, `hostTemplate` fields on `listeners.{kafka,http,admin,schemaRegistry}.external.*`. - `tlsroute.go` renders TLSRoute resources (bootstrap + per-broker) with proper SNI hostnames. - `service.gateway.go` renders ClusterIP backend services. - LoadBalancer / NodePort service rendering skips gateway-opted listeners so they coexist on different ports. - `secrets.go` constructs the per-listener gateway-aware advertised address. Operator side (`operator/`): - `Redpanda` CRD gains the `external.gateway` and per-listener fields; goverter conversion auto-generated. - V2 scheme registers `gatewayv1alpha2` (TLSRoute + TLSRouteList + ListOptions) so the controller-runtime cache can List/Watch the chart-rendered TLSRoute resources. The chart's lightweight TLSRoute struct stays for gotohelm rendering; the type the operator watches via `Types()` is the upstream `gatewayv1alpha2.TLSRoute`. - RBAC adds `gateway.networking.k8s.io/tlsroutes` perms. Validated end-to-end on EKS 1.34 with Envoy Gateway v1.2.6, TLS Passthrough mode, OMB at 10 Mbps + Console k6 in tandem: #1447 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… and CRD Closes #1308. Squashed rebase of the original 14-commit branch onto current main; drops stale golden-file drift and a review-cycle import-reordering revert that had accumulated through iteration. Chart side (`charts/console/`): - New `gateway` values block alongside `ingress`. Mutual exclusion enforced in render: enabling both fails with a clear error. - `gateway.go` renders a single HTTPRoute attached to the supplied parent Gateway(s) via `parentRefs` + optional `sectionName`. - Notes template shows the Gateway URL when gateway is enabled (and the Ingress URL when not). - Tests: mutual-exclusion, gateway-only, gateway→ingress switch, gateway removal scenarios. Operator side (`operator/`): - `Console` CRD gains a `spec.gateway` field with the same shape as the chart's values; goverter conversion auto-generated. - V2 scheme registers `gatewayv1` so the Console reconciler can watch HTTPRoutes. - RBAC adds `gateway.networking.k8s.io/httproutes` perms. - Console controller's `SetupWithManager` skips the HTTPRoute watch if the Gateway API CRDs aren't installed in the cluster (graceful degradation; same pattern used for ServiceMonitor). Bumps `sigs.k8s.io/gateway-api` to v1.5.1 (workspace-wide). Validated end-to-end on EKS 1.34 in tandem with PR #1447's TLSRoute support; both routes coexisted cleanly on the same Envoy Gateway. See: #1329 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
54e6d17 to
b6d0d18
Compare
… and CRD Closes #1308. Squashed rebase of the original 14-commit branch onto current main; drops stale golden-file drift and a review-cycle import-reordering revert that had accumulated through iteration. Chart side (`charts/console/`): - New `gateway` values block alongside `ingress`. Mutual exclusion enforced in render: enabling both fails with a clear error. - `gateway.go` renders a single HTTPRoute attached to the supplied parent Gateway(s) via `parentRefs` + optional `sectionName`. - Notes template shows the Gateway URL when gateway is enabled (and the Ingress URL when not). - Tests: mutual-exclusion, gateway-only, gateway→ingress switch, gateway removal scenarios. Operator side (`operator/`): - `Console` CRD gains a `spec.gateway` field with the same shape as the chart's values; goverter conversion auto-generated. - V2 scheme registers `gatewayv1` so the Console reconciler can watch HTTPRoutes. - RBAC adds `gateway.networking.k8s.io/httproutes` perms. - Console controller's `SetupWithManager` skips the HTTPRoute watch if the Gateway API CRDs aren't installed in the cluster (graceful degradation; same pattern used for ServiceMonitor). Bumps `sigs.k8s.io/gateway-api` to v1.5.1 (workspace-wide). Validated end-to-end on EKS 1.34 in tandem with PR #1447's TLSRoute support; both routes coexisted cleanly on the same Envoy Gateway. See: #1329 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
b6d0d18 to
42a0e50
Compare
… and CRD Closes #1308. Squashed rebase of the original 14-commit branch onto current main; drops stale golden-file drift and a review-cycle import-reordering revert that had accumulated through iteration. Chart side (`charts/console/`): - New `gateway` values block alongside `ingress`. Mutual exclusion enforced in render: enabling both fails with a clear error. - `gateway.go` renders a single HTTPRoute attached to the supplied parent Gateway(s) via `parentRefs` + optional `sectionName`. - Notes template shows the Gateway URL when gateway is enabled (and the Ingress URL when not). - Tests: mutual-exclusion, gateway-only, gateway→ingress switch, gateway removal scenarios. Operator side (`operator/`): - `Console` CRD gains a `spec.gateway` field with the same shape as the chart's values; goverter conversion auto-generated. - V2 scheme registers `gatewayv1` so the Console reconciler can watch HTTPRoutes. - RBAC adds `gateway.networking.k8s.io/httproutes` perms. - Console controller's `SetupWithManager` skips the HTTPRoute watch if the Gateway API CRDs aren't installed in the cluster (graceful degradation; same pattern used for ServiceMonitor). Bumps `sigs.k8s.io/gateway-api` to v1.5.1 (workspace-wide). Validated end-to-end on EKS 1.34 in tandem with PR #1447's TLSRoute support; both routes coexisted cleanly on the same Envoy Gateway. See: #1329 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
42a0e50 to
cdc9b28
Compare
V2 operator path validation — closing out the open caveatFollowing up on the prior tandem comment, which validated the chart-side HTTPRoute path and noted the v2 operator path was still open. Validated today (2026-05-19) on a fresh EKS 1.34 cluster: the v2 path works end-to-end. Stack
Console CR (this PR's V2 path)apiVersion: cluster.redpanda.com/v1alpha2
kind: Console
metadata:
name: my-console
namespace: redpanda
spec:
cluster:
clusterRef:
name: redpanda
gateway:
enabled: true
parentRefs:
- name: redpanda-gateway
sectionName: console-https
hostnames:
- console.example.com
path: /
pathType: PathPrefixThe Console reconciler from this PR rendered a single HTTPRoute owned by the Console CR: What this validates (vs the chart-side run)
Scheme-registration bug surfaced during V2 validation (fix on PR #1447)The chart-side path doesn't iterate Trying to register both types at the same v1alpha2 GVK panics with Fix (will be backported to PR #1447's squash commit): define a chart-local // charts/redpanda/tlsroute.go
type TLSRouteList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []TLSRoute `json:"items"`
}
// charts/redpanda/chart.go
func addTLSRouteToScheme(s *runtime.Scheme) {
gv := schema.GroupVersion{Group: "gateway.networking.k8s.io", Version: "v1alpha2"}
s.AddKnownTypeWithName(gv.WithKind("TLSRoute"), &TLSRoute{})
s.AddKnownTypeWithName(gv.WithKind("TLSRouteList"), &TLSRouteList{})
metav1.AddToGroupVersion(s, gv)
}Once this lands in PR #1447, V2 Redpanda CR reconciles run cleanly and the V2 Console CR path becomes usable on top of it. 10 Mbps OMB on the TLSRoute (concurrent with the Console HTTPRoute up and serving)Same workload shape as the prior tandem run: 1 × 12 partitions, 1 KiB records, 4 producers / 4 consumers, target 1,250 msg/s, 2 min warmup + 10 min steady-state. Kafka throughput (steady-state)
Publish latency — steady-state aggregate
Within run-to-run noise. The p99.9 ticked up ~4 ms vs the chart-direct run — plausibly the extra reconcile overhead from the V2 operator continuously managing the StatefulSet, but well within the bound; no errors or backlog. End-to-end latency — aggregate quantiles
Raw artifacts (in
|
Closes #1361. Squashed rebase of the original 14-commit branch onto current main; consolidates the iterative CI/lint fixes and includes the v1alpha2 scheme registration fix surfaced during the tandem PR #1329 + #1447 e2e test on EKS 1.34. Design: - User brings their own Gateway (TLSRoute-capable, e.g. Envoy Gateway). The chart only manages TLSRoute + ClusterIP backend services. - Per-listener `gateway: true` opt-in enables gradual migration. Traditional NodePort/LoadBalancer listeners and TLSRoute listeners coexist on different ports. - SNI-based routing: each broker gets a unique hostname via `host` / `hostTemplate` per listener. - Bootstrap TLSRoute handles initial client connections; per-broker TLSRoutes handle direct broker connections after metadata discovery. Chart side (`charts/redpanda/`): - `external.gateway` block with `enabled`, `parentRefs`, `advertisedPort`. - Per-listener `gateway`, `host`, `hostTemplate` fields on `listeners.{kafka,http,admin,schemaRegistry}.external.*`. - `tlsroute.go` renders TLSRoute resources (bootstrap + per-broker) with proper SNI hostnames. - `service.gateway.go` renders ClusterIP backend services. - LoadBalancer / NodePort service rendering skips gateway-opted listeners so they coexist on different ports. - `secrets.go` constructs the per-listener gateway-aware advertised address. Operator side (`operator/`): - `Redpanda` CRD gains the `external.gateway` and per-listener fields; goverter conversion auto-generated. - V2 scheme registers `gatewayv1alpha2` (TLSRoute + TLSRouteList + ListOptions) so the controller-runtime cache can List/Watch the chart-rendered TLSRoute resources. The chart's lightweight TLSRoute struct stays for gotohelm rendering; the type the operator watches via `Types()` is the upstream `gatewayv1alpha2.TLSRoute`. - RBAC adds `gateway.networking.k8s.io/tlsroutes` perms. Validated end-to-end on EKS 1.34 with Envoy Gateway v1.2.6, TLS Passthrough mode, OMB at 10 Mbps + Console k6 in tandem: #1447 (comment) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
Adds support for Kubernetes Gateway API
HTTPRouteresources to the Console chart and operator, allowing users to expose Console via Gateway API controllers (e.g. Envoy Gateway, Istio, Cilium) as an alternative to classic Ingress.Closes #1308
Supersedes #1309 (recreated from upstream branch for CI)
Original chart work by @w1ndhunter.
Prerequisites
Gateway API CRDs must be installed in your cluster before enabling this feature. The CRDs are not bundled with the Redpanda Helm chart or operator — they are maintained by the Kubernetes Gateway API project.
Install Gateway API CRDs
Install a Gateway controller
You also need a Gateway API-compatible controller running in your cluster. Common options include:
Create a
Gatewayresource that theHTTPRoutewill attach to:Changes
Console Helm Chart
gatewayvalues block alongside existingingressblockgateway.gowithHTTPRoute()rendering functionHTTPRouteadded toRender()manifest list andTypes()scheme registrationgatewayv1.Install(Scheme)for Gateway API type serializationnotes.goto show Gateway URLs (mutually exclusive with Ingress)NewRenderStaterejects both gateway and ingress enabled simultaneouslyOperator (Console CRD)
GatewayConfigandGatewayParentReferencetypes inconsole_types.goGatewayfield added toConsoleValuesandRedpandaConsolestructsgateway.networking.k8s.io/httproutespermissionsgatewayv1types in V2 schemesigs.k8s.io/gateway-apifrom v1.4.1 → v1.5.1Console Controller Tests — Gateway API CRD Loading⚠️
Bug Fix: InUseServerCerts external TLS cert mounting
InUseServerCertsincharts/redpanda/values.go(andoperator/multicluster/values.go) where acontinuestatement on the internal listener's TLS check would skip registering external sub-listener certs. External certs are now checked independently of internal TLS state.Usage Examples
Helm Chart — Gateway API
Helm Chart — Classic Ingress (unchanged)
Console CRD (Operator) — Gateway API
Console CRD (Operator) — Switching from Gateway to Ingress
To switch, remove the
gatewaystanza and addingress:The operator will remove the HTTPRoute and create an Ingress instead.
Test plan
go build ./...passes inoperator/andcharts/console/go test ./...passes incharts/console/— includes:TestIngressGatewayMutualExclusion— both enabled → errorTestGatewayRemoval/gateway_removed_from_config— no gateway stanza → no HTTPRouteTestGatewayRemoval/gateway_explicitly_disabled—enabled: false→ no HTTPRouteTestGatewayRemoval/switch_from_gateway_to_ingress— gateway→ingress produces Ingress, no HTTPRouteTestTemplategolden tests (gateway-templating case)TestController/gateway-enabled— basic gateway with parentRefsTestController/gateway-custom-path— custom path and multiple parentRefsgatewayfield with correct OpenAPI validationgateway.enabled: true→ HTTPRoute createdgateway.enabled: true→ HTTPRoute created🤖 Generated with Claude Code