Expose SPIFFE client-auth registration through the operator CRDs - #6500
Expose SPIFFE client-auth registration through the operator CRDs#6500jhrozek wants to merge 5 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## spiffe-integration-split3-5 #6500 +/- ##
===============================================================
+ Coverage 78.36% 78.40% +0.04%
===============================================================
Files 776 776
Lines 76175 76253 +78
===============================================================
+ Hits 59693 59788 +95
+ Misses 16477 16460 -17
Partials 5 5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
JAORMX
left a comment
There was a problem hiding this comment.
The CRD conversion and generated artifacts are largely aligned, but the panel found material validation/lifecycle gaps:
-
principalPatternrejects~in path segments (cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go:731-738), although it is valid in SPIFFE URI path segments. This makes the served CRD schema narrower than the authoritative runtime parser. Please accept the full permitted segment grammar (including~) and add concrete/wildcard admission coverage. -
Context-independent SPIFFE errors can still lead
MCPExternalAuthConfigto reportValid=True: bundle endpoint URLs are not structurally validated and overlapping principal policies are only rejected later by runtime validation (cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go:647-653, 890-891, 2366-2373). Validate these at admission or in the config controller; defer only checks that genuinely require workload-derived allowlists. -
The multi-issuer validator owns background
jwk.Cacheworkers but neither it nor the auth server closes them; constructor error paths also leave earlier caches running (pkg/authserver/server/tokenexchange/multi_issuer_validator.go:415-445,549;pkg/authserver/server_impl.go:396-400). Add lifecycle cleanup on server close and on partial construction failure, with tests.
The red Go Vulnerability Check is inherited from the unchanged dependency in the stacked base and appears unrelated.
5834d9c to
6dbb23e
Compare
b758757 to
f57d37e
Compare
6dbb23e to
e038ceb
Compare
f57d37e to
c5e6f6e
Compare
JAORMX
left a comment
There was a problem hiding this comment.
Re-review at c5e6f6e6654575c784b132321caf72c543d9dd3a: none of the three prior blockers are resolved, and the same inline-configuration parity gap exists here.
- CRD
principalPatternstill rejects valid SPIFFE~path segments (cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go:731-738). - Malformed bundle endpoints and overlapping principal patterns still bypass validation on the owning config CRD and can be reported
Valid=True(mcpexternalauthconfig_types.go:647-653,883-893;controllers/mcpexternalauthconfig_controller.go:106-176). - JWK cache workers still have no cleanup on server shutdown or partial validator construction (
pkg/authserver/server/tokenexchange/multi_issuer_validator.go:433-470,557-578;pkg/authserver/server_impl.go:375-400). VirtualMCPServeralso still rejects advertised SPIFFE-only inline configuration because its guard ignoresinboundGrants.spiffeClientAuth(cmd/thv-operator/controllers/virtualmcpserver_controller.go:670-685).
CI is green; no local tests were run.
JAORMX
left a comment
There was a problem hiding this comment.
Changes requested:
cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go:737:principalPatternexcludes~, which the authoritative SPIFFE parser permits in path segments. Permit it and add admission coverage.cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go:649-653,2366-2373andcmd/thv-operator/pkg/controllerutil/authserver.go:1022-1024: malformed bundle endpoints and overlapping principal patterns are publishedValid=Trueuntil a consuming workload fails. Validate context-independent invariants at the owning config boundary.cmd/thv-operator/controllers/virtualmcpserver_controller.go:670: includeinboundGrants.spiffeClientAuthin inline auth-server validation; the shared CRD currently admits a SPIFFE-only shape that this guard rejects.
e038ceb to
ae26bc4
Compare
c5e6f6e to
ff2fc59
Compare
JAORMX
left a comment
There was a problem hiding this comment.
Changes requested:
cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go:737:principalPatternstill rejects~, which the authoritative SPIFFE parser permits in path segments. Permit it and add concrete and wildcard admission coverage.cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go:649-653,890-891andcmd/thv-operator/controllers/mcpexternalauthconfig_controller.go:106-176: malformed bundle endpoints and overlapping principal patterns can be publishedValid=Trueat the owning config boundary and only fail when a consuming workload validates the runtime configuration. Validate these context-independent invariants at admission or in the config controller.cmd/thv-operator/controllers/virtualmcpserver_controller.go:670-685: includeinboundGrants.spiffeClientAuthin inline auth-server validation; a SPIFFE-only inline shape admitted by the shared CRD is still rejected here.
All current CI checks are green; no local tests were run.
ae26bc4 to
ce2447f
Compare
ff2fc59 to
3b10dc8
Compare
JAORMX
left a comment
There was a problem hiding this comment.
Re-review at 3b10dc81849b9aae787cc77b600cc5714f2c9daf:
cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go:737still rejects~in SPIFFE principal path segments.~is valid in SPIFFE URI path segments, so the served CRD schema and runtime pattern validation are both narrower than the authoritative grammar. Permit it and add concrete plus terminal-wildcard admission coverage for bothMCPExternalAuthConfigandVirtualMCPServer.
The previous ~ blocker remains unresolved.
ce2447f to
1d2f7a3
Compare
3b10dc8 to
5545daa
Compare
1d2f7a3 to
5314ea2
Compare
5545daa to
a314f80
Compare
JAORMX
left a comment
There was a problem hiding this comment.
Deep re-review at 5545daa9c87e7ef9b53f893141aad1f65f8b6434 confirms the prior SPIFFE grammar blocker and finds another correctness gap:
- High: SPIFFE
resourcesare independently modeled but discarded while constructing static OAuth clients (pkg/authserver/spiffe_association_registry.go:56-58;pkg/authserver/server/registration/spiffe_client.go:31-44,69-72). Both RFC 8693audienceand RFC 8707resourceare then authorized viaGetAudience()(pkg/authserver/server/tokenexchange/handler.go:810-816,820-851). For disjointaudiences: [A]andresources: [R],resource=Ris denied andresource=Ais incorrectly permitted. Preserve and authorize resources separately; add four-way resource/audience tests. - Medium: durable SPIFFE placeholder reservations are never removed if an association disappears.
spiffe_decorator.go:60-83reconciles desired IDs only and neither decorator nor memory/Redis storage has a removal path (pkg/authserver/storage/spiffe_decorator.go:25-47,memory.go:528-546,redis.go:542-562). A removed/renamed SPIFFE ID stays unavailable indefinitely. Reconcile removals only for placeholders owned by this auth-server configuration; prove removed IDs become usable while active ones stay protected.
The prior valid-~ principal-pattern blocker remains.
|
Pushing back on the
So the CRD regex at Happy to be wrong here — if there's a reason to vendor go-spiffe with the backcompat tag, or if the concern is about the SPIFFE-ID spec's normative text rather than this specific library's default build, let me know and I'll take another look. Otherwise I'd like to close this one out as-is rather than keep carrying it forward. |
5314ea2 to
291104f
Compare
a314f80 to
4565c8d
Compare
291104f to
40a2370
Compare
4565c8d to
66edc12
Compare
JAORMX
left a comment
There was a problem hiding this comment.
Re-review at 66edc122efed74572d9a4d9784a1ef83e5455afe: malformed bundle endpoints and overlapping principal patterns are now validated at the owning configuration boundary, the base now preserves/enforces RFC 8707 resources, and the JWK cache lifecycle issue was fixed on main. Those prior concerns are resolved.
One blocker remains unchanged:
cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go:731-738rejects~inprincipalPatternpath segments, although~is valid in a SPIFFE ID path segment. This makes the served v1beta1 schema narrower than the authoritative SPIFFE grammar. Permit it in both the CRD marker and runtime pattern validation, and add concrete plus terminal-wildcard admission coverage for bothMCPExternalAuthConfigandVirtualMCPServer.
The updated head is a stack rebuild; this layer did not change relative to the previously reviewed implementation. All checks are green.
40a2370 to
d634568
Compare
The SPIFFE client-auth epic needs a place to configure SPIFFE association policy without inventing a parallel trust/grant path next to the existing delegate-client and trusted-issuer configuration. As more inbound grant families (RFC 8693 token exchange, RFC 7523 JWT-bearer, SPIFFE) accumulate, they need one canonical surface to configure and reason about instead of three independent ones, without breaking deployments that already rely on the legacy fields. Add pkg/authserver/inbound_grants.go with NormalizeInboundGrants, which reconciles a new canonical RunConfig.InboundGrants surface (per-family token_exchange/jwt_bearer sub-configs whose issuer_policies reference a trusted_issuers entry by name) against the legacy top-level delegate_clients and the RFC 8693/7523 fields embedded directly on trusted_issuers. Legacy and canonical configuration for the same grant family are mutually exclusive and rejected at validation time; the two families are otherwise independent, and omitting inbound_grants entirely preserves released behavior. Thread the normalized result through RunConfig.Validate, the embedded-auth-server runner, and buildProvider/discovery, adding a DisableTokenExchange capability so RFC 8693 registration and discovery advertisement can be turned off together and can't drift out of sync. Add TrustedIssuer.Name so canonical issuer_policies can reference an issuer without duplicating its fields. SPIFFE client authentication (InboundGrants.SPIFFEClientAuth, defined in the previous commit) is deliberately kept a sibling of TokenExchange and JWTBearer here, not nested under either: SPIFFE authenticates a client, it does not by itself grant it anything, so making it subordinate to RFC 8693 enablement would mean disabling token exchange silently drops every SPIFFE association, and every SPIFFE-authenticated client would be implicitly token-exchange-capable. It is validated and wired directly from RunConfig.InboundGrants in RunConfig.Validate/embeddedauthserver.go, independent of this file's legacy/canonical projection, so authentication method and grant-family enablement stay separately configurable. Update docs/arch/17-token-exchange-delegation.md for the new inbound_grants shape and the now-conditional token-exchange discovery advertisement, and add a runner-level test proving the canonical delegate-client, SPIFFE-client, and jwt_bearer paths reach a running server (the existing tests only covered normalization in isolation). SPIFFE client-auth associations always require the token-exchange grant (the only grant type they may declare), independent of the legacy/canonical token-exchange projection above: NormalizeInboundGrants now sets Capabilities.TokenExchange true whenever InboundGrants.SPIFFEClientAuth is non-empty, so a SPIFFE-only configuration cannot leave it false and silently disable the RFC 8693 grant handler server-wide -- which would reject every SPIFFE client's own token requests before authentication is even checked. Guarded by a regression test in this package (not just the runner-level test above) since the equivalent fix was previously lost during a rebase when its only coverage lived one package away. DCR (RFC 7591 /oauth/register) now rejects a registration whose effective grant types include token-exchange when it is disabled server-wide, instead of accepting the client and only failing later, confusingly, at /oauth/token. The check runs on the post-defaulting grant types validateGrantTypes already computes (a private_key_jwt client with an empty grant_types is implicitly token-exchange-only), so it catches both the explicit and implicit cases the same way scope validation already gates DCR on ScopesSupported. Corrected two stale doc references caught in review: the SPIFFE client-policy field path (inbound_grants.spiffe_client_auth, not nested under token_exchange) and the JWT-bearer legacy/canonical conflict wording (family-wide across all issuers, not per-issuer). Refs #6200 Signed-off-by: Jakub Hrozek <jakub@stacklok.com>
The SPIFFE client-auth epic needs a place to configure SPIFFE association policy without inventing a parallel trust/grant path next to the existing delegate-client and trusted-issuer configuration. As more inbound grant families (RFC 8693 token exchange, RFC 7523 JWT-bearer, SPIFFE) accumulate, they need one canonical surface to configure and reason about instead of three independent ones, without breaking deployments that already rely on the legacy fields. Add pkg/authserver/inbound_grants.go with NormalizeInboundGrants, which reconciles a new canonical RunConfig.InboundGrants surface (per-family token_exchange/jwt_bearer sub-configs whose issuer_policies reference a trusted_issuers entry by name) against the legacy top-level delegate_clients and the RFC 8693/7523 fields embedded directly on trusted_issuers. Legacy and canonical configuration for the same grant family are mutually exclusive and rejected at validation time; the two families are otherwise independent, and omitting inbound_grants entirely preserves released behavior. Thread the normalized result through RunConfig.Validate, the embedded-auth-server runner, and buildProvider/discovery, adding a DisableTokenExchange capability so RFC 8693 registration and discovery advertisement can be turned off together and can't drift out of sync. Add TrustedIssuer.Name so canonical issuer_policies can reference an issuer without duplicating its fields. SPIFFE client authentication (InboundGrants.SPIFFEClientAuth, defined in the previous commit) is deliberately kept a sibling of TokenExchange and JWTBearer here, not nested under either: SPIFFE authenticates a client, it does not by itself grant it anything, so making it subordinate to RFC 8693 enablement would mean disabling token exchange silently drops every SPIFFE association, and every SPIFFE-authenticated client would be implicitly token-exchange-capable. It is validated and wired directly from RunConfig.InboundGrants in RunConfig.Validate/embeddedauthserver.go, independent of this file's legacy/canonical projection, so authentication method and grant-family enablement stay separately configurable. Update docs/arch/17-token-exchange-delegation.md for the new inbound_grants shape and the now-conditional token-exchange discovery advertisement, and add a runner-level test proving the canonical delegate-client, SPIFFE-client, and jwt_bearer paths reach a running server (the existing tests only covered normalization in isolation). SPIFFE client-auth associations always require the token-exchange grant (the only grant type they may declare), independent of the legacy/canonical token-exchange projection above: NormalizeInboundGrants now sets Capabilities.TokenExchange true whenever InboundGrants.SPIFFEClientAuth is non-empty, so a SPIFFE-only configuration cannot leave it false and silently disable the RFC 8693 grant handler server-wide -- which would reject every SPIFFE client's own token requests before authentication is even checked. Guarded by a regression test in this package (not just the runner-level test above) since the equivalent fix was previously lost during a rebase when its only coverage lived one package away. DCR (RFC 7591 /oauth/register) now rejects a registration whose effective grant types include token-exchange when it is disabled server-wide, instead of accepting the client and only failing later, confusingly, at /oauth/token. The check runs on the post-defaulting grant types validateGrantTypes already computes (a private_key_jwt client with an empty grant_types is implicitly token-exchange-only), so it catches both the explicit and implicit cases the same way scope validation already gates DCR on ScopesSupported. Corrected two stale doc references caught in review: the SPIFFE client-policy field path (inbound_grants.spiffe_client_auth, not nested under token_exchange) and the JWT-bearer legacy/canonical conflict wording (family-wide across all issuers, not per-issuer). Refs #6200 Signed-off-by: Jakub Hrozek <jakub@stacklok.com>
The SPIFFE client-auth epic needs a place to configure SPIFFE association policy without inventing a parallel trust/grant path next to the existing delegate-client and trusted-issuer configuration. As more inbound grant families (RFC 8693 token exchange, RFC 7523 JWT-bearer, SPIFFE) accumulate, they need one canonical surface to configure and reason about instead of three independent ones, without breaking deployments that already rely on the legacy fields. Add pkg/authserver/inbound_grants.go with NormalizeInboundGrants, which reconciles a new canonical RunConfig.InboundGrants surface (per-family token_exchange/jwt_bearer sub-configs whose issuer_policies reference a trusted_issuers entry by name) against the legacy top-level delegate_clients and the RFC 8693/7523 fields embedded directly on trusted_issuers. Legacy and canonical configuration for the same grant family are mutually exclusive and rejected at validation time; the two families are otherwise independent, and omitting inbound_grants entirely preserves released behavior. Thread the normalized result through RunConfig.Validate, the embedded-auth-server runner, and buildProvider/discovery, adding a DisableTokenExchange capability so RFC 8693 registration and discovery advertisement can be turned off together and can't drift out of sync. Add TrustedIssuer.Name so canonical issuer_policies can reference an issuer without duplicating its fields. SPIFFE client authentication (InboundGrants.SPIFFEClientAuth, defined in the previous commit) is deliberately kept a sibling of TokenExchange and JWTBearer here, not nested under either: SPIFFE authenticates a client, it does not by itself grant it anything, so making it subordinate to RFC 8693 enablement would mean disabling token exchange silently drops every SPIFFE association, and every SPIFFE-authenticated client would be implicitly token-exchange-capable. It is validated and wired directly from RunConfig.InboundGrants in RunConfig.Validate/embeddedauthserver.go, independent of this file's legacy/canonical projection, so authentication method and grant-family enablement stay separately configurable. Update docs/arch/17-token-exchange-delegation.md for the new inbound_grants shape and the now-conditional token-exchange discovery advertisement, and add a runner-level test proving the canonical delegate-client, SPIFFE-client, and jwt_bearer paths reach a running server (the existing tests only covered normalization in isolation). SPIFFE client-auth associations always require the token-exchange grant (the only grant type they may declare), independent of the legacy/canonical token-exchange projection above: NormalizeInboundGrants now sets Capabilities.TokenExchange true whenever InboundGrants.SPIFFEClientAuth is non-empty, so a SPIFFE-only configuration cannot leave it false and silently disable the RFC 8693 grant handler server-wide -- which would reject every SPIFFE client's own token requests before authentication is even checked. Guarded by a regression test in this package (not just the runner-level test above) since the equivalent fix was previously lost during a rebase when its only coverage lived one package away. DCR (RFC 7591 /oauth/register) now rejects a registration whose effective grant types include token-exchange when it is disabled server-wide, instead of accepting the client and only failing later, confusingly, at /oauth/token. The check runs on the post-defaulting grant types validateGrantTypes already computes (a private_key_jwt client with an empty grant_types is implicitly token-exchange-only), so it catches both the explicit and implicit cases the same way scope validation already gates DCR on ScopesSupported. Corrected two stale doc references caught in review: the SPIFFE client-policy field path (inbound_grants.spiffe_client_auth, not nested under token_exchange) and the JWT-bearer legacy/canonical conflict wording (family-wide across all issuers, not per-issuer). Refs #6200 Signed-off-by: Jakub Hrozek <jakub@stacklok.com>
The SPIFFE client-auth epic needs a place to configure SPIFFE association policy without inventing a parallel trust/grant path next to the existing delegate-client and trusted-issuer configuration. As more inbound grant families (RFC 8693 token exchange, RFC 7523 JWT-bearer, SPIFFE) accumulate, they need one canonical surface to configure and reason about instead of three independent ones, without breaking deployments that already rely on the legacy fields. Add pkg/authserver/inbound_grants.go with NormalizeInboundGrants, which reconciles a new canonical RunConfig.InboundGrants surface (per-family token_exchange/jwt_bearer sub-configs whose issuer_policies reference a trusted_issuers entry by name) against the legacy top-level delegate_clients and the RFC 8693/7523 fields embedded directly on trusted_issuers. Legacy and canonical configuration for the same grant family are mutually exclusive and rejected at validation time; the two families are otherwise independent, and omitting inbound_grants entirely preserves released behavior. Thread the normalized result through RunConfig.Validate, the embedded-auth-server runner, and buildProvider/discovery, adding a DisableTokenExchange capability so RFC 8693 registration and discovery advertisement can be turned off together and can't drift out of sync. Add TrustedIssuer.Name so canonical issuer_policies can reference an issuer without duplicating its fields. SPIFFE client authentication (InboundGrants.SPIFFEClientAuth, defined in the previous commit) is deliberately kept a sibling of TokenExchange and JWTBearer here, not nested under either: SPIFFE authenticates a client, it does not by itself grant it anything, so making it subordinate to RFC 8693 enablement would mean disabling token exchange silently drops every SPIFFE association, and every SPIFFE-authenticated client would be implicitly token-exchange-capable. It is validated and wired directly from RunConfig.InboundGrants in RunConfig.Validate/embeddedauthserver.go, independent of this file's legacy/canonical projection, so authentication method and grant-family enablement stay separately configurable. Update docs/arch/17-token-exchange-delegation.md for the new inbound_grants shape and the now-conditional token-exchange discovery advertisement, and add a runner-level test proving the canonical delegate-client, SPIFFE-client, and jwt_bearer paths reach a running server (the existing tests only covered normalization in isolation). SPIFFE client-auth associations always require the token-exchange grant (the only grant type they may declare), independent of the legacy/canonical token-exchange projection above: NormalizeInboundGrants now sets Capabilities.TokenExchange true whenever InboundGrants.SPIFFEClientAuth is non-empty, so a SPIFFE-only configuration cannot leave it false and silently disable the RFC 8693 grant handler server-wide -- which would reject every SPIFFE client's own token requests before authentication is even checked. Guarded by a regression test in this package (not just the runner-level test above) since the equivalent fix was previously lost during a rebase when its only coverage lived one package away. DCR (RFC 7591 /oauth/register) now rejects a registration whose effective grant types include token-exchange when it is disabled server-wide, instead of accepting the client and only failing later, confusingly, at /oauth/token. The check runs on the post-defaulting grant types validateGrantTypes already computes (a private_key_jwt client with an empty grant_types is implicitly token-exchange-only), so it catches both the explicit and implicit cases the same way scope validation already gates DCR on ScopesSupported. Corrected two stale doc references caught in review: the SPIFFE client-policy field path (inbound_grants.spiffe_client_auth, not nested under token_exchange) and the JWT-bearer legacy/canonical conflict wording (family-wide across all issuers, not per-issuer). Refs #6200 Signed-off-by: Jakub Hrozek <jakub@stacklok.com>
The SPIFFE client-authentication runtime model (trust domains + static workload-to-client associations) already exists in pkg/authserver, but was only reachable by hand-authoring an auth-server RunConfig file -- there was no way to declare it through VirtualMCPServer or MCPExternalAuthConfig. Add SPIFFETrustDomains and InboundGrants.SPIFFEClientAuth to the shared EmbeddedAuthServerConfig CRD type, with CEL admission validation for everything derivable from the object's own spec (paired configuration, no duplicate names/trust domains/client IDs/principal patterns, full cross-referencing between domains and clients, method-subset enforcement, principal/trust-domain consistency, and client-ID hygiene). Wire the new fields through to authserver.RunConfig via the same converter pattern used for DelegateClients/TrustedIssuers, and extend the existing reconcile-time revalidation (validateDelegateClientsAndTrustedIssuers) to cover SPIFFE trust domains too. Cross-field checks that need reconcile-time-derived values (AllowedAudiences, ScopesSupported -- neither is CRD-exposed) are left to that reconcile-time path rather than a premature admission-time Go check: an earlier attempt at that check passed nil for both, which the runtime validator treats as "validate against nothing" rather than "skip", silently rejecting any resources or custom scopes entry. A syntactically valid SPIFFE config is admitted by CEL but still fails reconciliation with a terminal error until RunConfig's "not yet enforced" placeholder gate is lifted by a future PR -- no live SVID verification exists yet, so this is expected, not a regression. Signed-off-by: Jakub Hrozek <jakub@stacklok.com>
66edc12 to
e44e5c8
Compare
JAORMX
left a comment
There was a problem hiding this comment.
Re-review at e44e5c8 (reviewed only d634568..e44e5c8):
Resolved: bundle endpoints and overlapping principals are validated at the owning configuration boundary; SPIFFE-only inline config is accepted; resources are preserved; generated artifacts are synchronized.
Remaining blockers:
- cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go:731-738; pkg/authserver/spiffe_trust.go:663-690: principalPattern still rejects ~, which is valid in SPIFFE path segments. Permit it in CRD and runtime validation, regenerate output, and add concrete plus terminal-wildcard coverage for both CRDs.
- cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go:756-780,2368-2374: context-independent malformed/empty audiences, scopes, and resource URI shapes can be admitted and reported Valid=True because all permission validation is deferred. Keep derived allowlist checks at reconciliation, but validate independent shape constraints at the owning boundary.
Exact-head CI is green.
Summary
The SPIFFE client-authentication runtime model (trust domains + static workload-to-client associations) already exists in
pkg/authserver, but was only reachable by hand-authoring an auth-server RunConfig file directly — there was no way to declare it throughVirtualMCPServer/MCPExternalAuthConfig.An earlier, never-reviewed attempt at this CRD exposure existed on an abandoned branch, but assumed a stale runtime shape (SPIFFE clients nested under
inboundGrants.tokenExchange) that no longer matches the reviewed model (inboundGrants.spiffeClientAuthis now a sibling oftokenExchange/jwtBearer, with a larger field surface: abundleSourcediscriminated union,resources, and a fixedgrantTypes). This PR redoes the CRD exposure from scratch against the current model, reusing the old attempt's validation contract (its CEL test matrix) as an acceptance checklist, not its code.spiffeTrustDomainsandinboundGrants.spiffeClientAuthto the sharedEmbeddedAuthServerConfig, with CEL admission validation for everything derivable from the object's own spec (paired configuration, no duplicate names/trust-domains/client-IDs/principal-patterns, full cross-referencing between domains and clients, method-subset enforcement, client-ID hygiene).authserver.RunConfigvia the same converter pattern used fordelegateClients/trustedIssuers, and extend reconcile-time revalidation to cover SPIFFE trust domains.grantTypesas a CRD field — the runtime only accepts exactly one value (urn:ietf:params:oauth:grant-type:token-exchange), so the converter sets it rather than exposing a field with one legal value.allowedAudiences,scopesSupported— neither is CRD-exposed) are deliberately left to the existing reconcile-time revalidation path rather than a premature admission-time Go check — an earlier draft of this PR added such a check passingnilfor both, which the runtime validator treats as "validate against nothing" rather than "skip," silently rejecting anyresourcesentry or non-defaultscopesvalue. Caught and removed during adversarial review; see commit history for detail.Refs #6199, #6200, #6205
Known, intentional consequence: a syntactically valid SPIFFE config is admitted by CEL but still fails reconciliation with a terminal error until
RunConfig's "not yet enforced" placeholder gate (pkg/authserver/config.go) is lifted by a future PR — no live SVID verification exists yet, so this is expected, not a regression.Type of change
Test plan
task test)task lint-fix)31-case CEL admission test matrix (
spiffe_cel_test.go) ported from the old attempt's test list and extended for thebundleSourcediscriminated union andresources; unit tests for the two new converter functions; a reconcile-level test proving a SPIFFE client withresources/customscopesis accepted at the admission-equivalent Go layer but still correctly validated (accepted or rejected as appropriate) once real derived values reachBuildAuthServerRunConfig.API Compatibility
v1beta1API — both new fields are optional and additive.Changes
Large diff (~3,384 lines), dominated by generated CRD YAML/docs (
deploy/charts/operator-crds/**,docs/operator/crd-api.md— roughly 2,200 of the changed lines) and the new CEL test file. Hand-written surface:mcpexternalauthconfig_types.go(new CRD types + 11 CEL rules),controllerutil/authserver.go(converter wiring), and their tests.Does this introduce a user-facing change?
Yes — operators can now declare
spiffeTrustDomains/inboundGrants.spiffeClientAuthonVirtualMCPServer/MCPExternalAuthConfig. Reconciliation of a non-empty config will fail with a clear terminal error until live SVID verification lands (see "Known, intentional consequence" above).Implementation plan
Approved implementation plan
Expose SPIFFE client-auth registration through the operator CRDs
Context
The SPIFFE client-auth epic (#6199, refs #6200/#6205) already landed a full
runtime model in
pkg/authserver(SPIFFETrustDomainRunConfig,SPIFFEClientAuthRunConfig,ValidateSPIFFETrust, the static-clientregistry/storage overlay, etc.), but that model is only reachable by hand-
authoring an auth-server
RunConfigfile — there is no way to declare itthrough
VirtualMCPServer/MCPExternalAuthConfigCRDs.An old, never-reviewed branch (
spiffe-integration-split3-5/6/7, abandonedmid-epic when the stack was reworked through review) had already built this
CRD-exposure feature once, but its commit (
0c921b295, "Expose SPIFFEregistration in CRDs") assumed a runtime shape that no longer matches: it
nested SPIFFE clients under
InboundGrants.TokenExchange.SPIFFEClientsandnever modeled
BundleSource/Resources/GrantTypes. The current, reviewedruntime model puts
SPIFFEClientAuthas a top-level sibling ofTokenExchangeonInboundGrantsRunConfig(deliberately independent of thetoken-exchange capability toggle — see the doc comment on
InboundGrantsRunConfiginpkg/authserver/spiffe_trust.go), and has amaterially larger
SPIFFEClientAuthRunConfig/SPIFFETrustDomainRunConfigsurface than the old commit ever covered. Rather than merge that stale
commit, we're redoing the CRD-exposure work from scratch against the current
model, reusing the old commit's validation contract (its CEL test matrix)
as an acceptance checklist, not its code.
Known, intentional consequence:
RunConfig.Validate()currentlyhard-rejects any non-empty
SPIFFETrustDomainsviavalidateSPIFFENotYetEnforced— a deliberate placeholder until real SVIDverification lands (see
pkg/authserver/config.go:328-345). Wiring CRDfields into
RunConfig.SPIFFETrustDomainsmeans a syntactically valid CRDspec will be admitted by CEL/webhook validation but will still fail
reconciliation with a terminal
InvalidEmbeddedAuthServerConfigErroruntilthat gate is lifted by a future PR. This is expected, not a bug — call it
out explicitly in the PR description.
Scope
In scope: two new CRD types (
SPIFFETrustDomainConfig,SPIFFEClientConfig) plus a bundle-source discriminated union, added to theshared
EmbeddedAuthServerConfig(used by bothMCPExternalAuthConfigandVirtualMCPServer— no VMCP-specific code needed, same as the old commitfound). CEL admission validation for everything expressible from the
object's own spec. Go-level bridging into the existing
authserver.ValidateSPIFFETrust/NewSPIFFETrustConfigfor everything thatneeds derived values (
AllowedAudiences,ScopesSupported— neither isCRD-exposed; both are derived at reconcile time, confirmed via
cmd/thv-operator/pkg/controllerutil/authserver.go:883-895) or the fullSPIFFE-ID/URL grammar. Reconcile-time revalidation via the existing
validateDelegateClientsAndTrustedIssuerspattern.Out of scope: bundle fetching/loading, live SVID verification, lifting
the
validateSPIFFENotYetEnforcedgate (all still placeholder by design),and the legacy-field-removal refactor that was tangled into the old commit
(
0c921b295) — that already landed separately via0501c3fea/split3-5.CRD types —
cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.goMirror the runtime shape exactly (field-for-field with
SPIFFETrustDomainRunConfig/SPIFFEClientAuthRunConfiginpkg/authserver/spiffe_trust.go), using the same validation-marker idioms asneighboring types (
DelegateClientConfigat line 361,TrustedIssuerConfigat line 404) —
+kubebuilder:validation:MinLength/MaxLength,MinItems/MaxItems,+listType=atomic/set,+optionalon every pointerfield,
//nolint:lllon types carrying long CEL rules.SPIFFEAuthenticationMethodenum (spiffe_x509,spiffe_jwt),SPIFFEBundleSourceTypeenum (bundle_endpoint,workload_api),SPIFFEBundleEndpointProfileenum (https_web,https_spiffe) —straight string-const mirrors of the runtime enums.
SPIFFEBundleSourceConfig(discriminated union, required field onSPIFFETrustDomainConfig— the runtime field has noomitempty):Type SPIFFEBundleSourceTypeEndpoint *SPIFFEBundleEndpointSourceConfig(URL string,Profile SPIFFEBundleEndpointProfile)WorkloadAPI *SPIFFEWorkloadAPIBundleSourceConfig(empty struct)XValidationrules reusing the exact pattern already used forMCPExternalAuthConfig's own type-discriminated union (line 67-75):self.type == 'bundle_endpoint' ? has(self.endpoint) : !has(self.endpoint)and the
workload_api/workloadAPImirror.SPIFFETrustDomainConfig:Name,TrustDomain(pattern-validated,copy the old commit's regex — it's a reasonable approximation and the
type doc comment should say runtime parsing via
spiffeid.TrustDomainFromStringremains authoritative, matching the existing "CEL is best-effort" convention
already used elsewhere in this file),
Methods(MinItems=1,MaxItems=2,enum),
BundleSource SPIFFEBundleSourceConfig(required, no+optional).SPIFFEClientConfig(lives in a newSPIFFEClientAuth []SPIFFEClientConfigfield onInboundGrantsConfig— a siblingof
TokenExchange/JWTBearer, not nested underTokenExchange, matchingInboundGrantsRunConfig's shape):TrustDomainRef,PrincipalPattern(pattern-validated + the oldcommit's path-traversal CEL rule
self.principalPattern.split('/').all(segment, segment != '.' && segment != '..')),ClientID,Methods(MinItems=1,MaxItems=2),Audiences(
MinItems=1,MaxItems=50),Scopes(MinItems=1,MaxItems=50),Resources(+optional,MaxItems=50— new vs. the old commit; RFC8707 resources, Go-validated against
AllowedAudiencessince that'sderived, not CRD-known).
GrantTypesas a CRD field. The runtime only acceptsexactly
["urn:ietf:params:oauth:grant-type:token-exchange"](
validateSPIFFEGrants,spiffe_trust.go:758-766) — a field with onelegal value is a field the converter should just set, not a knob users
configure. Set it in the converter (
GrantTypes: []string{authserver.SPIFFEGrantTypeTokenExchange}).SPIFFETrustDomains []SPIFFETrustDomainConfigstays onEmbeddedAuthServerConfigdirectly (top-level, matchingRunConfig.SPIFFETrustDomains) —MinItems=1,MaxItems=50, +listType=atomic, +optional, same placement the old commit used.CEL rules on
EmbeddedAuthServerConfig(add alongside its existingXValidationblock): reuse the old commit's semantic set, adjusted forthe new field path (
self.inboundGrants.spiffeClientAuth, notself.inboundGrants.tokenExchange.spiffeClients):spiffeTrustDomainsandinboundGrants.spiffeClientAuthempty/non-empty together)nametrustDomainvaluetrustDomainRefresolves to a declared domainmethods⊆ referenced domain'smethodsprincipalPattern's trust-domain segment matches the referenceddomain's
trustDomainclientIdprincipalPatternclientIdmust not start withsynthetic:(reserved) or look like anabsolute URL (reserved for CIMD) — reuse
storage.ValidateRegisterableClientID's intent as the CEL approximation,Go-side stays authoritative
Go bridging — same file, new
validateSPIFFETrustConfighelperAlongside the existing
validateEmbeddedAuthServer(): convertcfg.SPIFFETrustDomains→[]authserver.SPIFFETrustDomainRunConfigandcfg.InboundGrants.SPIFFEClientAuth→[]authserver.SPIFFEClientAuthRunConfig(setting
GrantTypesto the fixed value here), callauthserver.ValidateSPIFFETrust(trustDomains, inboundGrants, nil, nil)for aconfig-shape-only check at admission time (no
AllowedAudiences/ScopesSupportedavailable yet — same limitation already accepted forDelegateClients/TrustedIssuersat this layer).Controller wiring —
cmd/thv-operator/pkg/controllerutil/authserver.gobuildSPIFFETrustDomainRunConfigs([]mcpv1beta1.SPIFFETrustDomainConfig) []authserver.SPIFFETrustDomainRunConfigand
buildSPIFFEClientAuthRunConfigs([]mcpv1beta1.SPIFFEClientConfig) []authserver.SPIFFEClientAuthRunConfig(pure field copy + slice clone, no error — same shape as
buildTrustedIssuerRunConfigsat line 269, since nothing here needs secretresolution).
buildInboundGrantsRunConfig(line 824): setgrants.SPIFFEClientAuth = buildSPIFFEClientAuthRunConfigs(config.SPIFFEClientAuth)as a sibling assignment alongside the existing
TokenExchange/JWTBearerblocks.
BuildAuthServerRunConfig(line 889): setconfig.SPIFFETrustDomains = buildSPIFFETrustDomainRunConfigs(authConfig.SPIFFETrustDomains)next to the existing
config.InboundGrants = inboundGrantsassignment.validateDelegateClientsAndTrustedIssuers(line 1034): extend the early-return guard to also check
len(config.SPIFFETrustDomains) == 0, and addSPIFFETrustDomains: config.SPIFFETrustDomainstovalidationConfigsoRunConfig.Validate()catches SPIFFE misconfiguration (including the"not yet enforced" gate) as a reconcile error, not a pod crash loop —
consistent with why this function exists at all.
Tests
(
spiffe_cel_test.go, ~30 cases: paired-config, duplicate name/trustDomain/clientId/principalPattern, unreferenced/unknown trust-domain-ref,
methods-not-a-subset, principal/trust-domain-segment mismatch, reserved
clientId prefixes, trust-domain string edge cases, principal path-segment
edge cases) rewritten against the new field path
(
inboundGrants.spiffeClientAuth, not.tokenExchange.spiffeClients) andextended for
BundleSource's discriminated union and the newResourcesfield.
admitted by CEL but produces a terminal
InvalidEmbeddedAuthServerConfigErrorreconcile failure (proving the "not yet enforced" gate is reachable end-to-
end through the CRD path, not just the RunConfig-file path already tested
in
pkg/authserver).validateSPIFFETrustConfig, following this file's existing table-drivenconvention.
Verification
task operator-manifeststhentask operator-generate— confirm CRDYAML and deepcopy regenerate cleanly (watch for the same missing-JSON-tag
class of controller-gen failure hit earlier in this session).
task lint-fix— 0 issues.task test— full suite green, including the new CEL/reconcile tests(these need
envtest/a real apiserver — same suite that already runscmd/thv-operator/test-integration/mcp-external-auth/*_cel_test.go).VirtualMCPServerwith valid SPIFFE fields against alocal kind cluster (or envtest) and confirm: CRD admission succeeds, then
the reconciler surfaces the expected "not yet enforced" terminal
condition — not a crash loop, not silent success.
Delegation
Hand off implementation to a fresh agent (this is CRD/operator work spanning
Go API types, CEL, and controller wiring —
kubernetes-expertor a freshgo-expert-developer), then run an adversarial Opus review pass beforecommitting, matching this session's established mechanism for anything
security/protocol-relevant. Land as
spiffe-integration-split3-6on top ofthe already-verified
spiffe-integration-split3-5.Special notes for reviewers
This PR was implemented by a fresh agent against the approved plan above, then went through one full adversarial review round: the reviewer found two real bugs (an admission-time Go check that unconditionally rejected any
resources/custom-scopesentry due to a bad nil-handling assumption in the plan itself, and missingtask crdref-genoutput) plus an untested design deviation — all fixed and re-verified before this PR was opened. Stacked on #6499.