Allow opt-in internal/private remoteUrl hosts for co-located MCP backends (operator ValidateRemoteURL has no allowlist)

[premctl]

Allow opt-in internal/private remoteUrl hosts for co-located MCP backends (operator ValidateRemoteURL has no allowlist)

Type: enhancement · Area: thv-operator (MCPServerEntry / MCPRemoteProxy) · Version: v0.34.0 (latest release, 2026-07-07; also present on main)

Summary

The operator's ValidateRemoteURL unconditionally rejects any remoteUrl whose host is a well-known internal hostname (anything ending in cluster.local, kubernetes.default[.svc], localhost, metadata.google.internal) or a literal private/loopback IP. There is no allowlist, per-resource opt-in, or operator flag to permit a specific internal endpoint. This blocks a legitimate and (we think) common topology: a vMCP co-located in the same cluster as one of its backend MCP servers, where the backend must be reached in-mesh so that Istio/SPIFFE workload identity is preserved for the backend's authorization policy.

We'd like an explicit, default-off way to allow a known internal backend URL — mirroring the AllowPrivateIP opt-in that already exists for the registry and JWKS/OIDC upstream paths.

Where it happens (code)

• cmd/thv-operator/pkg/validation/url_validation.go — ValidateRemoteURL(rawURL string) error
  • blockedHostnames = [localhost, kubernetes.default.svc.cluster.local, kubernetes.default.svc, kubernetes.default, cluster.local, metadata.google.internal] (suffix-matched)
  • internalCIDRs = [0.0.0.0/8, 127.0.0.0/8, 169.254.0.0/16, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, ::/128, ::1/128, fe80::/10, fc00::/7] (applied to literal-IP hosts only; no DNS resolution)
  • The function takes only rawURL string — no options/allowlist parameter.
• Called unconditionally by:
  • cmd/thv-operator/controllers/mcpserverentry_controller.go → validateRemoteURL(...)
  • cmd/thv-operator/controllers/mcpremoteproxy_controller.go

Reproduce

Create an MCPServerEntry pointing at an in-cluster service:

apiVersion: toolhive.stacklok.dev/v1beta1
kind: MCPServerEntry
metadata:
  name: local-backend
spec:
  groupRef: { name: my-group }
  transport: streamable-http
  remoteUrl: "http://my-mcp.my-ns.svc.cluster.local:8080/mcp"

Status:

RemoteURLValidated = False  (RemoteURLInvalid)
message: remote URL host is not allowed: hostname
  "my-mcp.my-ns.svc.cluster.local" matches blocked internal hostname "cluster.local"
Valid = False (ConfigInvalid) → backend never registers with the vMCP

Why the internal URL is the correct target here

Our vMCP runs in the same cluster as one of its backends. That backend's /mcp is protected by an Istio AuthorizationPolicy that allows only the vMCP's SPIFFE identity (spiffe://…/ns/<ns>/sa/<vmcp-sa>). To present that identity, the vMCP must call the backend in-mesh (its ClusterIP), sidecar-to-sidecar. Reaching it via the backend's public ingress host instead hairpins through a shared ingress gateway, so the backend authenticates the ingress gateway's identity, not the vMCP's — and the policy correctly denies it. So for a co-located backend the in-cluster URL isn't an SSRF risk, it's the only path that carries the right identity.

Current workaround, and why it's fragile

Because ValidateRemoteURL does no DNS resolution and only literal-string-matches the blocklist, a namespace-qualified short name (http://my-mcp.my-ns:8080/mcp) passes validation and still resolves in-cluster (pod dnsPolicy: ClusterFirst, ndots:5 search-expands it to the same ClusterIP). It works today — but it's a hack: it depends on the blocklist staying literal-string and on ndots search behavior. A future change that also blocks .svc or that resolves-then-CIDR-checks would silently re-break every co-located backend.

Proposal

Add an explicit, default-off opt-in (either, or both):

1. Per-resource field (preferred — least privilege, self-documenting):

   spec:
     remoteUrl: "http://my-mcp.my-ns.svc.cluster.local:8080/mcp"
     allowPrivateEndpoint: true   # default false; skips the internal-host/private-IP block for THIS entry

   ValidateRemoteURL gains an options arg; the reconcilers pass the field through.

2. Operator-level allowlist env (coarser alternative):
   THV_OPERATOR_ALLOWED_INTERNAL_HOSTS="my-mcp.my-ns.svc.cluster.local,.my-ns.svc.cluster.local" — host/suffix allowlist consulted in ValidateRemoteURL.

Precedent already in the codebase: AllowPrivateIP is a first-class opt-in for the registry (pkg/registry/..., --allow-private-ip) and for JWKS/OIDC upstreams (pkg/vmcp/auth/factory/incoming.go: AllowPrivateIP: oidcCfg.ProtectedResourceAllowPrivateIP || oidcCfg.JwksAllowPrivateIP; documented in docs/server/swagger.yaml as "allows JWKS/OIDC endpoints on private IP addresses"). This request is the same pattern applied to remote backend URLs.

Security notes

• Default behavior is unchanged (deny) — SSRF protection stays on for every entry that doesn't opt in.
• Opt-in is authored in cluster-scoped CRs / operator config by a cluster admin, not by end users or MCP tool input.
• Scope can be kept tight (a specific host / suffix), rather than a blanket "allow all private."

[aponcedeleonch]

Thanks for the detailed writeup, the code references all check out against current main. Confirmed:

• ValidateRemoteURL(rawURL string) error in cmd/thv-operator/pkg/validation/url_validation.go takes only the URL, with the blocklist and CIDR set exactly as you described, and does no DNS resolution.
• Both mcpserverentry_controller.go and mcpremoteproxy_controller.go call it unconditionally, with no per-resource or operator-level bypass.
• The AllowPrivateIP precedent is real for the registry (--allow-private-ip) and JWKS/OIDC upstreams (JWKSAllowPrivateIP / ProtectedResourceAllowPrivateIP), but it doesn't extend to backend remoteUrl.

So the capability you're asking for doesn't exist today, and the co-located-backend in-mesh SPIFFE topology is a legitimate case. I like the per-resource allowPrivateEndpoint option better than the env allowlist. It's least-privilege and self-documenting, and it keeps the default deny intact. We'll want to settle field-level vs operator-env before anyone picks it up. If you'd be up for contributing it, happy to help scope the change.

[premctl]

Thanks for confirming. Agreed on the per-resource field over the operator env — least-privilege, self-documenting, and it keeps the default deny intact.

Proposed shape:

• allowPrivateEndpoint bool (default false) on both MCPServerEntry and MCPRemoteProxy, plumbed into ValidateRemoteURL as an option (rather than the bare rawURL string).
• One refinement: even when true, keep rejecting link-local / cloud-metadata — 169.254.0.0/16 (+ its IPv4-mapped IPv6 form) and the metadata.google.internal / kubernetes.default* names. Those are never a legitimate backend and are the real SSRF target. Opting in should relax RFC-1918 + cluster.local/.svc reachability only; the "never" set stays unconditional. So the flag widens the private-network allowance, it doesn't disable SSRF protection.
• Doc the intended use: a co-located in-cluster backend reached in-mesh, so workload-identity authz still applies.

I'm up for taking this one — I'll open a PR; happy to nail down the exact carve-out set / field name here first if you'd prefer.