Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/upstream-projects.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ projects:

- id: toolhive
repo: stacklok/toolhive
version: v0.35.0
version: v0.38.0
# toolhive is a monorepo covering the CLI, the Kubernetes
# operator, and the vMCP gateway. It also introduces cross-
# cutting features that land in concepts/, integrations/,
Expand Down
60 changes: 59 additions & 1 deletion docs/toolhive/guides-k8s/remote-mcp-proxy.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,62 @@ via `AssumeRoleWithWebIdentity` and signs requests with SigV4. See the

:::

### Customize the remote proxy pod

Use `podTemplateSpec` to set pod-level options that aren't exposed as
first-class fields on `MCPRemoteProxy`, such as security contexts, node
selectors, tolerations, and affinity rules. The field follows the standard
Kubernetes
[`PodTemplateSpec`](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-template-v1/#PodTemplateSpec)
format, and you only need to specify the fields you want to add or override.

This example sets resource limits on the proxy container and adds a node
selector:

```yaml {15-27} title="analytics-proxy-custom-pod.yaml"
apiVersion: toolhive.stacklok.dev/v1beta1
kind: MCPRemoteProxy
metadata:
name: analytics-proxy
namespace: toolhive-system
spec:
remoteUrl: https://mcp.analytics.example.com
proxyPort: 8080
transport: streamable-http

oidcConfigRef:
name: analytics-oidc
audience: analytics-mcp-proxy

podTemplateSpec:
spec:
nodeSelector:
workload-tier: platform
containers:
- name: toolhive # This name must be "toolhive"
resources:
limits:
cpu: '500m'
memory: '512Mi'
requests:
cpu: '100m'
memory: '128Mi'
```

:::info[Container name requirement]

To override the proxy container, use `name: toolhive`. The operator applies
overrides by matching the container name; any other name adds a sidecar instead.
This differs from [`MCPServer`](./run-mcp-k8s.mdx#customize-the-mcp-server-pod),
where the main container is `mcp`.

:::

If the pod template fails validation, the operator sets the `PodTemplateValid`
condition to `False`, moves the resource to phase `Failed`, and stops updating
the Deployment. See [Check remote proxy status](#check-remote-proxy-status) to
inspect the failure message.

### Inject custom headers

Some remote MCP servers require custom headers for tenant identification, API
Expand Down Expand Up @@ -742,7 +798,9 @@ The status shows:
- **Phase**: Current state (Pending, Ready, Failed, Terminating)
- **URL**: Internal cluster URL
- **External URL**: External URL if exposed via Ingress
- **Conditions**: Detailed status conditions
- **Conditions**: Detailed status conditions, including `PodTemplateValid` when
you use [`podTemplateSpec`](#customize-the-remote-proxy-pod). Check the
`message` field of any condition reporting `status: "False"` to see why.

## Telemetry and observability

Expand Down
14 changes: 11 additions & 3 deletions docs/toolhive/reference/cli/thv_llm_setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Configure detected AI tools to use the LLM gateway
### Synopsis

Detect installed AI tools (Claude Code, Gemini CLI, Cursor, VS Code, Xcode,
Claude Desktop) and patch each tool's configuration to route through the LLM
gateway.
Claude Desktop, Codex) and patch each tool's configuration to route through the
LLM gateway.

Token-helper tools (Claude Code, Gemini CLI) are configured to call
"thv llm token" to obtain a fresh OIDC token on demand.
Expand All @@ -27,6 +27,14 @@ which also calls "thv llm token". It reads its configuration only at launch, so
fully quit and relaunch it after setup. Pass --models to list the models it
should offer until the gateway serves model discovery itself.

Codex CLI and the ChatGPT desktop app use the same user configuration:
~/.codex/config.toml by default, or %USERPROFILE%\.codex\config.toml on Windows.
ToolHive adds a custom model_provider whose auth command invokes "thv llm token"
directly (no shell), keeping existing model_providers and mcp_servers entries
untouched. Desktop app detection is supported on macOS and Windows; the canonical
--client target remains "codex". If the desktop app is running, fully quit and
reopen it after setup.

Proxy-mode tools (Cursor, VS Code, Xcode) are configured to send requests to
the localhost reverse proxy started by "thv llm proxy start".

Expand All @@ -49,7 +57,7 @@ thv llm setup [flags]
--anthropic-path-prefix string Path prefix appended to the gateway URL when writing ANTHROPIC_BASE_URL for direct-mode tools (e.g. /anthropic). When omitted, the gateway is probed automatically.
--audience string OIDC audience (optional)
--callback-port int OIDC callback port (omit to keep current; default: ephemeral)
--client string Configure only this AI tool by name (e.g. claude-code, cursor). Omit to configure all detected tools.
--client string Configure only this AI tool by name (e.g. claude-code, cursor, codex). Omit to configure all detected tools.
--client-id string OIDC client ID
--gateway-url string LLM gateway base URL (must use HTTPS)
-h, --help help for setup
Expand Down
27 changes: 27 additions & 0 deletions static/api-specs/toolhive-api.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1102,6 +1102,19 @@ components:
An empty string signals "use the default for this transport type" during config merging.
Examples: "golang:1.26-alpine", "node:24-alpine", "python:3.14-slim"
type: string
runtime_env:
additionalProperties:
type: string
description: |-
RuntimeEnv contains environment variables to inject into the Dockerfile's
final runtime stage. Unlike BuildEnv (pkg/container/templates.TemplateData.BuildEnv),
which only affects the builder stage, these variables are baked into the
shipped image and are present in the running container's process
environment at startup. Use this for values a packaged MCP server reads at
process start (e.g. feature flags, cache backend selection), not for
build-time package manager configuration.
Keys must be uppercase with underscores, values are validated for safety.
type: object
type: object
github_com_stacklok_toolhive_pkg_core.Workload:
properties:
Expand Down Expand Up @@ -2572,6 +2585,13 @@ components:
pkg_api_v1.createRequest:
description: Request to create a new workload
properties:
allow_docker_gateway:
description: |-
Whether to permit outbound connections to Docker gateway addresses
(host.docker.internal, gateway.docker.internal, 172.17.0.1). These are
blocked by default in the egress proxy even when network isolation is on.
Only applicable to Docker deployments with network isolation enabled.
type: boolean
authz_config:
description: Authorization configuration
type: string
Expand Down Expand Up @@ -3104,6 +3124,13 @@ components:
pkg_api_v1.updateRequest:
description: Request to update an existing workload (name cannot be changed)
properties:
allow_docker_gateway:
description: |-
Whether to permit outbound connections to Docker gateway addresses
(host.docker.internal, gateway.docker.internal, 172.17.0.1). These are
blocked by default in the egress proxy even when network isolation is on.
Only applicable to Docker deployments with network isolation enabled.
type: boolean
authz_config:
description: Authorization configuration
type: string
Expand Down
5 changes: 5 additions & 0 deletions static/api-specs/toolhive-crds/mcpremoteproxies.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,11 @@
],
"type": "object"
},
"podTemplateSpec": {
"description": "PodTemplateSpec defines the pod template to use for the MCPRemoteProxy\nThis allows for customizing the pod configuration beyond what is provided by the other fields.\nNote that to modify the specific container the remote proxy runs in, you must specify\nthe `toolhive` container name in the PodTemplateSpec.\nThis field accepts a PodTemplateSpec object as JSON/YAML.",
"type": "object",
"x-kubernetes-preserve-unknown-fields": true
},
"proxyPort": {
"default": 8080,
"description": "ProxyPort is the port to expose the MCP proxy on",
Expand Down