Skip to content

Optimizer mode: allow an operator-pinned set of always-advertised tools #6514

Description

@premctl

Problem

In optimizer mode the vMCP advertises exactly two tools, find_tool and call_tool, and there is no way for an operator to keep any real tool visible alongside them. #4357 is fixing the description of that pair so agents are more likely to reach for it. This issue is about the case where a better description is not enough and the operator knows, ahead of time, which handful of tools should simply be in the client's tool list.

The list is unconditional:

  • pkg/vmcp/session/optimizerdec/decorator.go:91Tools() returns a copy of optimizerTools, which is OptimizerTools() (:62) and nothing else.
  • pkg/vmcp/server/serve_optimizer.go:81 — the Serve path builds its sdkTools from the same optimizerdec.OptimizerTools(), so both eras behave identically.
  • test/e2e/thv-operator/virtualmcp/virtualmcp_optimizer_composite_test.go:188-201 asserts HaveLen(2) / ConsistOf("find_tool", "call_tool") — composite tools are hidden behind find_tool too, so defining a composite tool is not an escape hatch.
  • ToolOverride (pkg/vmcp/config/config.go:621) is workload-keyed, so it cannot reach the meta-tools either.

Concretely, on a vMCP fronting ~10 read-only kubernetes-mcp-server backends (one per cluster, tool names prefixed with the cluster ID), a request like "list pods in namespace X in cluster Y" names its target unambiguously — but nothing in the advertised pair mentions Kubernetes, pods, or any cluster name, so clients reach for kubectl via bash instead. The server is only used when the user names it explicitly. This is the same failure mode as #4357, from the other side: the useful routing signal is the tool names themselves, and no amount of description text on find_tool reproduces ~135 cluster-prefixed names.

Turning the optimizer off fixes routing and costs the whole catalog (baseline_tokens 36552 here, ~96% reported savings). That is the trade this issue is trying to avoid making: the choice today is all tools or none.

Proposed solution

Let the operator declare a small set of tools that stay advertised in optimizer mode, on top of find_tool / call_tool.

type OptimizerConfig struct {
    // ... existing fields ...

    // AlwaysAdvertise lists backend tool names that stay in the advertised tool
    // list in optimizer mode, alongside find_tool and call_tool. Use it to keep a
    // small, high-traffic subset directly callable so agents route to this server
    // without a find_tool round-trip. Names not present in the session's tool set
    // are ignored. When empty, only the meta-tools are advertised (today's
    // behaviour).
    // +optional
    AlwaysAdvertise []string `json:"alwaysAdvertise,omitempty" yaml:"alwaysAdvertise,omitempty"`
}

Both call sites already have the full advertisable set in hand, so the seam is small:

  • Serve path (serve_optimizer.go): coreTools is already in scope where optimizerFactory is called; filter it by the allowlist and append those definitions to sdkTools, wiring each to its existing core handler rather than to an optimizer meta-handler.
  • Legacy decorator (optimizerdec): NewDecorator takes the allowlist and Tools() returns OptimizerTools() plus the matching subset of d.MultiSession.Tools().

Pinning would only affect advertisement. The optimizer index stays as-is, so a pinned tool is still discoverable through find_tool, and call_tool still resolves it — no behaviour change for clients that ignore the pinned entries.

Notes

  • Admission/authz must run on the pinned entries the same way it runs on the rest, so a pinned tool the caller cannot use is filtered out rather than advertised — cf. Optimizer find_tool returns tools the user is not authorized to use #4374.
  • token_metrics.baseline_tokens should probably exclude what is already advertised, otherwise reported savings overstate the win once tools are pinned.
  • A glob or prefix form ("*_namespaces_list") would help fleets where the same tool exists on every backend, but exact names are enough to be useful and are simpler to validate.

Out of scope

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageIssue needs initial triage by a maintainer

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions