Replace FastMCP with toolregistry-server: multi-protocol, profiles, policy tags

[Oaklight]

Summary

Replace the FastMCP-based server.py with a toolregistry + toolregistry-server implementation. Same tool functions, same Claude Desktop experience by default, but the server now supports:

• Multi-transport MCP (stdio / SSE / streamable HTTP)
• Optional OpenAPI / REST surface (pip install uxarray-mcp[openapi])
• Namespace grouping (session/, hpc/, io/, prompt/, compute/, ...)
• Two profiles: conservative core default and opt-in deferred-full with BM25 discovery
• Policy tags on every tool from day one (READ_ONLY, FILE_SYSTEM, NETWORK, SLOW, experimental, stateful)
• Admin panel for live tool management (enable/disable/inspect)

fastmcp is no longer a dependency. No tool implementation code (tools/, domain/, remote/) was modified — provenance, HPC dispatch, session/workflow state, and fallback semantics are all unchanged.

What changed

File	Change
src/uxarray_mcp/registry.py	NEW — build_registry(profile=...) with namespace plan, policy tags, BM25 search hints, prompt-as-tool wiring
src/uxarray_mcp/server.py	Rewrite — make_registry(), make_mcp_server(), run(transport=...) replacing FastMCP
src/uxarray_mcp/__init__.py	Export make_registry/make_mcp_server instead of mcp
src/uxarray_mcp/cli.py	serve gains --profile, --transport, --host, --port
pyproject.toml	fastmcp → toolregistry + toolregistry-server[mcp]; added [openapi] optional extra
tests/test_server.py	Rewrite — 18 tests covering profile shape, tags, prompts, live call, MCP server construction
tests/test_vector_calc.py	2 tests adapted from from server import mcp to make_registry()
AGENTS.md	Full rewrite for new architecture
CHANGELOG.md	Added [Unreleased] section
CONTRIBUTING.md	fastmcp → toolregistry
conda/recipe/meta.yaml	Python 3.12, fastmcp → toolregistry deps
docs/conf.py	Removed fastmcp autodoc mock

Two profiles

core (default, 27 tools visible)

(top level)  11  — original gateway tools (get_capabilities, analyze_dataset, ...)
session/      8  — create_session, register_dataset, get/reset_session_state, ...
hpc/          4  — endpoint_status, get/set_execution_mode, validate_hpc_setup
io/           1  — list_datasets (read-only, no front-door equivalent)
prompt/       3  — first_look, vorticity_analysis, hpc_diagnose

export_to_netcdf/export_to_csv are reachable in core via run_analysis(operation="export", ...).

deferred-full (58 loaded, 28 visible + 30 deferred)

Core set stays visible. 30 raw implementation tools (compute/, shape/, inspect/, plot/, io/, agent/) loaded with defer=True. Agents find them via discover_tools (BM25 search). Operators promote from admin panel.

Prompt → tool conversion

The 3 @mcp.prompt() decorators are now regular tools under prompt/ namespace. They return the same instruction text as before — the LLM calls prompt-first_look(path="...") and gets a multi-step analysis plan to execute.

Backward compatibility

• uxarray-mcp serve still starts MCP stdio with the same default tool surface
• Claude Desktop mcpServers snippets work unchanged
• python -m uxarray_mcp still defaults to serve
• All 54 tool functions in uxarray_mcp.tools behave identically

New capabilities are opt-in:

uxarray-mcp serve --profile deferred-full
uxarray-mcp serve --transport sse --port 8001

Test results

301 passed, 3 failed (pre-existing upstream shapely bug), 5 skipped

The 3 failures (test_plot_mesh_healpix, test_plot_mesh_healpix_custom_size, test_analyze_dataset_includes_plots_by_default) reproduce identically on upstream main — they are a shapely GEOSException unrelated to this PR.

Design decisions (discussed via Slack)

• Core surface = current front-door + control/status + list_datasets + prompts per maintainer spec
• run_scientific_agent stays deferred/experimental until explicitly promoted
• export_to_* stays behind run_analysis dispatcher in core; direct calls in deferred
• Policy tags assigned based on function signatures (use_remote → NETWORK) and explicit overrides
• discover_tools default top_k controlled by the calling LLM; BM25 search hints added for domain synonyms (e.g. "vorticity" → calculate_curl)

How to try

pip install -e .
uxarray-mcp serve --help
# or:
python -c "from uxarray_mcp.server import make_registry; r = make_registry(); print(r.list_tools())"

[rajeeja]

Hey Peng, thanks so much for putting this together. Really nice that this removes the fastmcp dependency and goes to the Anthropic MCP SDK directly — cleaner dependency chain overall. Ran it locally end-to-end: 304 tests pass, both packages install cleanly from PyPI, admin panel defaults to localhost, and discover_tools is working nicely. Few things before we merge:

One thing that needs a decision:
The namespace prefixes show up on the MCP wire — Claude sees session-create_session, hpc-endpoint_status etc. instead of the flat names from before. For the 11 gateway tools that's fine (they're already flat), but the session/hpc tools are new names from the client's perspective. Does toolregistry have a way to keep namespaces internally but expose flat names over MCP? Or is the intent that clients just adapt to the prefixed names?

Smaller things:

• This feels like a 0.2.0 bump given we're replacing the server engine — what do you think?
• discover_tools("compute vorticity wind curl") returns prompt-vorticity_analysis at rank 1 and compute-calculate_curl at rank 2. The compute tool should win that query — probably just needs a BM25 hint tweak in registry.py.
• README and docs/remote-hpc.md still mention fastmcp in a couple spots — happy to clean that up in a follow-up if easier.
• How are you thinking about toolregistry-server longer term? If something needs patching on our end are you the right person to ping, or should we plan to fork it eventually?

[rljacob]

Is this targeted to work only with claude code?

[Oaklight]

Thanks for running it end-to-end and the detailed feedback. Pushed a follow-up commit addressing everything below.

Namespace prefixes on the wire:

Keeping the prefixed names (session-create_session, hpc-endpoint_status, etc.) is intentional — they give LLMs clearer context about what a tool does, and the admin panel / policy filters use the namespace to group and gate tools. That said, if a specific deployment needs flat names, toolregistry supports registering without a namespace — just pass namespace=None in registry.py. See: https://toolregistry.readthedocs.io/en/latest/usage/tool_management/

For now I'd leave the prefixes as-is and see if any client actually has trouble with them. LLMs handle hpc-endpoint_status just fine — it's arguably more self-documenting than bare endpoint_status.

Version bump:

Bumped to 0.2.0 in both pyproject.toml and __init__.py.

BM25 ranking fix:

Good catch. The issue was that prompt-vorticity_analysis had "vorticity", "wind", and "curl" all over its name + docstring, while calculate_curl's search hints were too narrow. Fixed two things:

• Enriched _SEARCH_HINTS for calculate_curl (and other compute tools) with the domain terms users actually type
• Rewrote the vorticity_analysis prompt tool's docstring to describe what it is (a plan generator) rather than echoing the same computation keywords

Before: prompt-vorticity_analysis rank 1 (26.3) > compute-calculate_curl rank 2 (24.3)
After: compute-calculate_curl rank 1 (26.1) > prompt-vorticity_analysis rank 4 (7.5)

Docs cleanup:

Cleaned up remaining fastmcp references in docs/architecture.md, execution_control.py, and remote_tools.py comments.

toolregistry-server maintenance:

I maintain the toolregistry ecosystem (toolregistry, toolregistry-server, toolregistry-hub) including PyPI releases and Docker images. For any issues or patches, open an issue or PR on the relevant repo — happy to discuss and review. No need to plan for a fork.

[Oaklight]

Is this targeted to work only with claude code?

Hey @rljacob Robert, good to see you here!

Not Claude-only — toolregistry-server is designed to expose the same tool set over multiple protocols from one process:

• MCP (stdio / SSE / streamable HTTP) — used by Claude Code, Codex CLI, OpenCode, and other MCP-compatible agent platforms
• OpenAPI / REST — used by cloud services, custom agent environments, chat UIs, or anything that can send HTTP requests (pip install uxarray-mcp[openapi])

Users pick whichever protocol fits their stack. The tool implementations, provenance, and HPC dispatch are shared across all transports.

[rajeeja]

Really appreciate the thoroughness here — the multi-protocol surface and policy tags are exactly what we need for non-Claude clients and access control, and the BM25 ranking fix landed well.

One thing I want to think through with you before we merge: toolregistry is now a hard dependency for basic stdio MCP. For users who just want uxarray-mcp serve with any MCP-compatible AI client, they'll see two unfamiliar packages before getting started — that could generate questions. FastMCP had broad name recognition in the MCP ecosystem; toolregistry is newer and less visible to the average user.

I'm also thinking out loud about longevity — discover_tools / deferred pool are genuinely useful now, but as models get better at handling large tool catalogs (already happening), those features may carry less weight over time. Multi-protocol and policy tags feel more durable.

Not asking you to roll back — more asking: is there a path where uxarray-mcp serve (default, no flags) stays on the raw mcp SDK for the simple case, and toolregistry features come in via an extra or a flag? That way we're not asking every new user to bet on the full stack, and it gives toolregistry room to build reputation before it's load-bearing for the defaults.

Happy to pair on what that would look like if it's useful.

[Oaklight]

Thanks for the call earlier, Rajeev — much easier to talk through this stuff live.

For anyone following along, the short version of what we discussed:

The features that are useful here — namespaces, policy tags, deferred/discovery, admin panel, multi-protocol — all live in toolregistry core. MCP is just one protocol adapter on top. If MCP's spec changes or something better comes along, the adapter swaps out and the core stays. The three packages (core, server, hub) are split by release cadence, not by ownership — they're maintained together with cross-repo compatibility CI on every release.

On the "models will get better at large tool lists" point: progressive disclosure and runtime discover_tools aren't workarounds for weak models — they're how you manage a growing tool surface responsibly. As models improve, the value of structured discovery goes up, not down, because you can expose more tools with confidence that the right one gets found.

On the dependency concern: the alternative to depending on toolregistry isn't "no dependency" — it's reimplementing the same namespace/tag/discovery/profile machinery inside this repo. That's a real ongoing maintenance cost for features that already exist and are tested.

That said, this is your project. The PR is here and working (CI green, 301 tests pass). Happy to answer questions from anyone on the team — feel free to loop me in.

[Oaklight]

On the conda question — toolregistry and toolregistry-server are PyPI-only right now. No conda-forge feedstock yet.

That means the current conda/recipe/meta.yaml in this PR won't solve cleanly in a pure conda environment — conda doesn't pull from PyPI.

Two paths forward:

1. Short term: add pip as a run dependency in the conda recipe and have a post-install step do pip install toolregistry-server[mcp]. Not ideal by conda-forge standards but unblocks conda users immediately. Up to you whether that's acceptable for your user base.
2. Proper fix: I submit conda-forge feedstocks for toolregistry and toolregistry-server. That takes a PR review cycle on conda-forge (usually a few days). Once they're in, the recipe just lists them as normal conda deps and everything solves cleanly.

I'll get started on the feedstocks. In the meantime, if your conda users need to try this branch, pip install toolregistry-server[mcp] inside a conda env works fine — it's pure Python, no compiled extensions.

[Copilot]

Pull request overview

Replaces the existing FastMCP-based MCP server entrypoint with a ToolRegistry/toolregistry-server based implementation, enabling namespaced tool grouping, profile-based tool visibility (core vs deferred-full), and policy tagging while keeping the underlying tool implementations intact.

Changes:

• Added build_registry(profile=...) to define the tool surface (namespaces, profiles, prompt-as-tool, tags, deferred pool).
• Rewrote server startup to construct/run an MCP server via toolregistry-server transports (stdio/SSE/HTTP) and updated CLI flags accordingly.
• Updated dependency metadata and rewrote/expanded server-related tests and docs to match the new architecture.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/uxarray_mcp/registry.py	New registry builder defining namespaces, profiles, deferred tools, prompt-as-tool helpers, and policy tags.
src/uxarray_mcp/server.py	Replaced FastMCP wiring with ToolRegistry + toolregistry-server MCP server creation and multi-transport run().
src/uxarray_mcp/cli.py	serve command now supports --profile, --transport, --host, --port and calls server.run().
src/uxarray_mcp/__init__.py	Exports make_registry/make_mcp_server and bumps version to 0.2.0.
src/uxarray_mcp/tools/remote_tools.py	Updated comment to remove FastMCP reference.
src/uxarray_mcp/tools/execution_control.py	Updated docstring to remove FastMCP reference.
tests/test_server.py	Rewritten to validate registry profiles, bucket coverage, tags, prompt tools, live call, and MCP server construction.
tests/test_vector_calc.py	Adapted tests to use make_registry() and prompt tools as namespaced tools.
pyproject.toml	Dependency swap to toolregistry-server, adds openapi extra, bumps version and mypy python_version.
docs/conf.py	Removes fastmcp autodoc mock import.
docs/architecture.md	Updates architecture layer labeling to ToolRegistry server and removes FastMCP wording.
CONTRIBUTING.md	Updates guidance to reflect toolregistry-based server dependency boundaries.
conda/recipe/meta.yaml	Updates Python min/version and dependency from fastmcp to toolregistry-server.
CHANGELOG.md	Adds Unreleased notes describing the server engine rewrite and new capabilities.
AGENTS.md	Rewritten agent guidance reflecting the new registry/server architecture and profiles.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[rajeeja]

@Oaklight uv.lock stale, and if possible, try to address the co-pilot comments. If not, I can create a future PR. Thanks so much for this contribution.