Replace FastMCP with toolregistry-server: multi-protocol, profiles, policy tags

AGENTS.md

@@ -4,42 +4,93 @@ This file provides guidance for AI coding agents working on this repository.
 
 ## Project overview
 
-**uxarray-mcp-server** is an MCP (Model Context Protocol) server that exposes
+**uxarray-mcp-server** is a multi-protocol server that exposes
 [UXarray](https://uxarray.readthedocs.io/) — a library for working with
-unstructured climate and atmospheric meshes — to AI agents such as Claude.
+unstructured climate and atmospheric meshes — to AI agents and HTTP
+clients.  It is powered by
+[toolregistry](https://github.com/Oaklight/ToolRegistry) +
+[toolregistry-server](https://github.com/Oaklight/toolregistry-server),
+which provide:
+
+- **MCP** (stdio / SSE / streamable HTTP) for Claude Desktop, Claude
+  Code, and any MCP-compatible client.
+- **OpenAPI / REST** (optional extra) for curl, OpenAI Assistants,
+  Anthropic Messages API, Gemini, LangChain, plain scripts.
+- **Python API** — `from uxarray_mcp.server import make_registry`.
 
 It supports local execution and optional remote execution on HPC clusters via
 [Globus Compute](https://globus-compute.readthedocs.io/).
 
-Public MCP tools: `get_capabilities`, `analyze_dataset`, `run_analysis`,
-`plot_dataset`, `diagnose_endpoint`, `probe_path_access`, `run_workflow`,
-`resume_workflow`, `get_status`, `get_result`, and `manage_session`.
+### Tool surface — two profiles
+
+The tool surface is built by `uxarray_mcp.registry.build_registry()`:
+
+- **`core` (default, ~27 tools)** — 11 front-door gateway tools at the
+  top level, 12 control/status tools under `session/` and `hpc/`
+  namespaces, `io-list_datasets`, and 3 prompt helpers under `prompt/`.
+  No deferred tools, no BM25 discovery.  This is what clients see by
+  default when running `uxarray-mcp serve`.
+
+- **`deferred-full` (~58 loaded, ~28 visible)** — core set stays visible.
+  30 raw implementation tools are loaded with `defer=True` so they don't
+  appear in the initial tool list.  Agents discover them via
+  `discover_tools` (BM25 search), and operators promote them from the
+  admin panel.
+
+### Core tools
+
+Front-door / gateway (top level): `get_capabilities`, `analyze_dataset`,
+`run_analysis`, `plot_dataset`, `diagnose_endpoint`, `probe_path_access`,
+`run_workflow`, `resume_workflow`, `get_status`, `get_result`,
+`manage_session`.
+
+Session state (`session/`): `create_session`, `register_dataset`,
+`get_session_state`, `reset_session_state`, `get_result_handle`,
+`get_operation_status`, `list_operations`, `get_workflow_status`.
+
+HPC control (`hpc/`): `endpoint_status`, `get_execution_mode`,
+`set_execution_mode`, `validate_hpc_setup`.
+
+IO (`io/`): `list_datasets`.
+
+Prompt helpers (`prompt/`): `first_look`, `vorticity_analysis`,
+`hpc_diagnose`.
 
 Low-level implementation functions such as `inspect_mesh`, `calculate_area`,
-`plot_mesh`, and `endpoint_status` remain importable from `uxarray_mcp.tools`
-for tests, scripts, and internal composition, but they are not registered as
-individual MCP tools.
+`plot_mesh`, and `calculate_curl` remain importable from `uxarray_mcp.tools`
+for tests, scripts, and internal composition.  In `deferred-full` profile
+they are available through `discover_tools`; in `core` they are not
+registered.
 
 Documentation: `docs/` (Sphinx, built to ReadTheDocs).
 
 ## Key design decisions
 
-- **Domain/tool separation** — pure computation lives in `domain/`, MCP wiring
-  lives in `tools/`. The same domain functions run locally or get serialized
-  and sent to an HPC worker via Globus Compute. Never put MCP, provenance, or
-  I/O logic in `domain/`.
+- **Domain/tool separation** — pure computation lives in `domain/`, server
+  wiring lives in `registry.py` + `server.py`.  The same domain functions
+  run locally or get serialized and sent to an HPC worker via Globus
+  Compute.  Never put MCP, provenance, or I/O logic in `domain/`.
+
+- **Two-profile surface** — the core profile keeps a small, predictable
+  baseline for existing clients.  New tools start in `deferred-full` and
+  get promoted when they are stable, commonly useful, documented, and
+  have clear provenance/security behavior.
+
+- **Policy tags from day one** — every tool carries `ToolTag` values
+  (`READ_ONLY`, `FILE_SYSTEM`, `NETWORK`, `SLOW`) and custom tags
+  (`experimental`, `stateful`) so downstream policy code (admin filters,
+  auth gates, audit logs) has concrete metadata.
 
-- **Small MCP front-door surface** — MCP clients see intent-shaped tools, not
-  every implementation function. Route new analysis behavior through
-  `run_analysis`, plotting through `plot_dataset`, endpoint diagnostics through
-  `diagnose_endpoint`, and session state through `manage_session` /
-  `get_status` / `get_result`.
+- **Prompt-as-tool** — the former `@mcp.prompt()` decorators (`first_look`,
+  `vorticity_analysis`, `hpc_diagnose`) are now regular tools under the
+  `prompt/` namespace.  They return instruction text that guides the LLM
+  through a multi-step analysis.  This removes the `fastmcp` dependency.
 
-- **Single remote execution path** — there are no separate `*_hpc` tools. The
-  dispatchers accept `use_remote` and `endpoint` where remote execution is
-  meaningful. When `use_remote=True` and an endpoint is unavailable, tools either
-  fall back locally when the path is local/readable or raise a clear endpoint
-  readiness error.
+- **Single remote execution path** — there are no separate `*_hpc` tools.
+  The dispatchers accept `use_remote` and `endpoint` where remote execution
+  is meaningful.  When `use_remote=True` and an endpoint is unavailable,
+  tools either fall back locally when the path is local/readable or raise a
+  clear endpoint readiness error.
 
 - **Provenance on everything** — every tool result must pass through
   `attach_provenance()`. No tool should return a dict without `_provenance`.
@@ -67,27 +118,30 @@ Documentation: `docs/` (Sphinx, built to ReadTheDocs).
 
 ```
 src/uxarray_mcp/
-  server.py                 # FastMCP server — small public front-door surface
-  provenance.py             # attach_provenance() used by all tools
-  domain/                   # Pure computation — no MCP, no I/O
-    mesh.py                 # Grid loading, HEALPix support
-    area.py                 # Face area statistics
-    variable.py             # Variable metadata and stats
-    zonal.py                # Zonal mean computation
-    plotting.py             # render_mesh, render_variable, render_zonal_mean
-  remote/                   # HPC execution layer
-    config.py               # HPCConfig, load_config()
-    agent.py                # UXarrayComputeAgent (Academy + Globus Compute)
-    compute_functions.py    # Self-contained remote functions (no uxarray_mcp imports)
-    health.py               # cached endpoint status + worker probes
-  tools/                    # MCP tool functions
-    frontdoor.py            # Public MCP dispatch tools (run_analysis, plot_dataset, etc.)
-    inspection.py           # Core local implementation functions
-    plotting.py             # Visualization implementation functions
-    remote_tools.py         # HPC-enabled implementation wrappers
-    execution_control.py    # Endpoint diagnostics and mode/config helpers
-    capabilities.py         # get_capabilities — tool discovery
+  registry.py             # build_registry() — namespace plan, tags, prompt-as-tool
+  server.py               # make_registry(), make_mcp_server(), run() — multi-transport
+  cli.py                  # uxarray-mcp serve/setup/doctor/endpoints/install-claude
+  provenance.py           # attach_provenance() used by all tools
+  domain/                 # Pure computation — no MCP, no I/O
+    mesh.py               # Grid loading, HEALPix support
+    area.py               # Face area statistics
+    variable.py           # Variable metadata and stats
+    zonal.py              # Zonal mean computation
+    plotting.py           # render_mesh, render_variable, render_zonal_mean
+  remote/                 # HPC execution layer
+    config.py             # HPCConfig, load_config()
+    agent.py              # UXarrayComputeAgent (Academy + Globus Compute)
+    compute_functions.py  # Self-contained remote functions (no uxarray_mcp imports)
+    health.py             # cached endpoint status + worker probes
+  tools/                  # Tool implementations
+    frontdoor.py          # Public dispatch tools (run_analysis, plot_dataset, etc.)
+    inspection.py         # Core local implementation functions
+    plotting.py           # Visualization implementation functions
+    remote_tools.py       # HPC-enabled implementation wrappers
+    execution_control.py  # Endpoint diagnostics and mode/config helpers
+    capabilities.py       # get_capabilities — tool discovery
 tests/
+  test_server.py          # Registry profile shape, tags, prompts, live calls
   test_inspect_mesh.py
   test_inspect_variable.py
   test_calculate_area.py
@@ -97,32 +151,34 @@ tests/
   test_capabilities.py
   test_scientific_agent.py
   test_execution_control.py
-  test_hpc_safety.py        # Pre-flight + fallback (mocked Globus SDK)
-  test_remote_agent.py      # Academy agent tests (requires hpc extra)
-  test_server.py            # Tool registration verification
-docs/                       # Sphinx documentation (MyST Markdown + RST)
-  release.md                # Release automation notes (PyPI + conda-forge)
+  test_hpc_safety.py      # Pre-flight + fallback (mocked Globus SDK)
+  test_remote_agent.py    # Academy agent tests (requires hpc extra)
+evals/                    # BM25 tool retrieval + schema rejection regression
+docs/                     # Sphinx documentation (MyST Markdown + RST)
+  release.md              # Release automation notes (PyPI + conda-forge)
 scripts/
-  hpc_doctor.py             # CLI diagnostic tool (also exposed as ``uxarray-mcp doctor``)
-  improv_endpoint.sh        # Argonne Improv endpoint setup + Python 3.12 upgrade
-  ucar_endpoint.sh          # NCAR/Casper (UCAR) endpoint setup
-  chrysalis_endpoint.sh     # Argonne Chrysalis endpoint setup
-  hpc_build_yac.py          # Build YAC + YAXT on a Globus Compute worker
-  yac_smoke_test.py         # Verify worker-side YAC import + basic surface
-  agentic_hpc_loop.py       # Example HPC workflow script
-conda/recipe/meta.yaml      # Seed recipe for conda-forge feedstock
-config.yaml.example         # Template — private config is normally written by the CLI
+  hpc_doctor.py           # CLI diagnostic tool (also exposed as ``uxarray-mcp doctor``)
+  improv_endpoint.sh      # Argonne Improv endpoint setup + Python 3.12 upgrade
+  ucar_endpoint.sh        # NCAR/Casper (UCAR) endpoint setup
+  chrysalis_endpoint.sh   # Argonne Chrysalis endpoint setup
+  hpc_build_yac.py        # Build YAC + YAXT on a Globus Compute worker
+  yac_smoke_test.py       # Verify worker-side YAC import + basic surface
+  agentic_hpc_loop.py     # Example HPC workflow script
+conda/recipe/meta.yaml    # Seed recipe for conda-forge feedstock
+config.yaml.example       # Template — private config is normally written by the CLI
 ```
 
 ## Tech stack
 
-- **Python** ≥ 3.11
-- **FastMCP** ≥ 3.4.0 — MCP server framework
+- **Python** ≥ 3.12, < 3.13 (pinned for Globus Compute pickle compat)
+- **toolregistry** ≥ 0.11.0 — tool registration, schema generation, policy tags
+- **toolregistry-server** ≥ 0.3.3 — MCP + OpenAPI adapters
 - **UXarray** ≥ 2025.12.0 — unstructured mesh analysis
 - **Matplotlib** ≥ 3.9.0 + **Holoviews** ≥ 1.19.0 — visualization
 - **PyYAML** ≥ 6.0 — config file parsing
 - **uv** — package management and script runner (not conda, not pip directly)
 - Optional HPC: `globus-compute-sdk` ≥ 4.5.0, `academy-py` ≥ 0.3.1
+- Optional REST: `toolregistry-server[openapi]`
 
 ## Code style
 
@@ -134,7 +190,7 @@ config.yaml.example         # Template — private config is normally written by
 - **Imports**: sorted by ruff/isort. First-party = `uxarray_mcp`.
 - Comments should explain *why*, not *what*.
 - Use `from __future__ import annotations` in files that use `X | Y` syntax,
-  since Python 3.11 is the minimum and PEP 604 union syntax needs it there.
+  since Python 3.12 is the minimum.
 
 All checks are enforced via pre-commit — **every commit must pass
 `uv run pre-commit run --all-files`**.
@@ -161,11 +217,16 @@ uv run pytest tests/test_remote_agent.py tests/test_hpc_safety.py -v
 ```
 
 When to add tests:
-- Any new public MCP tool — register it in `server.py`, export it from
-  `tools/__init__.py`, document it in `docs/tools.md`, and add tests.
+- Any new tool — add it to the appropriate bucket in `registry.py`
+  (`_CONTROL_TOOLS`, `_CORE_EXTRA_TOOLS`, or `_DEFERRED_TOOLS`), export
+  it from `tools/__init__.py`, document it in `docs/tools.md`, and add
+  tests.  The `test_namespace_plan_covers_every_public_tool` test will
+  fail if a tool in `__all__` is not assigned to any bucket.
 - Any new implementation operation — prefer adding it behind an existing
   front-door tool (`run_analysis`, `plot_dataset`, `diagnose_endpoint`, or
-  `manage_session`) unless there is a strong reason for a new public MCP tool.
+  `manage_session`) unless there is a strong reason for a new public tool.
+  New operations start in `_DEFERRED_TOOLS` and graduate to core via the
+  promotion path.
 - Any new error path — especially file-not-found and empty-file guards.
 - Any bug fix — add a test that would have caught it.
 
@@ -219,13 +280,14 @@ regenerate `uv.lock`.
 
 ## Common mistakes to avoid
 
-- Importing `mcp` or `fastmcp` in `domain/` — domain functions must be
-  importable without MCP installed (they run on the remote worker).
+- Importing `mcp` or `toolregistry` in `domain/` — domain functions must be
+  importable without server dependencies installed (they run on the remote
+  worker).
 - Returning a plain dict from a tool without calling `attach_provenance()`.
-- Adding a new public MCP tool when an existing front-door operation would do.
-- Forgetting to register an intentional new public MCP tool with `mcp.tool()` in
-  `server.py`.
-- Forgetting to export a new tool from `tools/__init__.py`.
+- Adding a new tool to `tools/__init__.__all__` without assigning it to a
+  bucket in `registry.py` — the coverage test will catch this.
+- Adding a deferred tool without a `search_hint` in `_SEARCH_HINTS` —
+  BM25 discovery works much better with domain synonyms.
 - Using `/home/...` paths on Improv when the file actually lives under
   `/gpfs/fs1/home/...` — check `probe_path_access` first on a new cluster.
 - Adding a `local import io` inside a function when `io` is used for testable

CHANGELOG.md

@@ -3,6 +3,35 @@
 All notable changes are recorded here. Dates are ISO 8601 (UTC). The project
 uses Semantic Versioning for public releases.
 
+## Unreleased
+
+### Changed
+- **Server engine**: replaced FastMCP with
+  [toolregistry](https://github.com/Oaklight/ToolRegistry) +
+  [toolregistry-server](https://github.com/Oaklight/toolregistry-server).
+  `fastmcp` is no longer a dependency.
+- **Two-profile tool surface**: `core` (~27 tools, conservative default) and
+  `deferred-full` (all tools loaded, 30 deferred behind BM25 discovery).
+- **Namespace grouping**: control tools under `session/` and `hpc/`, IO under
+  `io/`, prompts under `prompt/`.
+- **Policy tags**: every tool carries `ToolTag` metadata (`READ_ONLY`,
+  `FILE_SYSTEM`, `NETWORK`, `SLOW`) and custom tags (`experimental`,
+  `stateful`) from day one.
+
+### Added
+- `src/uxarray_mcp/registry.py` — `build_registry(profile=...)` with namespace
+  plan, policy tags, BM25 search hints, and prompt-as-tool wiring.
+- Prompt-as-tool: `first_look`, `vorticity_analysis`, `hpc_diagnose` (formerly
+  `@mcp.prompt()` decorators) are now regular tools under `prompt/` namespace.
+- CLI: `uxarray-mcp serve` now accepts `--profile`, `--transport`, `--host`,
+  `--port`.
+- Multi-transport MCP: stdio (default), SSE, streamable HTTP.
+- Optional OpenAPI/REST surface via `pip install uxarray-mcp[openapi]`.
+
+### Removed
+- `fastmcp` dependency.
+- `@mcp.prompt()` decorators (replaced by `prompt/` namespace tools).
+
 ## 0.1.0 — 2026-06-04
 
 Initial public release.

CONTRIBUTING.md

@@ -60,8 +60,8 @@ CI must be green before a PR is merged.
 - Type checker: `mypy` (enforced by pre-commit).
 - Annotate all new public functions.
 - Comments explain *why*, not *what*.
-- No `domain/` imports of `mcp` or `fastmcp` — domain functions run on remote
-  HPC workers that do not have `uxarray_mcp` installed.
+- No `domain/` imports of `mcp` or `toolregistry` — domain functions run on
+  remote HPC workers that do not have `uxarray_mcp` installed.
 
 ## Adding Dependencies
 

conda/recipe/meta.yaml

@@ -1,6 +1,6 @@
 {% set name = "uxarray-mcp" %}
-{% set version = "0.1.1" %}
-{% set python_min = "3.11" %}
+{% set version = "0.2.0" %}
+{% set python_min = "3.12" %}
 
 package:
   name: {{ name }}
@@ -27,7 +27,7 @@ requirements:
     # a second output/variant after globus-compute-sdk and academy-py solver
     # behavior is validated on conda-forge.
     - python >={{ python_min }}
-    - fastmcp >=3.4.0
+    - toolregistry-server >=0.3.3
     - holoviews >=1.19.0
     - matplotlib-base >=3.9.0
     - pyyaml >=6.0

docs/architecture.md

@@ -11,8 +11,8 @@ The UXarray MCP Server is organized into three layers:
   </div>
   <div class="arch-arrow">↓</div>
   <div class="arch-box">
-    <strong>FastMCP Server</strong><br>
-    Registers tools from <code>uxarray_mcp.server</code> and exposes them to the client
+    <strong>ToolRegistry Server</strong><br>
+    Registers tools via <code>uxarray_mcp.registry</code> and exposes them over MCP / OpenAPI
   </div>
   <div class="arch-arrow">↓</div>
   <div class="arch-box">
@@ -77,7 +77,7 @@ transport itself. Globus Compute is still the actual remote execution system.
 
 ## Tools Layer (`tools/`)
 
-The FastMCP server exposes a small front-door tool surface. Those tools route
+The server exposes a small front-door tool surface. Those tools route
 to lower-level implementation functions in `tools/`, which handle input
 validation, domain calls, remote dispatch, provenance, and structured results.
 

docs/conf.py

@@ -84,4 +84,4 @@
 
 autodoc_member_order = "bysource"
 autodoc_typehints = "description"
-autodoc_mock_imports = ["fastmcp"]
+autodoc_mock_imports = []

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "uxarray-mcp"
-version = "0.1.1"
+version = "0.2.0"
 description = "MCP server for analyzing unstructured meshes with UXarray"
 readme = "README.md"
 keywords = ["uxarray", "mcp", "unstructured grids", "scientific computing", "globus compute"]
@@ -25,14 +25,17 @@ license = { file = "LICENSE" }
 # support and broaden requires-python back to >=3.11.
 requires-python = ">=3.12,<3.13"
 dependencies = [
-    "fastmcp>=3.4.0",
+    "toolregistry-server[mcp]>=0.3.3",
     "holoviews>=1.19.0",
     "matplotlib>=3.9.0",
     "pyyaml>=6.0",
     "uxarray>=2025.12.0",
 ]
 
 [project.optional-dependencies]
+openapi = [
+    "toolregistry-server[openapi]>=0.3.3",
+]
 hpc = [
     "academy-py>=0.3.1",
     "globus-compute-sdk>=4.5.0",
@@ -61,7 +64,7 @@ dev = [
 ]
 
 [tool.mypy]
-python_version = "3.11"
+python_version = "3.12"
 check_untyped_defs = true
 ignore_missing_imports = true
 warn_unused_ignores = true