Skip to content

feat(altimate-code): experimental direct MCP data tools (Claude reasons, altimate-code supplies tools) - #6

Draft
anandgupta42 wants to merge 14 commits into
mainfrom
feat/toolkit-mode
Draft

anandgupta42 wants to merge 14 commits into
mainfrom
feat/toolkit-mode

Conversation

@anandgupta42

@anandgupta42 anandgupta42 commented Oct 17, 2026 •

Copy link
Copy Markdown

Experimental — draft, not for merge. It needs an altimate-code build with altimate-code mcp serve (AltimateAI/altimate-code#1543, also a draft experiment).

Why

Today this plugin hands each data task to altimate-code run. A second model does the work, and Claude Code's own reasoning loop is bypassed. This change lets Claude Code use altimate-code's data tools directly instead. Claude's model does the reasoning, so the user's Claude Code plan covers the LLM cost. altimate-code supplies read-only tools and runs no model.

What changed

MCP server. plugins/altimate-code/.mcp.json starts altimate-code mcp serve. That gives Claude Code five tools:

  • warehouse_list
  • schema_inspect, with a value profile
  • sql_execute, read-only by default
  • sql_check
  • sql_materialize, which copies data into a session-local scratch DuckDB for joins across databases

Skills.

  • New data-question skill (24 lines, generic). Inspect value profiles before choosing columns or join keys, compute in the database, materialize the small side for joins across connections, and check keys and fan-out when they affect the answer.
  • The altimate-code routing skill now answers data questions with the tools. It keeps altimate-code run delegation for long multi-step jobs such as dbt builds and migrations. The SessionStart hook and /altimate command text match.

Version and docs.

  • Version 1.1.0-experimental.
  • The README and CHANGELOG describe the requirement and the read-only limits. Read-only is enforced by the database on Postgres, SQLite and DuckDB. Elsewhere it relies on SQL classification only, so use a read-only database user.

How tested

  • Static: JSON parsing, YAML front matter, bash -n and ShellCheck on the hook script, the plugin validators, and git diff --check.
  • Live, with Claude Code 2.1.302 on Sonnet. I loaded the plugin from this branch with --plugin-dir. altimate-code on PATH was a branch build of #1543. Bash and web tools were disabled. The data was a 60-row SQLite orders table, found through project connection discovery (.altimate-code/connections.json).
    • The plugin loaded as 1.1.0-experimental, and the data-question skill was available.
    • The server plugin:altimate-code:altimate-code connected.
    • Claude answered through warehouse_list and sql_execute, and the answer was correct: cust4 and cust2 tie with 5 paid orders, and the paid orders total 12,555. The cost was $0.07.
  • Gap found: Claude first called schema_inspect with only a warehouse, to list its tables, and the tool rejected the call because table is required. A table-list mode for schema_inspect is a follow-up on the altimate-code side.
  • Not run: remote warehouses through the plugin, and the delegation path for long jobs. That path is unchanged in behaviour apart from the routing text.

Update: analyst agent and answer_commit

  • New agents/analyst.md: the analyst method as a plugin agent. Start with claude --agent altimate-code:analyst (or set it as the agent setting) to make it the main agent. It finishes with the MCP answer_commit tool and replies with the returned FINAL ANSWER: text.
  • The data-question skill's last step now uses answer_commit.
  • Needs the MCP server from feat(mcp): altimate-code mcp serve exposes data tools to MCP clients without an LLM altimate-code#1543 (sixth tool answer_commit).
  • Codex read-only review: no P0/P1.
  • Measured on a failing-question set (18 questions x 5, Claude Code subscription, same model): plain 58/90, plugin only 56/90, plugin + analyst agent + answer_commit 58/90. All 90 trials ended with an accepted commit (14 rejected commits). Gain is limited to questions where the agent changed answers; it is not a clear win overall.

🤖 Generated with Claude Code

Update 2026-10-21 (head e378906)

Shrinks the text the plugin adds to every request, one commit per piece:

  • 40a4fe2: agent, skills and docs point at the five default MCP tools (warehouse_list, warehouse_add, sql_execute with tables, sql_materialize, answer_commit).
  • b59bb98: drop the data-question skill (duplicate of the analyst agent and the server instructions; never loaded).
  • 2b53d88: shorter review and routing skill descriptions (bodies unchanged).
  • 7abc72f: analyst agent 3,798 -> 988 chars, keeping the answer-shape rules (one field per requested item, entity and related item separate) and the value/readings/policy checks.
  • e378906: SessionStart text 537 -> 215 chars.

Local Codex review at e378906: no P0/P1. Open follow-ups: restore the dbt_project.yml qualifier in the review skill description (P2) and repoint CONTRIBUTING.md from the removed skill (P3).
Internal eval (5 trials per question, 12 questions): accuracy held with these cuts; first request ~4.5k tokens smaller, most of it from running without the Skill tool.

Update 2026-10-21 (head 8e3ae4e)

Update 2026-10-21 (head bce15b1)

  • b63baea: the analyst answers in plain text: one FINAL ANSWER: line giving every requested item in order (an entity and its related item both); answer_commit is optional.
  • bce15b1: the analyst starts from the connection list the server now sends at startup; warehouse_list refreshes it.
  • Local Codex review at bce15b1: no findings.

Update 2026-10-21: own task set on a real warehouse (head bce15b1, no change)

  • 10 questions of our own on BigQuery plus Postgres, 3 trials each, plain Claude Code against this plugin: accuracy 30/30 against 30/30, cost per run 17% lower with the plugin, BigQuery bytes billed 68% lower and median time at parity. With no connections configured, the analyst added them with warehouse_add in 4 of 9 runs and used Bash clients in the other 5 (all passed).

anandgupta42 and others added 5 commits October 17, 2026 13:11
…mode routing

- `.mcp.json` starts `altimate-code mcp serve`, so Claude Code calls
  altimate-code's read-only data tools (warehouse_list, schema_inspect,
  sql_execute, sql_check, sql_materialize) directly; Claude's own model does
  the reasoning and no second LLM runs.
- New lean `data-question` skill: inspect value profiles before choosing
  columns or keys, compute in the database, materialize the small side for
  cross-database joins, check keys/fan-out when they matter.
- The `altimate-code` routing skill now answers data questions with the tools
  and keeps `altimate-code run` delegation for long multi-step jobs; the
  SessionStart hook and `/altimate` text match.
- Version 1.1.0-experimental; README and CHANGELOG note it needs an
  altimate-code build with `mcp serve` and describe the read-only limits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… triggers

- Data questions prefer the altimate-code MCP tools and fall back to
  `altimate-code run` (analyst) when the server is missing or fails to start,
  telling the user which path was used.
- Restore delegation triggers for lineage, impact, schema diff, parity, PII,
  cost attribution, optimization, SCD2/incremental checks, docs and diffs.
- PRIVACY.md describes the local MCP server and that tool results reach the
  model provider; `sql_check` engines and materialize-first noted.
- Delegation here-docs use a per-call delimiter checked against the task text.
- CONTRIBUTING.md covers `.mcp.json` and the `data-question` skill.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… skill triggers

- SessionStart output now carries compact routing (MCP tools for data
  questions, analyst/fast-edit/builder delegation otherwise).
- Skill descriptions trigger on ordinary data questions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The MCP server's default toolset is now `warehouse_list`, `warehouse_add`, `sql_execute` (with `tables` for schemas and value profiles), `sql_materialize` and `answer_commit`; `schema_inspect` and `sql_check` need `mcp serve --all-tools`. The analyst agent, skills, READMEs and changelog no longer tell the model to call the hidden tools, and a missing connection is added with `warehouse_add`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anandgupta42 and others added 9 commits October 20, 2026 16:52
…instructions carry its method

The skill duplicated the analyst agent and the MCP server instructions, and its listing was sent with every request (no session loaded it). The routing skill, /altimate and README now point at the server instructions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Skill descriptions are listed in every request; the bodies load only on use. review 1,278 -> 268 chars, altimate-code 440 -> 214 chars. Bodies unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agent prompt is re-read on every request. Keep: warehouse_add for a missing connection, value check via `tables`, every condition must filter, two readings into `readings`, policy conditions against the row, and the answer_commit finish (one field per requested item, entity and related item separate, FINAL ANSWER reply, recommit on rejection). Method steps already in the server instructions are no longer repeated. Body 3,798 -> 988 chars.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hook runs once per session, but its text stays in the context and is re-read on every request. One sentence (537 -> 215 chars) keeps the routing: MCP tools for data questions, `altimate-code run` delegation otherwise, /review for dbt/SQL PRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…call

The MCP server now lets a later batch step use an earlier step's rows (`{{label.column}}`, `{{label}}`); one line in the agent points at it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The server now expands `{{label.column}}` to bare comma-separated values, so the example reads `IN ({{label.column}})`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ional

The analyst now ends with one `FINAL ANSWER:` line giving every requested item in order (an entity and its related item both), which saves the commit round trip. `answer_commit` stays available.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…artup

The MCP server now lists connections (and small local catalogs) in its instructions, so the analyst starts without a `warehouse_list` call; `warehouse_list` refreshes them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant