feat(code-mode): lazy tool support (progressive disclosure)#726
Open
AlemTuzlak wants to merge 6 commits into
Open
feat(code-mode): lazy tool support (progressive disclosure)#726AlemTuzlak wants to merge 6 commits into
AlemTuzlak wants to merge 6 commits into
Conversation
…discovery
Introduces a single optional LazyToolsConfig ({ includeDescription:
'none' | 'first-sentence' | 'full' }, default 'none') plus shared
renderLazyCatalogEntry/firstSentence helpers. chat() now accepts
lazyToolsConfig and threads it into LazyToolManager, which renders the
discovery-tool catalog accordingly. The 'none' default is byte-identical
to the previous names-only output.
Tools marked `lazy: true` are kept out of the execute_typescript system
prompt (their type stubs omitted) and listed in a "Discoverable APIs"
catalog instead. A new discover_tools companion tool returns each lazy
tool signature on demand; lookups tolerate the optional external_ prefix
so the model can pass the catalog name verbatim. All tool bindings are
still injected into the sandbox (documentation-only lazy). createCodeMode
now returns { tool, discoveryTool, tools, systemPrompt } (additive;
tool/systemPrompt unchanged) and honors the shared lazyToolsConfig.
Adds a wire-journal E2E (api.lazy-tools-wire route + fixture + spec) that asserts the lazy discovery tool's wire description renders names-only for 'none', name plus first sentence for 'first-sentence', and the full description for 'full'. routeTree.gen.ts is the router plugin's auto-regen for the new route.
New docs/code-mode/lazy-tools.md (with config.json nav entry) covering lazy tools, the discover_tools flow, and lazyToolsConfig.includeDescription for both Code Mode and plain chat(). Updates the ai-code-mode and ai-core/tool-calling agent skills to document the new API, and adds the changeset (minor for @tanstack/ai and @tanstack/ai-code-mode).
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (7)
✅ Files skipped from review due to trivial changes (3)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthroughAdds “lazy tool” progressive disclosure: lazy tools are omitted from initial prompts, appear in a discoverable catalog whose verbosity is configurable via ChangesLazy Tools Feature
Sequence Diagram (high-level discovery flow) sequenceDiagram
participant Model
participant discover_tools
participant execute_typescript
Model->>discover_tools: discover_tools({ toolNames })
discover_tools-->>Model: tools[{ name, description, typeStub }]
Model->>execute_typescript: call external_<tool>(...)
execute_typescript-->>Model: execution result
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
testing/e2e/tests/lazy-tools-wire.spec.tsParsing error: "parserOptions.project" has been provided for Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🚀 Changeset Version Preview4 package(s) bumped directly, 27 bumped as dependents. 🟥 Major bumps
🟨 Minor bumps
🟩 Patch bumps
|
|
View your CI Pipeline Execution ↗ for commit bab6ada
💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗ ☁️ Nx Cloud last updated this comment at |
@tanstack/ai
@tanstack/ai-anthropic
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-mcp
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
The chat lazy-tool discovery page now documents the new optional
lazyToolsConfig.includeDescription ('none' default / 'first-sentence' /
'full') that tunes the pre-discovery catalog, with a cross-link to the
Code Mode lazy tools page. Bumps the example model ids to gpt-5.2 and
sets updatedAt on the docs config entry.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (4)
packages/ai/tests/lazy-tools.test.ts (1)
1-61: ⚡ Quick winCo-locate this unit test with the source module.
This test should live next to the lazy-tools implementation per repo guideline (for example under
packages/ai/src/activities/chat/tools/).As per coding guidelines,
**/*.test.ts: Place unit tests alongside source code in*.test.tsfiles.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai/tests/lazy-tools.test.ts` around lines 1 - 61, The tests for firstSentence and renderLazyCatalogEntry are currently in the shared tests folder and must be co-located with the lazy-tools source module; move the lazy-tools.test.ts file that contains tests referencing firstSentence and renderLazyCatalogEntry into the same directory as the lazy-tools implementation (next to the module that exports those functions), keep the filename ending in .test.ts, and update the import paths in the test to use the local module import so the tests run alongside the source per repo guidelines.Source: Coding guidelines
packages/ai-code-mode/skills/ai-code-mode/SKILL.md (1)
383-388: 💤 Low valueAdd language identifier to fenced code block.
Markdown linters require language identifiers on fenced code blocks. Since this is a flow/sequence diagram rather than executable code, consider using
textor remove the fence and use indentation.📝 Proposed fix
-``` +```text Model sees: "Discoverable APIs: external_fetchStocks" Model calls: discover_tools({ toolNames: ["fetchStocks"] }) Response: { tools: [{ name: "external_fetchStocks", description: "...", typeStub: "declare function external_fetchStocks(...)" }] } Model writes inside execute_typescript: const result = await external_fetchStocks({ ticker: "AAPL" })</details> <details> <summary>🤖 Prompt for AI Agents</summary>Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.In
@packages/ai-code-mode/skills/ai-code-mode/SKILL.mdaround lines 383 - 388,
The fenced code block containing the flow lines starting with "Model sees:
"Discoverable APIs: external_fetchStocks"" in SKILL.md is missing a language
identifier; update that triple-backtick fence to use a language tag (e.g.,linters accept it, ensuring the block that shows the discover_tools/execute_typescript sequence remains unchanged.Source: Linters/SAST tools
docs/code-mode/lazy-tools.md (2)
119-125: 💤 Low valueAdd language identifier to fenced code block.
The fenced code block showing example catalog output should specify a language identifier (e.g.,
text) to satisfy markdownlint.📝 Suggested fix
-``` +```text ### Discoverable APIs - external_fetchArchive - external_runReport - external_exportData</details> <details> <summary>🤖 Prompt for AI Agents</summary>Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.In
@docs/code-mode/lazy-tools.mdaround lines 119 - 125, Add a language
identifier to the fenced code block containing the "### Discoverable APIs"
sample output so markdownlint passes; update the opening backticks to include a
language liketext(e.g., change the block that lists "external_fetchArchive",
"external_runReport", and "external_exportData" to start with ```text).</details> <!-- cr-comment:v1:8c8fdffc4537652ad3f97ae4 --> _Source: Linters/SAST tools_ --- `141-147`: _💤 Low value_ **Add language identifier to fenced code block.** The fenced code block showing example catalog output with descriptions should specify a language identifier (e.g., `text`) to satisfy markdownlint. <details> <summary>📝 Suggested fix</summary> ```diff -``` +```text ### Discoverable APIs - external_fetchArchive — Retrieve historical weather archive data for a date range. - external_runReport — Generate a summary report for a given time period. - external_exportData — Export query results to CSV or JSON format. ``` ``` </details> <details> <summary>🤖 Prompt for AI Agents</summary>Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.In
@docs/code-mode/lazy-tools.mdaround lines 141 - 147, The fenced code block
containing the "Discoverable APIs" section is missing a language identifier. Add
the language identifiertextto the opening backticks of this code block
(changetotext) to satisfy markdownlint requirements. This applies to
the code block that lists the external functions like external_fetchArchive,
external_runReport, and external_exportData.</details> <!-- cr-comment:v1:661e2d957bbb1a6bad28ead9 --> _Source: Linters/SAST tools_ </blockquote></details> </blockquote></details> <details> <summary>🤖 Prompt for all review comments with AI agents</summary>Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.Inline comments:
In@docs/code-mode/lazy-tools.md:
- Line 71: Update the docs examples to use the newer OpenAI model name: in the
createCodeMode examples (identify the createCodeMode invocation and any
openaiText(...) calls), replace openaiText("gpt-5.2") with
openaiText("gpt-5.5"); ensure all occurrences in this file (including the
instances around the createCodeMode block and the examples at the other noted
locations) are changed so the documentation matches the NEWEST model names
defined in OPENAI_CHAT_MODELS.In
@packages/ai-code-mode/tests/create-discovery-tool.test.ts:
- Line 1: Reorder the named imports from vitest so they are alphabetized to
satisfy ESLint sort-imports: change the import list in the test file (the import
statement that currently lists describe, it, expect) to list expect first, then
describe, then it (i.e., import { expect, describe, it } from 'vitest'),
preserving the same module and spacing.In
@packages/ai/tests/lazy-tools.test.ts:
- Line 1: The named imports from 'vitest' are out of alphabetical order and
trigger sort-imports linting; update the import specifier in
packages/ai/tests/lazy-tools.test.ts to list the members alphabetically (use
describe, expect, it) so the import becomes alphabetically ordered and satisfies
the linter.In
@testing/e2e/tests/lazy-tools-wire.spec.ts:
- Around line 63-67: Remove the global journal reset in the test.beforeEach hook
to avoid mutating shared state; specifically delete the await
request.delete(http://127.0.0.1:${aimockPort}/v1/_requests) call inside
test.beforeEach and rely on the existing per-test isolation header (X-Test-Id)
instead, ensuring tests continue to set and use X-Test-Id when calling the
aimock server; no other changes to request setup are required.
Nitpick comments:
In@docs/code-mode/lazy-tools.md:
- Around line 119-125: Add a language identifier to the fenced code block
containing the "### Discoverable APIs" sample output so markdownlint passes;
update the opening backticks to include a language liketext(e.g., change the
block that lists "external_fetchArchive", "external_runReport", and
"external_exportData" to start with ```text).- Around line 141-147: The fenced code block containing the "Discoverable APIs"
section is missing a language identifier. Add the language identifiertextto
the opening backticks of this code block (changetotext) to satisfy
markdownlint requirements. This applies to the code block that lists the
external functions like external_fetchArchive, external_runReport, and
external_exportData.In
@packages/ai-code-mode/skills/ai-code-mode/SKILL.md:
- Around line 383-388: The fenced code block containing the flow lines starting
with "Model sees: "Discoverable APIs: external_fetchStocks"" in SKILL.md is
missing a language identifier; update that triple-backtick fence to use a
language tag (e.g., ```text) or replace the fenced block with an indented code
block so markdown linters accept it, ensuring the block that shows the
discover_tools/execute_typescript sequence remains unchanged.In
@packages/ai/tests/lazy-tools.test.ts:
- Around line 1-61: The tests for firstSentence and renderLazyCatalogEntry are
currently in the shared tests folder and must be co-located with the lazy-tools
source module; move the lazy-tools.test.ts file that contains tests referencing
firstSentence and renderLazyCatalogEntry into the same directory as the
lazy-tools implementation (next to the module that exports those functions),
keep the filename ending in .test.ts, and update the import paths in the test to
use the local module import so the tests run alongside the source per repo
guidelines.</details> <details> <summary>🪄 Autofix (Beta)</summary> Fix all unresolved CodeRabbit comments on this PR: - [ ] <!-- {"checkboxId": "4b0d0e0a-96d7-4f10-b296-3a18ea78f0b9"} --> Push a commit to this branch (recommended) - [ ] <!-- {"checkboxId": "ff5b1114-7d8c-49e6-8ac1-43f82af23a33"} --> Create a new PR with the fixes </details> --- <details> <summary>ℹ️ Review info</summary> <details> <summary>⚙️ Run configuration</summary> **Configuration used**: defaults **Review profile**: CHILL **Plan**: Pro **Run ID**: `f340d8aa-df1b-43c4-a30b-4d8305872f91` </details> <details> <summary>📥 Commits</summary> Reviewing files that changed from the base of the PR and between 22c9b42baec74914b720e440f29bd02be04eb164 and 20b8d2ab63e333a5cb7ac9f8ecfd65343be77630. </details> <details> <summary>📒 Files selected for processing (25)</summary> * `.changeset/code-mode-lazy-tools.md` * `docs/code-mode/lazy-tools.md` * `docs/config.json` * `packages/ai-code-mode/skills/ai-code-mode/SKILL.md` * `packages/ai-code-mode/src/create-code-mode-tool.ts` * `packages/ai-code-mode/src/create-code-mode.ts` * `packages/ai-code-mode/src/create-discovery-tool.ts` * `packages/ai-code-mode/src/create-system-prompt.ts` * `packages/ai-code-mode/src/index.ts` * `packages/ai-code-mode/src/types.ts` * `packages/ai-code-mode/tests/create-code-mode.test.ts` * `packages/ai-code-mode/tests/create-discovery-tool.test.ts` * `packages/ai-code-mode/tests/create-system-prompt.test.ts` * `packages/ai/skills/ai-core/tool-calling/SKILL.md` * `packages/ai/src/activities/chat/index.ts` * `packages/ai/src/activities/chat/tools/lazy-tool-manager.ts` * `packages/ai/src/activities/chat/tools/lazy-tools.ts` * `packages/ai/src/index.ts` * `packages/ai/src/types.ts` * `packages/ai/tests/lazy-tool-manager.test.ts` * `packages/ai/tests/lazy-tools.test.ts` * `testing/e2e/fixtures/lazy-tools-wire/basic.json` * `testing/e2e/src/routeTree.gen.ts` * `testing/e2e/src/routes/api.lazy-tools-wire.ts` * `testing/e2e/tests/lazy-tools-wire.spec.ts` </details> </details> <!-- This is an auto-generated comment by CodeRabbit for review status -->
- Use gpt-5.5 (newest OpenAI chat model in model-meta) in docs, skill, and the e2e route instead of gpt-5.2. - e2e: drop the beforeEach DELETE /v1/_requests journal reset; the spec already isolates per-test via the X-Test-Id header, so the global reset only risked racing adjacent parallel specs. - Alphabetize vitest named imports in the two new test files (sort-imports). - Add `text` language identifiers to the Discoverable APIs example fences in the lazy-tools doc and code-mode skill (markdownlint).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds lazy tool support (progressive disclosure) to Code Mode, plus a shared, optional
lazyToolsConfigthat also tuneschat()'s existing lazy-tool discovery. The goal is context savings: with a large tool catalog, you no longer pay for every tool's type stub + description in theexecute_typescriptsystem prompt on every turn.Lazy behavior is triggered purely by marking a tool
lazy: true—lazyToolsConfigis optional everywhere and only tunes how much of each lazy tool's description shows in the pre-discovery catalog.How it works
LazyToolsConfig { includeDescription: 'none' | 'first-sentence' | 'full' }(default'none'), accepted by bothchat()andcreateCodeMode().'none'is byte-identical to today's chat catalog.lazy: trueare kept out of the system prompt's full documentation (type stubs omitted) and listed in a lightweight "Discoverable APIs" catalog. A newdiscover_toolscompanion tool returns each lazy tool's TypeScript signature on demand; the model then callsexternal_<name>(...)insideexecute_typescript. Lookups tolerate the optionalexternal_prefix so the model can pass the catalog name verbatim.lazydefers only prompt documentation, not callability. This is stateless/serverless-safe.API
createCodeMode(config)now returns{ tool, discoveryTool, tools, systemPrompt }:tool(=execute_typescript) andsystemPromptare unchanged — fully backward compatible.toolsis the array to spread intochat({ tools });discoveryToolisnullwhen there are no lazy tools.Testing
LazyToolManagercatalog rendering, eager/lazy partition,discover_toolsincl. prefix tolerance +includeDescription, return shape).pnpm test:prgreen.includeDescription(none/first-sentence/full).docs/code-mode/lazy-tools.md), both agent skills, and a changeset updated in the same PR.Follow-up
There is currently no code-mode E2E harness in
testing/e2e/(no route/fixture exercisingcreateCodeMode). A code-mode-specific lazy E2E (sequencingdiscover_tools→execute_typescriptagainst a sandbox driver) would need net-new harness wiring disproportionate to this change; the partition/discovery logic is comprehensively unit-covered. Recommend tracking code-mode E2E harness wiring as a separate issue.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation
Tests