Skip to content

Implement native tool-calling protocol (#15) - #77

Merged
portdeveloper merged 4 commits into
portdeveloper:mainfrom
MayurK-cmd:feat/native-tool-calling
Sep 22, 2026
Merged

portdeveloper merged 4 commits into
portdeveloper:mainfrom
MayurK-cmd:feat/native-tool-calling

Conversation

@MayurK-cmd

Copy link
Copy Markdown

Replace v0 hand-rolled JSON protocol with QVAC's native tool-calling via completion({ tools }) on capable models. Structured tool calls are more robust than regex-parsed JSON and unambiguous about the model's intent.

Closes #15

Changes:

  • src/tools.mjs: Add getToolDefinitions() for OpenAI-compatible Tool schemas and dispatchToolCall() to route tool calls by name to handlers. All v0 actions (get_address, get_balance, get_token_balance, send_mon, send_token) now have native tool equivalents with proper parameter schemas.

  • src/agent.mjs: Add completeWithTools(history, tools, onToken) that calls completion({ tools, ... }) and iterates over run.events to collect contentDelta (text) and toolCall (structured calls) events.

  • src/cli.mjs: Refactor processLine() to dispatch based on config.useNativeTools. Native path calls completeWithTools(), dispatches each tool call through existing handleAction() validation/confirmation flow. Fallback v0 JSON path uses complete() + parseAction(). Both paths share the same confirmation and policy enforcement logic. Status block displays active protocol mode.

  • src/config.mjs: Add useNativeTools config option read from USE_NATIVE_TOOLS env var (default 'true'). Users can set USE_NATIVE_TOOLS=false to fallback to v0 JSON protocol for small/dev models that don't support native tool-calling.

  • test/tools.test.mjs: Add 5 new tests for native tool-calling:

    • getToolDefinitions() returns valid OpenAI-compatible schema
    • Tools include all v0 actions with correct parameter schemas
    • dispatchToolCall() routes calls correctly and throws on unknown tools
  • test/native-tools-integration.mjs: Integration test verifying the dual-protocol dispatch path, config toggle, and tool schema validation.

  • test/native-tools-no-model.test.mjs: Comprehensive 28-test suite covering schema validation, dispatch routing, mock QVAC flow, config toggle, backward compatibility, end-to-end flow simulation, and protocol comparison. Tests work without requiring a local model by mocking QVAC completion events.

Verification:

  • All 66 tests pass (38 existing + 28 new), 0 failures
  • No regressions: v0 JSON protocol remains intact via USE_NATIVE_TOOLS toggle
  • Backward compatible: slash-commands (/address, /send, etc.) work in both modes
  • Build succeeds: dist/cli.mjs and dist/e2e.mjs both up to date
  • Ready for interactive testing once model is available

This is the natural upgrade path before integrating @tetherto/wdk-mcp-toolkit MCP server (once it ships), which will be a small connector change at this point.

What this changes

How I tested it

  • npm run build succeeds
  • npm run smoke prints SMOKE_OK
  • Ran the REPL and exercised the change
  • Dry-run send
  • Real gasless send (paste the tx hash if so):

Model / platform tested on:

Scope check

  • This stays within v0 scope (native MON, single account, testnet), OR
  • This grows the scope, and I've described the new surface above.

Conventions

  • I used npm (not pnpm/yarn) and did not add a global sodium-native override.
  • I did not commit .env, seeds, keys, or model weights.

@portdeveloper portdeveloper left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on. I ran the full suite at 10a9c7f and checked the flow against the native-tools example shipped in @qvac/sdk@0.14.1. The tests pass, but the live protocol still has blockers.

  • src/cli.mjs:371-384 drops every tool result. It stores a placeholder assistant message, calls handleAction, then returns to the user. QVAC's own example appends each result as a role: "tool" message and runs a follow-up completion so the model can use the result. The current code cannot turn a balance, refusal, or transaction receipt into the assistant's answer, and the next user turn has no tool result in history. Please carry the executed result into history and continue the completion loop until the model stops calling tools, with a finite call limit.

  • src/agent.mjs:86-96 silently discards toolCallError events, while the catch path converts model errors into an empty successful result. Native mode is the default here, so a malformed call can print a blank line and scripted mode can still exit zero. Please surface these errors to the caller and preserve the existing scripted failure behavior.

  • src/tools.mjs:364-441 exposes only the five actions that existed on this branch's old base. Current main also ships account, get_nfts, transfer_nft, and swap. This branch already conflicts with main; after the rebase, default native mode needs definitions and dispatch for the current action set so those features do not disappear.

  • The new tests mock the object they expect and never drive completeWithTools or processLine. For example, the end-to-end test stops after asserting that dispatchToolCall is a function. Please add a harness that feeds real completion events through the CLI boundary, observes the handler result entering a tool message, and verifies the follow-up turn. A toolCallError regression belongs in that path too.

Please rebase before the next revision so the action coverage is checked against current main.

@portdeveloper portdeveloper left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the revision. One blocker remains in src/cli.mjs:533-540: native write calls go straight through dispatchToolCall, and the comment explicitly says handleAction is skipped. That bypasses recipient resolution, spend-policy checks, the transaction preview, mainnet acknowledgement, and the y/N confirmation for send_mon and send_token.

Please route native writes through the existing handleAction safety boundary and keep direct dispatch only for read-only tools. Add a CLI-boundary regression that feeds a native send tool call through processLine, proves no wallet method runs before confirmation, and covers a policy refusal. The current integration tests reproduce the loop in test code instead of driving the production boundary, so they would stay green if this bypass regressed.

MayurK-cmd pushed a commit to MayurK-cmd/nad-agent that referenced this pull request Sep 6, 2026
PR portdeveloper#77 review blocker: native tool calls for writes (send_mon, send_token) were dispatched directly via dispatchToolCall, bypassing the resolveSend -> policy -> preview -> mainnet ack -> y/N confirmation flow that the v0 path and slash commands share. A compromised or malformed model could trigger a write without ever asking the operator.

- Refactor handleAction in cli.mjs to return a printable result string (or null for "none") so the native loop can capture the tool result.
- Extract the native tool-calling loop into src/nativeToolLoop.mjs with handleAction and dispatchToolCall as injected dependencies, so the routing is reachable from tests.
- In the loop, route writes (isWrite) through handleAction and reads through dispatchToolCall - the read fast path stays snappy, writes share the same boundary as the rest of the agent.
- Add test/native-tools-cli-boundary.test.mjs: drives the production loop with stubbed boundary seams, asserts writes never reach dispatchToolCall, and proves a policy/resolveSend refusal never bypasses the boundary. The previous integration tests re-implemented the loop in test code and would have stayed green if the bypass regressed; this one drives the real boundary.

Co-Authored-By: Claude Code <noreply@anthropic.com>
MayurK-cmd pushed a commit to MayurK-cmd/nad-agent that referenced this pull request Sep 6, 2026
PR portdeveloper#77 review blocker: native tool calls for writes (send_mon, send_token) were dispatched directly via dispatchToolCall, bypassing the resolveSend -> policy -> preview -> mainnet ack -> y/N confirmation flow that the v0 path and slash commands share. A compromised or malformed model could trigger a write without ever asking the operator.

- Refactor handleAction in cli.mjs to return a printable result string (or null for "none") so the native loop can capture the tool result.
- Extract the native tool-calling loop into src/nativeToolLoop.mjs with handleAction and dispatchToolCall as injected dependencies, so the routing is reachable from tests.
- In the loop, route writes (isWrite) through handleAction and reads through dispatchToolCall - the read fast path stays snappy, writes share the same boundary as the rest of the agent.
- Add test/native-tools-cli-boundary.test.mjs: drives the production loop with stubbed boundary seams, asserts writes never reach dispatchToolCall, and proves a policy/resolveSend refusal never bypasses the boundary. The previous integration tests re-implemented the loop in test code and would have stayed green if the bypass regressed; this one drives the real boundary.
@MayurK-cmd
MayurK-cmd force-pushed the feat/native-tool-calling branch from 7498e31 to ddbef4a Compare September 6, 2026 17:35
portdeveloper pushed a commit that referenced this pull request Sep 7, 2026
Adds src/mcp.mjs: optional MCP server wiring for QVAC's completion({ mcp })
path, alongside (not replacing) the existing v0 JSON-action protocol.

- src/mcp.mjs: loadMcpConfig() reads an optional mcp.json (same
  optional-file/NAD_*-override pattern as policy.json/address-book.json) —
  a list of MCP servers to spawn over stdio. connectMcpServers() connects
  each with the official @modelcontextprotocol/sdk, best-effort (a server
  that fails to start is skipped and reported, not fatal — same rule as a
  failed model load). summarizeMcpToolResult() turns an arbitrary MCP tool
  result into a bounded string for history/terminal.

- src/agent.mjs: completeWithMcp() runs the full agentic loop QVAC's own
  mcp-websearch example requires by hand — stream text, collect tool calls,
  invoke them, push { role: "assistant" } then { role: "tool" } turns back
  into history, and re-complete — until the model stops calling tools or a
  finite maxToolRounds is hit. Takes an injectable runCompletion for tests.

- src/cli.mjs: when MCP servers are configured and connected, natural-language
  turns route through completeWithMcp() instead of the v0 path. Every tool
  call is gated behind the same confirm + mainnet-ack prompt every wallet
  write already goes through — the tool catalog is discovered at runtime
  from an arbitrary server, so there's no way to tell a read from a write
  the way isWrite() does for the built-in actions, and asking every time is
  the safe default. Tool errors and a hit round limit are surfaced, not
  swallowed.

@tetherto/wdk-mcp-toolkit (the 35-tool wallet server named in #16 and the
README's "Upgrade path") is still a reserved placeholder on npm (0.0.0, no
code) as of this PR, so this wires any MCP server generically over stdio
instead — pointing mcp.json at the real package once it ships needs no code
changes here. #15 (native tool-calling via completion({ tools })) is being
worked on separately in #77; this does not touch that path or its files.

312 tests pass (17 new: mcp.test.mjs covers config validation, result
summarization, and connectMcpServers()'s best-effort failure handling
against the real MCP SDK; agent-mcp.test.mjs drives completeWithMcp()'s
loop end-to-end — follow-up turns, multi-call rounds, toolError surfacing,
round-limit enforcement, and stream-error tolerance — via an injected fake
completion() rather than asserting dispatch is a function). npm run build
succeeds; npm run doctor passes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@portdeveloper portdeveloper left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The write path now reaches handleAction, which addresses the confirmation bypass. One failure-propagation bug remains in src/cli.mjs:515-532: hadFailureRef is a copy of the CLI flag, and its value is never copied back after runNativeToolLoop returns.

I drove the production loop with a toolCallError in scripted mode: it set hadFailureRef.value to true while the CLI flag stayed false, so the exit expression still yields 0. Dispatch exceptions and the tool-call cap take the same path. Please propagate that state back without clearing failures set inside handleAction, and add a regression through the actual processLine/CLI boundary that checks the nonzero exit. The current boundary tests stub handleAction and stop at the extracted loop, so they miss this wiring bug.

@MayurK-cmd
MayurK-cmd force-pushed the feat/native-tool-calling branch from 6dc6605 to ddbef4a Compare September 8, 2026 03:21
MayurK-cmd pushed a commit to MayurK-cmd/nad-agent that referenced this pull request Sep 8, 2026
PR portdeveloper#77 review blocker: native tool calls for writes (send_mon, send_token) were dispatched directly via dispatchToolCall, bypassing the resolveSend -> policy -> preview -> mainnet ack -> y/N confirmation flow that the v0 path and slash commands share. A compromised or malformed model could trigger a write without ever asking the operator.

- Refactor handleAction in cli.mjs to return a printable result string (or null for "none") so the native loop can capture the tool result.
- Extract the native tool-calling loop into src/nativeToolLoop.mjs with handleAction and dispatchToolCall as injected dependencies, so the routing is reachable from tests.
- In the loop, route writes (isWrite) through handleAction and reads through dispatchToolCall - the read fast path stays snappy, writes share the same boundary as the rest of the agent.
- Add test/native-tools-cli-boundary.test.mjs: drives the production loop with stubbed boundary seams, asserts writes never reach dispatchToolCall, and proves a policy/resolveSend refusal never bypasses the boundary. The previous integration tests re-implemented the loop in test code and would have stayed green if the bypass regressed; this one drives the real boundary.
@MayurK-cmd
MayurK-cmd force-pushed the feat/native-tool-calling branch from 72c2b56 to f066002 Compare September 8, 2026 04:39
@MayurK-cmd
MayurK-cmd force-pushed the feat/native-tool-calling branch from f066002 to 60d8a08 Compare September 11, 2026 03:47
@MayurK-cmd

Copy link
Copy Markdown
Author

Squashed the previous review iterations into a single commit while preserving the final implementation and fixes. Please review the current 60d8a08 head.

@portdeveloper portdeveloper left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The failure flag is now copied back into the CLI without clearing failures from handleAction, which fixes the last wiring issue. Two blockers remain on 60d8a08.

  • src/tools.mjs:1127-1224 still advertises and dispatches only five native tools. Current code also supports account, get_nfts, transfer_nft, and swap, but the default native mode cannot call them. Please cover the current action set, keeping account switches behind the confirmation boundary. The test asserting exactly five definitions currently locks in this regression.
  • src/nativeToolLoop.mjs:63-144 silently succeeds when the outer turn limit is exhausted. I ran the production loop with one read tool call per completion: it executed all ten calls, returned with hadFailure.value === false, and printed no limit error. Since the inner guard checks greater than ten and the outer loop stops at ten, the usual one-call-per-turn loop never reaches the error path. Please report turn exhaustion and propagate a scripted failure. Cover that case through the actual CLI exit boundary; the existing tests stop at the extracted loop.

The current GitHub Actions run also needs maintainer authorization before its test check can run.

@portdeveloper portdeveloper left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The additional tools and explicit turn-limit error address the production changes from my last review. The revision still has blockers.

src/agent.mjs:165 listens for toolCallError while iterating run.events. The locked @qvac/sdk 0.14.1 emits toolError there, with the error in event.error; toolCallError belongs to its separate tool-call stream. I passed an SDK-schema-valid toolError through the exact completion function and production loop: it returned toolErrors: [] and left hadFailure false. Please handle the actual event shape and preserve its message through the scripted failure path.

Native mode also still initializes history with systemPrompt() from src/tools.mjs, which explicitly instructs the model to output one JSON action line. The native loop treats that text as a chat response and does not execute it. Please select a prompt appropriate to the enabled protocol and cover a native completion through the real CLI boundary.

test/native-tools-no-model.test.mjs:46 still asserts five tools, so the current suite fails with 9 !== 5. The tests labelled processLine/CLI boundary still invoke only the extracted loop with a stubbed handleAction. Please update the coverage for the current action set and add the previously requested regression through the actual CLI exit path for turn exhaustion and SDK tool errors. Rebase to include the Windows matrix and verify both OS jobs on the final revision.

@portdeveloper portdeveloper left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SDK toolError handling and protocol-specific prompt are fixed, and all 428 tests pass locally. There is still a scripted failure regression in src/nativeToolLoop.mjs:136-149: the read path prints a returned Refused: string without setting hadFailure.value.

I ran the production loop with the real dispatchToolCall and a tool call {name: "get_nfts", arguments: {address: "not-an-address"}}, followed by a text-only completion. It printed Refused: "not-an-address" is not a valid address. and finished with hadFailure.value === false. The CLI consequently reports exit 0. The existing v0 path uses isRefusal(out) and marks this as a failure.

Please apply the same refusal handling to native read results and keep the failure set even if a later completion succeeds. Add a regression using the real dispatch path. Also exercise the actual scripted CLI exit: test/native-tools-cli-exit.test.mjs currently reconstructs the stack and uses its own exitCode() function, so it does not verify processLine wiring or the process exit. A subprocess with an injected model can cover the refusal and the SDK-error/turn-limit cases without a wallet or live model.

@MayurK-cmd

Copy link
Copy Markdown
Author

Implemented in this commit:

  • src/nativeToolLoop.mjs — tool results that are Refused: strings now set hadFailure.value in scripted mode, same isRefusal(out) rule as the v0 path. Applies to both reads (via dispatchToolCall) and writes (via handleAction, whose own flag lives in cli.mjs module state rather than the loop's box — same hole). The flag is sticky: a refusal followed by a clean chat turn still exits non-zero.
  • Regression test with the real dispatch path: get_nfts + not-an-address prints Refused: "not-an-address" is not a valid address. and fails scripted even after a follow-up text turn. Confirmed it fails with the fix reverted.
  • Subprocess harness (test/helpers/shim-*.mjs): runs the real dist/cli.mjs in scripted mode with only the model faked (loader-hook @qvac/sdk stub — no GPU, no funded wallet; wallet init, loop, boundary, and process.exit are production code). Verified through the actual binary: refusal → exit 1, SDK toolError → exit 1 with code + message intact, turn exhaustion → exit 1, clean turn → exit 0.
    Results here: npm test 432 pass / 4 fail (the 4 are pre-existing mcp.test.mjs failures — @modelcontextprotocol/sdk isn't installed in my local env; unrelated to this branch), npm run build ✅, npm run doctor ✅, smoke's non-model steps ✅ against live testnet.

Note :

Not verifiable on my hardware: live-model behavior (tool-call emission quality against the 9 schemas, prompt efficacy, multi-turn convergence). If you'd like, I can do a rented-GPU run (Qwen3-8B, dry-run, testnet) and report the transcript — or happy for you to cover that side in review.

@portdeveloper portdeveloper left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks

@portdeveloper
portdeveloper merged commit 7e0ad55 into portdeveloper:main Sep 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

QVAC native tool-calling (completion({ tools }))

2 participants