Skip to content

fix(anthropic): route input_json deltas by content-block index#3009

Open
mcavage-docker wants to merge 1 commit into
docker:mainfrom
mcavage-docker:fix/anthropic-parallel-toolcall
Open

fix(anthropic): route input_json deltas by content-block index#3009
mcavage-docker wants to merge 1 commit into
docker:mainfrom
mcavage-docker:fix/anthropic-parallel-toolcall

Conversation

@mcavage-docker
Copy link
Copy Markdown

Summary

Parallel tool_use blocks streamed from Anthropic share a single streamAdapter.toolID field that gets overwritten on every ContentBlockStartEvent. Subsequent InputJSONDelta events for the first block carry the second block's tool ID, and the runtime accumulator (keyed by ToolCall.ID in pkg/runtime/streaming.go) concatenates both calls' argument fragments into one buffer. The first tool's args end up empty; the second tool's args contain interleaved fragments that fail to parse, surfacing as:

invalid character 's' looking for beginning of value
invalid character '-' after object key:value pair

(the exact character depends on where the chunk boundaries fall.)

This fix tracks tool IDs per content-block index via toolIDByBlock map[int64]string and routes InputJSONDelta events using eventVariant.Index. The same fix is applied to betaStreamAdapter, which shared the bug.

How it manifests in production

Observed against a local GM agent team (cagent v1.57.0) running Claude Opus with ~18 parallel MCP tool calls in a single turn. Five calls failed with the JSON errors above before reaching the MCP gateway; the other thirteen succeeded. Retrying the same calls in isolation always succeeded — which made it look like a transient parser corruption issue, but the actual condition is deterministic: it only fires when the provider interleaves input_json_delta events across blocks, which is more likely under heavier fan-out.

The bug is present on current main (verified at HEAD 03386b31). Bug was never reported via gh search for "parallel tool", "invalid character", or "tool call arguments" before this PR.

Test plan

pkg/model/provider/anthropic/parallel_toolcall_test.go (added in this PR) feeds a fake SSE decoder a sequence of two parallel tool_use blocks with interleaved input_json_delta events, mirroring what Anthropic emits for parallel tool calls. Covers both streamAdapter and betaStreamAdapter.

Without the fix:

toolu_AAA args=\"\"
toolu_BBB args=\`{\"refresh_id\":\"abc-{\"category\":\"tool_failure\",def\"}\"summary\":\"x\"}\`
JSON parse error: invalid character 'c' after object key:value pair

With the fix:

toolu_AAA args=\`{\"refresh_id\":\"abc-def\"}\`
toolu_BBB args=\`{\"category\":\"tool_failure\",\"summary\":\"x\"}\`
  • New regression tests added (`TestParallelToolCallIDsAreNotCrossWired`, `TestBetaParallelToolCallIDsAreNotCrossWired`)
  • Both new tests fail on `main` without the patch
  • Both new tests pass with the patch
  • Full `./pkg/model/provider/anthropic/...` suite passes
  • Full `./pkg/runtime/...` suite passes
  • Full `./pkg/model/...` suite passes
  • `gofmt` clean, `go vet` clean

Follow-up

`pkg/model/provider/bedrock/adapter.go` carries the same single-`currentToolID` pattern and is likely vulnerable to the same bug on parallel Bedrock Converse tool calls. Not patched here to keep the diff focused — happy to file a follow-up PR if useful.

🤖 Generated with Claude Code

Parallel tool_use blocks streamed from Anthropic share a single
streamAdapter.toolID field that is overwritten on every
ContentBlockStartEvent. Subsequent InputJSONDelta events for the first
block then carry the second block's tool ID, and the runtime
accumulator (keyed by ToolCall.ID in pkg/runtime/streaming.go)
concatenates both calls' argument fragments into one buffer. The first
tool's args end up empty; the second tool's args contain interleaved
fragments that fail to parse, surfacing as

  invalid character 's' looking for beginning of value
  invalid character '-' after object key:value pair

(the exact character depends on where the chunk boundaries fall).

Track tool IDs per content-block index via toolIDByBlock map[int64]string
and route InputJSONDelta events using eventVariant.Index. Apply the same
fix to betaStreamAdapter, which shared the bug.

Adds pkg/model/provider/anthropic/parallel_toolcall_test.go covering
both the standard and Beta adapters with interleaved input_json_delta
events; both tests fail without this fix and pass with it.

Note: pkg/model/provider/bedrock/adapter.go carries the same
single-currentToolID pattern and is likely vulnerable to the same bug
on parallel Converse tool calls; left for a follow-up PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mcavage-docker mcavage-docker requested a review from a team as a code owner June 5, 2026 13:35
Copy link
Copy Markdown

@docker-agent docker-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

The fix is correct and well-targeted. Replacing the single toolID string field with a toolIDByBlock map[int64]string keyed by content-block index is the right solution for routing InputJSONDelta events across parallel tool_use blocks. Both streamAdapter and betaStreamAdapter are patched consistently, and the new regression tests faithfully reproduce the interleaved-delta scenario described in the PR.

Verification summary

Finding Severity Verdict
Silent empty-string ToolCall ID on missing block index Medium ✅ DISMISSED — Anthropic API guarantees content_block_start before any delta; retry path restarts from the beginning so indices are re-populated before deltas arrive
toolIDByBlock not cleared on stream retry Medium ✅ DISMISSED — retry replaces the stream from the beginning; all content_block_start events are re-delivered before any deltas, overwriting any stale entries
Redundant nil-guard inside Recv() Low Informational note (see inline comment)

One low-severity observation is left as an inline comment for awareness. No blockers found.

switch block := eventVariant.ContentBlock.AsAny().(type) {
case anthropic.ToolUseBlock:
a.toolID = block.ID
if a.toolIDByBlock == nil {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW] Redundant nil guard on toolIDByBlock inside Recv() — slightly misleading

newStreamAdapter always initialises toolIDByBlock to a non-nil map, so the defensive if a.toolIDByBlock == nil check inside the ContentBlockStartEvent branch (line 101) will never fire for adapters created via the constructor.

The check exists because the test file constructs the adapter as a struct literal without calling the constructor (see parallel_toolcall_test.go line 129: adapter := &streamAdapter{retryableStream: ...}), leaving the map nil. The nil-guard silently rescues that path.

Two minor concerns:

  1. The guard implies the constructor-initialised map can somehow become nil, which is confusing for future readers.
  2. The test is actually exercising the nil-initialised (non-constructor) path, which diverges slightly from production usage.

Neither is a functional bug — Go map reads on a nil map safely return the zero value, and the lazy initialisation via the guard works correctly. Just worth being aware of if the guard is ever removed or the constructor changes.

@aheritier aheritier added area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) area/providers/anthropic For features/issues/fixes related to the usage of Anthropic models kind/fix PR fixes a bug (maps to fix: commit prefix) labels Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/providers/anthropic For features/issues/fixes related to the usage of Anthropic models area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) kind/fix PR fixes a bug (maps to fix: commit prefix)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants