Skip to content

fix(tangle): bound tool output as content, not as metadata - #311

Merged
drewstone merged 1 commit into
mainfrom
fix/tool-result-content-bound-20260911
Sep 11, 2026
Merged

fix(tangle): bound tool output as content, not as metadata#311
drewstone merged 1 commit into
mainfrom
fix/tool-result-content-bound-20260911

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Problem

validatedSandboxPromptResult holds every field except response/text/finalText to the metadata bound, CONTRACT_MAX_STRING_LENGTH = 16,384 characters per string.

toolInvocations carries whatever a tool returned. One webfetch of a paper or an API page is routinely tens or hundreds of kilobytes. It is content, not a field that describes the turn.

Meanwhile the Sandbox SDK serializes each tool value up to MAX_SERIALIZED_TOOL_VALUE_BYTES = 4 MiB (runtime-api-BcLo4lBU.js:63). Producer and consumer disagree by 256×: every tool result between 16 KiB and 4 MiB is written by one side and refused by the other.

Why it was expensive

The refusal does not fail the tool call. This validator runs inside the terminal result read — after the live stream drained and after the usage receipt was credited — so it converted a finished, fully paid turn into an unreconcilable retained execution, which a supervisor then reports as a child that did no work at all.

Measured 2026-09-11 in one Discovery Lab worktree:

  • 143 of 199 children across 16 pursuits, every one at iterations: 0, ms: 0, under the single reason retained provider execution requires reconciliation before replacement.
  • Per run: 9 of 11, 16 of 23, 17 of 21, 14 of 19. Zero pursuits with an observer journal were clean.
  • The lanes that died were the literature enumerate and extract stages — the ones that fetch papers. Directors doing local work survived.
  • Their six-stage sourcing graph blocked at whatever stage it reached in 24 of 24 invocations, producing zero charters.

Selection evidence: across 45 surviving children and 845 tool results, the result-length distribution decays smoothly and stops dead at exactly 16,384, with none at or above it and no pile-up at the bound (which truncation would produce).

Direct falsification against the shipped validator — the wall is exactly where the constant is:

  16384  (passes the bound)
  16385  REJECTED — value exceeds its JSON bound
  20000  REJECTED — value exceeds its JSON bound
 250000  REJECTED — value exceeds its JSON bound

Change

Add toolInvocations to the fields excluded from the metadata check. isBoundedEventContentJson was written for exactly this material — its comment says content "may contain a single large transcript or tool result" — and the whole record already passes it at CONTRACT_MAX_JSON_BYTES immediately above.

Tool output stays bounded: the content check still refuses an unbounded result. Every other field keeps its metadata limit.

Tests

One added to tangle-events.test.ts. It fails on the previous code with the exact production error, value exceeds its JSON bound. It pins the wall (MAX_STRING_LENGTH + 1 now passes), a realistic 200 KB fetch, the retained upper bound (2 MiB still refused), and that traceId keeps the metadata limit.

Provider suite 263 pass, typecheck clean. Patch version 1.1.8 → 1.1.9.

🤖 Generated with Claude Code

`validatedSandboxPromptResult` held every field but `response`/`text`/
`finalText` to the metadata bound, CONTRACT_MAX_STRING_LENGTH (16,384
characters per string). `toolInvocations` carries whatever a tool returned, and
one webfetch of a paper or an API page is routinely tens or hundreds of
kilobytes, so it belongs with the response text rather than with the fields that
describe the turn. `isBoundedEventContentJson` was written for exactly this
material — its own comment says content "may contain a single large transcript
or tool result" — and the whole record already passes it at
CONTRACT_MAX_JSON_BYTES immediately above.

The Sandbox SDK serializes each tool value up to MAX_SERIALIZED_TOOL_VALUE_BYTES
(4 MiB), so producer and consumer disagreed by 256x: every tool result between
16 KiB and 4 MiB was written by one side and refused by the other.

The cost of that disagreement was not a failed tool call. This validator runs
inside the terminal result read, AFTER the live stream has drained and the usage
receipt has been credited, so the refusal converted a finished, fully paid turn
into an unreconcilable retained execution that a supervisor reports as a child
that did no work at all. Measured 2026-09-11 in one Discovery Lab worktree: 143
of 199 children across 16 pursuits, every one at `iterations: 0` and `ms: 0`,
and the literature enumerate and extract stages that fetch papers were the ones
that died. Their six-stage sourcing graph blocked at whatever stage it reached
in 24 of 24 invocations and produced zero charters.

Selection evidence: across 45 surviving children and 845 tool results the length
distribution decays smoothly and stops dead at exactly 16,384, with none at or
above it and no pile-up at the bound.

Tool output stays bounded — the content check above still refuses an unbounded
result — and every other field keeps its metadata limit.

Tests: one added; it fails on the previous code with the exact production error,
`value exceeds its JSON bound`. Provider suite 263 pass, typecheck clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@drewstone
drewstone merged commit 01f1864 into main Sep 11, 2026
2 checks passed
@drewstone
drewstone deleted the fix/tool-result-content-bound-20260911 branch September 11, 2026 07:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant