Skip to content

Fence lossy serial-tool history certificates by producer policy - #989

Draft
bradhilton wants to merge 2 commits into
mainfrom
fix/serial-tool-prefix-eligibility
Draft

bradhilton wants to merge 2 commits into
mainfrom
fix/serial-tool-prefix-eligibility

Conversation

@bradhilton

@bradhilton bradhilton commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Serial tool projection can return one parsed call while retaining sampled tokens for multiple calls. The previous complete-turn check could therefore certify a canonical one-call history as equivalent to that longer sampled history.

This change denies complete-turn certification for explicit parallel_tool_calls=False when tools are configured or returned. Independently aligned reasoning prefixes, exact raw observations, non-tool turns, and true/omitted parallel settings retain their existing eligibility. Sampled token IDs, logprobs, usage, and served responses are unchanged. Both Responses adapters carry this setting into their chat observation; the Tinker direct observation path uses the same eligibility rule.

The vLLM Responses observer also preserves absent tools as None in its internal chat request. Passing an empty list caused the native chat validator to reject a valid Responses request after generation. Nonempty tools and the serial-policy setting remain intact.

ART now owns a certificate-policy scope helper. Native vLLM and Tinker derive their cache namespace internally, excluding old certificates without deleting them. Distributed consumers need the companion integration that derives this scope inside corrected workers and checks attached-candidate policy; merely forwarding a new scope label to an old producer is insufficient. SGLang uses the external observer/store rather than an ART-owned local cache.

Validation:

  • 150 focused tests pass across append-only observations, vLLM/SGLang history, token-prefix storage, and Tinker renderers.
  • Synthetic two-call/one-visible-call regression fails on the original parent and passes here; tests cover full/SSE observations, unchanged samples/logprobs/usage, reasoning preservation, choice isolation, and old local certificates.
  • Six Responses observer controls cover streamed/nonstreamed requests with omitted, empty and nonempty tools. Four absent/empty cases fail before the correction; all six pass afterward. An additional local check executes the retained vLLM 0.29 native tool validator with real Pydantic.
  • Required uv run prek run --all-files: lint, formatting, and lock checks pass. Full type checking reports 14 diagnostics in unchanged trainer/megatron code. The final candidate and untouched policy parent 67567571cffc4eb37a6cf9f2d01ac474c11aa88a produce byte-identical output for the same uv run ty check src tests command/environment; the original main parent also reproduced these diagnostics.

Draft pending the distributed consumer companion and review. The final absent-tools correction still needs runtime qualification. No deployment or cache migration is included.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant