Repository navigation
feat: public streaming + should_finalize contracts for final_turn_only - #175
Merged
Merged
Conversation
An integrator adopting v1.2.0 final_turn_only in a streaming chat surface had to read core.py to learn two load-bearing behaviors; a refactor could have silently flipped them with zero signal. Both are now documented, stated in docstrings, and pinned by dedicated contract tests (#174): - Streaming guarantee: final_turn_only + astream NEVER emits the structured answer as content chunks, on any path. New: single_pass mode now SUPPRESSES content chunks (with the schema riding natively on every loop call, content is the JSON envelope by construction) - closing the leak that was previously a documented caveat. Tool-call chunks still stream. structured_synthesis_start fires iff a separate synthesis call is made (reuse and should_finalize-skip paths emit no event, pinned). - should_finalize messages contract: the predicate receives the run's conversation view at convergence - prior history (with memory), this turn's USER message(s), ASSISTANT tool-call messages, and one Message(role=TOOL) per executed tool with tool_name, tool_call_id, and tool_result ALWAYS populated. To back the guarantee, _append_tool_result now defaults tool_result to the message content, so error results and policy denials populate it too (previously success-path only). Closes #174 Claude-Session: https://claude.ai/code/session_01L1JWChKMs9yTyK92qquhJa
…rmalization Self-review of the #174 PR (high-effort multi-agent review) caught the contract overpromising; all findings addressed: - The "structured answer NEVER streams on any path" claim was FALSE on the reuse path: without single_pass the loop streams model output as it always has, so a converged answer that validates has already streamed before reuse. The contract now states this precisely, and a new StreamChunk(event="structured_reuse") fires at the reuse decision so chat surfaces can convert/retract the just-streamed bubble. Pinned honestly in the contract tests (the reuse-leak assertion the old test avoided is now present). - tool_call_id was documented as always populated - false for text-parsed tool calls (ToolCallParser creates id=None). Contract corrected: tool_name + tool_result are the guaranteed pair; tool_call_id only for native tool-calling providers. - "tool_result ALWAYS populated" was false for histories persisted before this change (error/denial rows stored None). Message.from_dict now normalizes TOOL messages on load, with a test. - Changelog restructured: single_pass chunk suppression and the tool_result unification moved under "### Changed" with migration notes (read AgentResult.parsed; match denial/error text instead of tool_result is None). - _append_tool_result's now-redundant tool_result parameter removed (every call site passed content); policy-denial invariant test added; single_pass mode named once as ctx.single_pass_active instead of three inline reconstructions; contract test file made self-contained (no cross-test-module imports); version pin dropped from the AGENT.md section header. Claude-Session: https://claude.ai/code/session_01L1JWChKMs9yTyK92qquhJa
Owner
Author
|
Self-review (high-effort multi-agent review, 10 verified findings) — addressed in d0a27a3:
Full suite: 7,607 passed, 205 skipped. CI green on the update pending. |
…sult semantics Max-effort self-review (39 agents, 15 verified findings incl. nits) showed the #174 contract was only implemented on the astream() chunk surface. All findings addressed: - stream_handler (run/arun): the synthesis turn and single_pass loop turns no longer feed the token callback - the second streaming surface now honors the same suppression contract (mutation-verified tests). - astream() non-streaming-provider fallback: tool-call chunks are now yielded there too, so single_pass suppression cannot produce a zero- chunk stream on providers without native async streaming (mutation-verified after fixing a fake whose class attribute was shadowed by an instance attribute). - serve SSE: content-free event chunks are forwarded as {"type": "event", "event": ...} records instead of being dropped - the documented convert/retract flow is reachable through selectools.serve (test added). - tool_result unification REVERTED in favor of the sharper v1.2-compatible contract: success -> tool_result carries the result text; error/policy/ coherence denial -> None (the structural failure discriminator, now documented as such). from_dict round-trip fidelity restored (@stable API no longer rewrites persisted data); caller-seeded TOOL messages scoped out of the guarantee; Message docstring updated. - Contract wording corrected everywhere: synthesis event is once-per-run (validation retries do not re-emit); structured_reuse qualified as non-single_pass in all three doc surfaces; SYSTEM context injections added to the messages enumeration; Since markers added (v1.3.0); changelog rewritten with an explicit Changed section scoped to the @beta single_pass surface; llms-full.txt regenerated. - Case C pins that structured_reuse is NOT emitted in single_pass mode. Pre-existing, unrelated: 5 starlette auth tests fail on clean main when starlette is installed (zero CI coverage for that surface) - filed #176. Claude-Session: https://claude.ai/code/session_01L1JWChKMs9yTyK92qquhJa
Owner
Author
|
Second self-review at max effort (39 agents, 15 verified findings including nits) — all addressed in 30edc6c. The big one: the contract was only implemented on the
Also filed #176: 5 pre-existing starlette auth test failures on clean main, invisible in CI (no Local suite: 7,787 passed; the only failures are the pre-existing #176 set (verified failing on clean HEAD, never run in CI). |
johnnichev
added a commit
that referenced
this pull request
Jul 14, 2026
Bumps version 1.2.0 -> 1.3.0 and rolls the [Unreleased] changelog into a dated [1.3.0] section (both changelog files). Contents already merged via #175 (closes #174): the final_turn_only streaming behavior and should_finalize message view are now documented, tested public contracts enforced on astream() chunks, the stream_handler callback, and the serve SSE stream; single_pass no longer streams the structured answer (beta surface, Changed entry with migration note). Claude-Session: https://claude.ai/code/session_01L1JWChKMs9yTyK92qquhJa
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #174.
The problem
Two behaviors that streaming integrators MUST rely on were source-only knowledge — a future refactor could flip them with no type/signature change and no test signal.
1. Streaming contract (documented + tested + strengthened)
Guarantee:
final_turn_only=True+astream()never emits the structured answer as content chunks — on any path. Pinned clause-by-clause intests/agent/test_structured_streaming_contract.py:single_pass: loop prose streams; the synthesis call is non-streaming;StreamChunk(event="structured_synthesis_start")fires iff a synthesis call is made (reuse andshould_finalize-skip paths emit no event).single_pass: content chunks are now suppressed — with the schema riding natively on every loop call, any content the model emits is the JSON envelope by construction. Tool-call chunks still stream for activity display. This closes the leak that was previously a documented caveat, which is exactly what makes the issue's "brackets on BOTH paths" ask unnecessary: there is no synthesis content to bracket, on either path.The event/no-leak semantics are documented in AGENT.md as an explicit public contract and in the
single_passdocstring.2.
should_finalizemessages contract (documented + tested + backed)Documented in the docstring and AGENT.md:
messagesis the run's conversation view at convergence — prior history (with memory), this turn's USER message(s), the ASSISTANT tool-call messages, and oneMessage(role=TOOL)per executed tool withtool_name,tool_call_id, andtool_resultalways populated.To make that last guarantee true rather than aspirational:
_append_tool_resultnow defaultstool_resultto the message content, so error results and policy denials populate it too (previously only the success path did — an integrator matching ontool_resultwould have silently missed error turns).Tests
13 contract tests: single_pass no-leak + tool-call chunks still visible, prose-streams/JSON-doesn't on the synthesis path, event-iff-synthesis across all three convergence paths, event chunks are content-free, predicate message-view contents (run + astream), and the tool_result invariant on success/parallel/error paths (error path was RED before the fix).
Full suite: 7,605 passed, 205 skipped (py3.12). ruff clean.
Behavior change note:
single_pass+astream()stops streaming JSON as content — v1.2.0's documented caveat becomes a guarantee.single_passis 0 days old and opt-in; this is the fix its one consumer asked for.https://claude.ai/code/session_01L1JWChKMs9yTyK92qquhJa