Preserve complete sampled conditioning in exact history recovery

[bradhilton]

Captured length-stopped histories can fail rendered mask or message-boundary checks even when their complete sampled request/response chain is retained. Ordinary rendered assembly can also preserve sampled output IDs while changing their original prompt. This adds a bounded exact-source retry that requires the complete original conditioning, output IDs, logprobs and sampled STOP masks before returning.

The retry keeps the entire independently proved closing tail at the sampled-output endpoint. Additional native inter-request context is accepted only through complete nested captured prompts and stays nonsampled with NaN logprobs. Unsupported projections, overrides, incomplete sources, changed tails and non-nested prompts still refuse. Explicit thinking settings remain authoritative; only an absent automatically injected default can be omitted during the private retry. The stopped-message fallback also retains its full-context marker and completed-tail proofs.

This intentionally changes the public sampled-history contract: a recorded logprob cannot be carried across a known different complete token prefix. Explicit template/tokenizer overrides and text-equivalent reconciliation no longer allow that behavior. Three existing tests previously asserted such output; their expectations now assert refusal and retain supported original-conditioning alternatives. For divergent captured prompts, multi_history=True with the default reconcile_text_equivalent_tokenizations=False preserves separate histories already produced by the protocol; it does not force every incompatible view to split, and each history must still satisfy the checks. An override that changes sampled conditioning must be removed or replaced with matching source evidence; there is no implicit sample dropping or reconditioning. Missing-only prompt evidence retains the existing fallback and is not certified aligned.

The public additional-histories guide now documents migration, including keeping approximate/manual SFT data separate from captured sampled evidence. Refusal messages report only the source message ordinal, prefix lengths and first mismatch offset; they contain no token IDs, request IDs, text or hashes. Five focused API checks and independent source review cover this guidance/error-only follow-up; the strict decision logic is unchanged.

This forward-ports the experiment-tested repair onto current main while preserving its newer Dynamo metadata handling. It does not change _history.py or include the separate lineage-attribution proposal in #885. The mask exception discriminator uses the exact built-in error, message and deepest helper code rather than a formatting-sensitive line number.

Validation:

• The 18 focused public API compatibility cases pass after updating the three changed-contract expectations; unchanged parent was 18/18 and the initial draft was 15/18. Tests include real history/Pydantic/protocol modules while bypassing unrelated root/backend imports, with networking blocked and no Torch import.
• All 17 focused public source-fixture tests pass, including 4096-token length stops, whole-tail and conditioning negatives, and a decoder that raises the same error text. Ruff passes.
• Static checking of the original port reports the same ten existing source diagnostics and one configuration warning as untouched main in the available environment; its new proof test file has none.
• The preceding frozen repair passed three complete retained CPU cases: 9 histories/20 sampled sources, 5/36, and 21/30. Every full sampled-source prompt, output ID, logprob and STOP mask matched. The forward port has source-composition checks, not a repeated private-corpus run.
• Separate public-template checks verify explicit False remains closed-thinking on the initial render and retry. These use synthetic source IDs and do not claim a newly generated closed-thinking corpus.

Still draft: a subsequent controlled training input reached the guarded exact-source boundary refusal, so the three-case corpus does not establish general full-training coverage. That new case is being investigated separately without weakening the proof. Source review of this disclosed API contract and full project CI remain merge gates. Frozen experiment sources are unchanged by this draft follow-up.

[bradhilton]

Investigation update: later captured refusals were localized without weakening this draft's complete-conditioning guard. The offline literal-thinking renderer fix and preceding-boundary regression are in #967. The ordinary-result STOP-authority gap and successful scoped correction are recorded in #961.

A separate explicit sampled-training prototype retains ordinary success first, catches only deliberate source-owned representation refusals, proves the complete native singleton inventory, and joins contiguous sources only with exact full-prefix nesting. Original conditioning, IDs, logprobs, sampled STOP and source order remain mandatory, with actual first-occurrence ownership before float32 finite filtering. This does not turn multi_history=True into a generic repair and is not included in this PR or ART main.

The latest complete captured CPU validation passed all 31 source encounters from two original histories, including input nonmutation and worker serialization. Of 29 adjacent pairs, 28 were compatible and one was not. It emitted three histories/28,414 tokens versus the previously validated singleton fallback's 31/297,827, preserving 4,139 finite first-occurrence terms and 29 sampled STOP tokens. This is emitted geometry only: the complete singleton proof is still built and retained; speed, memory, GPU fit and numerical training are unmeasured.

Result 025e803dcf5abd3a2f5b897f97b712932e99d33a1c5bb2c9c6e2cf94fa8e02fb; independent alignment e4fa66cbfb4b613f8bfaeef0ad4243c6baadab7a12ebd450e0d46b5a000b05d1. Only bounded scalars are reported, and diagnostic resources are closed.

Native partition/subchain recovery and post-render STOP reconciliation remain source-pinned experiment modules. Upstreaming them needs a separately reviewed explicit sampled API with process/thread dispatch and per-model STOP authority; generic SFT/OUTPUT equivalence and the native-only/no-load contract remain separate. This draft's proof requirements are unchanged.