Preserve literal think markers in thinking-off histories

[bradhilton]

Qwen3.5 history rendering can interpret literal </think> markers as reasoning separators even when the captured generation disabled thinking. For example, HEAD\n</think>DISCARDED_PUBLIC_SEGMENT</think>\n\nTAIL loses its middle segment in the initial render.

This also affects the preceding turn. Parsing the next assistant's literal markers as nonempty reasoning makes its inferred span start at the open <think>\n prefix, before the rest of the served thinking-off scaffold \n</think>\n\n. The preceding length-stop boundary then ends too early and fails the native prompt check. A captured scalar diagnostic confirmed that branch and suffix mismatch; no private message text is included here.

Preserve literal content by adding an empty structured reasoning field only to the offline render copy, for the exact supported preserved template and a native choice whose own request explicitly disabled thinking. Check the original response as well as the normalized message so visible-only histories cannot hide structured reasoning. Explicit/aliased reasoning, thinking-on/unknown sources, request-only messages, unsupported templates, and original history objects retain their existing behavior. Inference templates and prefix/boundary/STOP predicates are unchanged.

Validation:

• The original 27 public CPU tests pass; Ruff and focused ty checks pass for this update. Current-head quality checks and the trainer-validation classifier pass; the GPU workload was skipped.
• One additional maintained regression exercises the complete history-tokenization path with the exact public template and a reversible newline-run codec. It observes the preceding boundary's two-token shortfall and original builder rejection, then checks corrected boundary acceptance, native arrays/logprobs, STOP flags and source nonmutation. The production fix is unchanged.
• Separately, 27 public controls using the actual offline Qwen tokenizer confirm the same contrast: stored/native boundary22/24 with literal markers,24/24 after the fix. Native IDs, flags, logprobs and sampled first-occurrence masks are preserved. Those public inputs are different from the captured failure; matching counts were not used as causal proof.

On the original captured input, the isolated minimal overlay let the original process worker return both histories (28,282 tokens) through ordinary tokenization, without recovery. The input was unchanged and serialized worker output matched the observed output. This establishes that the rendering refusal cleared on that input.

The literal-only validation still refused on the separate STOP-authority gap tracked in #961: one missing sampled STOP bit and zero extra bits were observed before refusal, with only three of six source proofs completed. These were partial counters, not evidence of exactly one missing bit overall.

A subsequent separate sampled-worker correction preserved this renderer fix and added model-bound STOP reconciliation after ordinary tokenization. On the same captured input, all six complete source-conditioning, token-ID, logprob, sampled STOP and objective checks passed: 8,881 finite first-occurrence terms and four sampled STOP tokens, with zero missing or extra STOP bits. One copied history received three certified STOP additions; the original input, both original history arrays and all six native source arrays were unchanged. The result remained two histories and 28,282 tokens. That correction is not included in this PR and does not resolve the generic native-only/no-load API contract.

This change contains only the rendering-copy fix and its public regressions. Separate strict conditioning work in #957 and the explicit sampled-contract native partition/subchain recovery remain outside this patch; the latter is currently an experiment integration, not a generic ART API. This branch preserves its base's existing checks. No frozen run or inference template was changed, and this PR adds no GPU/model workload or numerical-training/performance claim.

[bradhilton]

Current-head review at e65e4b18f6678f133c2e454698c6f7f6a0a2f66e: no unresolved correctness finding within this exact-template, offline-render-copy fix. The current source, public template and added regression match the previously reviewed bytes; source validation, native token evidence and the remaining tokenizer predicates are unchanged. The retained public tests and current-head quality CI pass. Trainer validation classified these paths as outside its GPU trigger, so the GPU job was skipped. No tests were rerun for this readiness review.

Ready for maintainer review. The captured literal-only worker now returns normally, while its independent STOP-authority refusal remains documented in #961. The separate explicit sampled-contract correction passed the complete six-source objective proof; that reconciliation and native subchain recovery are outside this PR. No frozen run, runtime pin or inference template changes accompany this review.