Reuse exact chat-template prefixes during long-history tokenization

[bradhilton]

Long histories with structured tool calls repeatedly render the same earlier prefixes while proving each sampled-output boundary. On a retained 229,871-token history, this change reduced uninstrumented tokenization from 433.04s to 263.00s (39.3%, 1.65x). The entire serialized tokenized result and every field digest matched the baseline.

This change reuses exact unchanged prefixes within a single history and normalizes tool arguments once per eligible segmented render. It still runs the existing assistant-span and output-boundary proofs. Reuse requires stock Transformers rendering, plain JSON context, and a conservative deterministic Jinja subset; other renderers retain the existing path. Prefix results use one baseline plus bounded deltas instead of retaining quadratic text.

Validation at ca8e021d6a53831bab4b7524b916694c201e67f6:

• 722 trajectory tests passed, including full tokenized-field/source-key differentials, native logprobs/stops, thinking on/off, reasoning/refusals, changed later generation prompts, bounded storage, text-only behavior, and custom-renderer/Jinja fallbacks.
• Ruff and pinned ty 0.0.59 passed for all four changed files. Required CI is green (Prek run 36098935013: 1,161 lightweight tests and 2,009 unit tests passed; skips remain explicit in the logs).
• Independent Astra and Cursor Fable source reviews are CLEAR. Two earlier eligibility findings were fixed and rereviewed on this head.
• Offline, single-thread CPU witness used the same frozen input and cached Qwen3.8-27B tokenizer revision; all worker groups retired. Peak RSS was approximately 1.66 GiB in both runs, including post-timing result hashing.

The improvement is a constant factor on one long-history witness: later changed prefixes still render, so the remaining quadratic render-call growth is not eliminated. Unsupported templates retain ordinary rendering. No public API, training behavior, or frozen-run adoption changes. Validation and source reviews are complete; this PR is ready for final review and is intentionally unmerged.