Preserve forked native conditioning in multi-history tokenization

docs/features/additional-histories.mdx

@@ -159,7 +159,25 @@ for unchanged, complete exchange histories. Recorded logprobs belong to those
 exact conditioned tokens. Chat, Responses, Messages, and Completions keep their
 existing protocol-specific projection rules; no separate tokenization API or
 native-representation option is needed. `multi_history=True` preserves the
-histories selected by the trajectory, including their order and model selection.
+selected source order and model selection. If a complete unchanged Chat history
+cannot use the native fast path and its recorded requests are not one continuous
+token prefix, ART can refine it into adjacent native streams. Each stream retains
+the final request in that run, all sampled responses' original conditioning and
+logprobs, and the original trajectory's reward. The existing first-occurrence
+mask still owns each sampled prefix edge once across streams. Earlier responses
+outside a run are that request's context; their assistant and STOP roles require
+the normal rendering proof. They retain native `EXACT` provenance but carry
+neither `SAMPLED` nor `OUTPUT`, so their generic output/SFT masks differ from an
+unsplit copied-response history. This may load a tokenizer to prove request
+roles. Unproved request roles still refuse.
+
+The internal exchange-training path uses the same refinement and one token weight
+per original trajectory. `History.tokenize()` and `multi_history=False` keep
+their single-linear representation. Explicit overrides, incomplete native
+records, and edited contexts do not authorize splitting. Extra streams may
+repeat context; there is no implied memory, speed, or numerical-training claim.
+Passing `chat_template` or `chat_template_kwargs` explicitly also disables
+refinement when the supplied value equals the recorded settings.
 
 Complete unchanged Chat histories end at the final recorded response token,
 including tool responses and responses stopped by a length limit. ART does not
@@ -240,13 +258,15 @@ Unused opaque request metadata remains valid on complete callback-free native
 paths, including a supplied tokenizer exposing only plain EOS metadata;
 callbacks require context that ART can compare.
 
-A response copied into a later, shortened prompt is output provenance, but it is
-not a fresh sample under that new prompt. ART keeps its `OUTPUT`, `ASSISTANT`,
+In an unsplit history, a response copied into a later, shortened prompt is output
+provenance, but it is not a fresh sample under that new prompt. ART keeps its
+`OUTPUT`, `ASSISTANT`,
 `EXACT`, and proven `STOP` flags while removing `SAMPLED` and the old conditional
 logprob. This requires the complete original sampled occurrence to remain in an
-earlier selected history; otherwise unchanged native replay is refused. The
-original occurrence retains its logprobs and ownership, including recorded NaNs
-before finite-value filtering. Tokenizing only the shortened view cannot prove
+earlier selected history. Multi-history tokenization may establish that coverage
+by refining complete native sources into streams; if it cannot, native replay
+is refused. The original occurrence retains its logprobs and ownership, including
+recorded NaNs before finite-value filtering. Tokenizing only the shortened view cannot prove
 that coverage; tokenize the containing trajectory instead. This correction does
 not change how generic output/SFT masks include copied assistant content.