[bugfix] render chat-template diffs without the generation prompt in SampleBuilder

[MayDomine]

Fixes #2.

Problem

SampleBuilder.append_text (meshy/utils/sample.py) built the prompt token stream incrementally by diffing successive apply_chat_template renders that each carried add_generation_prompt=True. Such renders are not prefixes of each other, so slicing new_ids[len(old_ids):] produces a corrupted stream for every multi-turn prompt.

Reproduced with the Qwen/Qwen3-1.7B tokenizer on a system+user pair: the builder emits

<|im_start|>system\n...<|im_end|>\n<|im_start|>assistant\nWhat is 2+2?<|im_end|>\n<|im_start|>assistant\n

— the <|im_start|>user header is replaced by a stray <|im_start|>assistant header, and the user text is masked/positioned as if it were assistant output. The token count is identical to the correct render, so nothing crashes; quality just degrades.

Blast radius: meshy/dataset/gsm8k.py (system+user — every prompt of the default grpo_gsm8k recipe), meshy/dataset/s9_math.py (any multi-message prompt). meshy/dataset/math.py builds a single user message and is unaffected, which is why this stayed hidden.

Fix

• build_sample now diffs renders made without the generation prompt — those are true prefixes of each other in ChatML-family templates — and appends the generation prompt once at the end (mask 0), so sample.tokens is exactly the prompt the inference engine should see.
• append_text is removed: its only caller was build_sample, and its stateless "append one message with the generation prompt" semantics cannot be made correct for a message that is not the last one. append_tokens (used by the rollout worker for engine output) is unchanged.
• A _render helper normalizes the apply_chat_template return value, which is a BatchEncoding on some transformers versions and a plain id list on others.
• scripts/bench_train_only.py's _Tokenizer copied the same diff logic and is fixed the same way; it additionally inserts the generation prompt before each assistant turn, matching what the engine was actually prompted with during the logged run.
• New tests/test_sample_builder.py: a deterministic ChatML-style fake tokenizer (no model download) asserts the built stream equals a one-shot add_generation_prompt=True render for single-, two- and four-message conversations, that masks cover exactly the assistant span, and that both apply_chat_template return types work. The equality tests fail on the old code.

Verified against the real Qwen3-1.7B tokenizer that the fixed builder output for [system, user] matches the one-shot render exactly.

Behavior changes

• Prompt token streams for multi-turn samples change (they are now correct), so reward/loss curves from runs before this fix are not directly comparable.
• SampleBuilder.append_text no longer exists; out-of-tree callers should use build_sample or append_tokens.
• Templates that rewrite earlier turns in longer renders (e.g. Qwen3 strips the empty <think> block from an assistant turn once a later message follows) still get a self-consistent stream, but it can differ from a one-shot render by exactly that rewritten span. No in-tree dataset puts assistant turns in the prompt, so this does not affect bundled recipes.

[zTonyZhao]

LGTM.