LTX-2.5 T2A guidance is combined in VELOCITY space; upstream combines x0, and the default rescale_scale=0.7 makes them differ

[localai-bot]

Found during the rebase of #1032 (row/LTX25-T2A-ONE-STAGE), verified in the source at head 3d9d9c9bb812c531b08c1443d4a4530b55f736fe. The branch is not merged, so this is a defect in code about to land rather than on main.

What upstream does

Upstream hands the denoiser an X0Model, so the guider combines denoised (x0) predictions:

• packages/ltx-pipelines/src/ltx_pipelines/utils/blocks.py:480-482 — the transformer is an X0Model
• packages/ltx-core/src/ltx_core/model/transformer/model.py:601-604 — X0Model.forward returns to_denoised(latent, v, timesteps)
• packages/ltx-core/src/ltx_core/utils.py:39-52 — to_denoised = sample - velocity * sigma
• packages/ltx-pipelines/src/ltx_pipelines/utils/denoisers.py:188-203 — the guider combines those x0 tensors

What we do

We combine the raw DiT velocities and convert once afterwards. At src/vllm/model_executor/models/ltx2_t2a.cpp, the local is named velocity, is assigned cond.audio — an Ltx2DitForward output — and is what the guider receives:

std::vector<float> velocity = cond.audio;
...
velocity = Ltx2MultiModalGuidance(g, cond.audio.data(), ...);
...
last_denoised = ToDenoised(latent, velocity, timesteps, tokens, width);

ToDenoised runs on the result of the guidance, not on its inputs.

Why it is not merely a refactor

The linear terms are invariant under x0 = latent - sigma*v, so the two forms agree exactly whenever rescale_scale == 0. The rescale branch is not invariant. At src/vllm/model_executor/models/ltx2_pipeline.cpp:504-522:

const float factor_raw = unbiased_std(cond) / unbiased_std(pred.data());
const float factor = rescale_scale * factor_raw + (1.0 - rescale_scale);
for (float& value : pred) value *= factor;

Ours is std(v_cond)/std(v_pred) scaling the velocity; upstream (guiders.py:268-271) is std(latent - sigma*v_cond)/std(latent - sigma*v_pred) scaling the whole x0. Scaling v by f yields latent - sigma*f*v, while scaling x0 yields f*(latent - sigma*v). The two differ by (f-1)*latent.

The default makes this the normal path, not a corner

rescale_scale = 0.7 is the shipped T2A default — params.audio_guider.rescale_scale = 0.7 at ltx2_pipeline.cpp:900, mirroring utils/constants.py:63 and args.py:1101-1106. So every default T2A render takes the divergent branch.

Magnitude — illustrative, NOT measured

On synthetic tensors (NumPy, cfg 7.0 / stg 1.0 / sigma 1.0) the two forms differ by 18.17 % relative RMS at rescale_scale = 0.7, and agree to 5.7e-6 at rescale_scale = 0.0 — the latter being the control that localises the divergence entirely to the rescale branch.

This is algebra on synthetic inputs. It is not a measurement against upstream on real weights: dgx.casa is currently down, and no LTX-2.5 checkpoint was loaded. The direction of the finding is proven by source; the magnitude on a real render is unmeasured and this issue does not claim it.

Nothing can currently see it

Every T2A assertion in the suite is self-consistency — arms differ, forward counts move — and there is no oracle-derived numeric golden for the guided step. A token or shape gate cannot see a guidance-space error, in the same way a token gate cannot see a dtype that is too wide.

What a fix owes

• Combine in x0 space, mirroring denoisers.py:188-203, or equivalently move the rescale into x0 space so the factor is computed and applied on latent - sigma*v.
• A red-first test that fails on the current form and passes on the mirrored one, with rescale_scale at its 0.7 default so the assertion sits on the shipped path. The rescale_scale = 0.0 agreement is the natural control.
• Note that the guider is shared with the video path (params.video_guider), so check whether the video arm has the same shape before fixing only the audio one.

Blocks #1032 in my view, because a default-path divergence from the reference is exactly what "Mirror vLLM" — here Lightricks, per .agents/oracles/ — exists to prevent. Related: #1005, #1013, campaign #644.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]

[localai-bot]

Fixed and gated on main.

The fix landed with #1032 (0b0b8900f). The per-pass lambda in src/vllm/model_executor/models/ltx2_t2a.cpp:322 is X0Model: it applies ToDenoised on the way out of every Ltx2DitForward (:333), so all three arms — cond :336, uncond :352, perturbed :362 — reach Ltx2MultiModalGuidance already denoised. That mirrors upstream's structure rather than merely its numbers: upstream hands _guided_denoise an X0Model (blocks.py:480-482, denoisers.py:61-62), and X0Model.forward returns to_denoised(latent, v, timesteps) (model.py:590-604). Moving the rescale instead would have put to_denoised inside calculate, where upstream does not have it.

A fresh reviewer confirmed the mirror against upstream at fd4ded7f, verified the conversion is applied exactly once on every path including the should_skip_step reuse, that nothing downstream still expects a velocity (Ltx2EulerStep recomputes it internally), and that the video arm is genuinely unaffected — Ltx2MultiModalGuidance has exactly one production call site, and the joint driver runs one unguided forward per step, which is the same tensor in either space.

The gate went further than the first fix. The review found three surviving mutations — perturbed-arm-only, uncond-arm-only, and a double ToDenoised — all passing green, meaning the original gate held the "every pass" claim for the conditional pass alone. The repair then found a fourth: the reviewer's double-application sat between the step-0 record and the Euler step, so the Euler check caught it, but one statement earlier it survived. Closing both needed two independent checks. All four now go red.

I verified this myself rather than accepting the table: mutating the perturbed arm back into velocity space builds with 0 compile errors, runs exit 1, and fails on exactly the two per-arm equation checks. (My first attempt referenced a lambda the repair had renamed, failed to build with 1 error, and established nothing — recording that because a non-building mutation reads exactly like a passing test.)

What is deliberately still owed, recorded under ## Owed in .agents/specs/ltx25-t2a-one-stage.md rather than left silent: the rescale's end-to-end numeric consequence is not asserted. It cannot be on the reduced fixture — measured, not assumed: the DiT responds to conditioning at ~1e-5 of its own output, so std(cond)/std(pred) is 1.0 to 1e-5 in both spaces and the two candidates separate by 7.6e-07 against a span of 3.41. A test there would have passed whether or not the bug existed. What is gated instead is the invariant cond == latent - sigma*velocity (exact in x0 space, off by the whole sample in velocity space — the RED prints |cond - velocity| = 0 exactly), plus the seam directly: 1.50e-07 disagreement at rescale 0.0 against 0.352 at the shipped 0.7.

The 18.17 % figure in the original report was synthetic-tensor algebra and was never restated as a measurement. Closing the defect; the real-weights verification rides with the checkpoint-pin work (#1048).