The two-stage LoRA rebind cost is measured: load-fuse is immediately undone by phase 0, and each pass is hours at #1202's rate

[localai-bot]

src/vllm/multimodal/ltx2_video.cpp:2843-2851 states the two-rebind cost of a two-stage render and records it as UNMEASURED on real weights, with a later perf row owning the number. This issue supplies that number. It is far worse than the trade appears, because each rebind runs the scalar fusion loop in #1202.

The design, as written

The comment is accurate and anticipated this:

WHAT A TWO-STAGE RENDER PAYS IS TWO REBINDS, not one [...] a2vid_two_stage loads FUSED, phase 0 asks kNoAdapters and rebinds off, phase 1 asks kAllAdapters and rebinds back on; the DiT is left fused, so the NEXT render pays the same two. [...] That cost is UNMEASURED on real weights.

So on a two-stage recipe the sequence is fuse at load -> un-fuse for phase 0 -> re-fuse for phase 1. The first and second cancel exactly: the load-time fusion is undone before any denoise step runs.

The measurement

On dgx (GB10, 20 cores), full/dev transformer (21,004,025,600 params) with the shipped 8.9 GB distilled adapter:

• three gdb stacks, all identical: vt::BF16ToF32 <- Ltx2FuseLoraIntoTensor <- Ltx2LoadDitFromSafetensors <- Ltx2VideoEngine::Load
• one thread at 99.9% of one core, 19 idle
• f32 working set grew 9.432 -> 10.235 GiB over 300-629 s: 2.3% of one pass in 10.4 minutes
• independent cross-check: sum of out*in*rank over the 1660 targeted modules = 8.53e12 MAC, consistent with ~0.53 GFLOP/s

Each pass is therefore hours, not seconds, and a two-stage render pays three of them (load + two rebinds) before the first denoise step.

Why this is worth its own issue rather than being folded into #1202

Fixing #1202 shrinks the constant but leaves the shape: the load-time fusion on a two-stage recipe is still provably wasted work, immediately undone by the phase-0 rebind. Two independent fixes are available and they are not the same change:

1. Make the fusion fast (Ltx2FuseLoraIntoTensor is a scalar single-threaded loop (~0.53 GFLOP/s), blocking every LoRA arm on the full 21B DiT #1202) — routes the rank-r product through the vt:: GEMM seam.
2. Do not fuse what phase 0 will immediately un-fuse. The recipe's phase 0 scope is known before Load runs, so the load could honour it and skip a full fuse/un-fuse round trip. Ltx2PipelineRecipe::phases is available at that point.

The comment notes the DiT is left fused so the next render pays the same two rebinds again — worth revisiting in the same breath, since the terminal state is chosen rather than forced.

Scope

Affects every two-stage LoRA-bearing recipe, a2vid_two_stage and ti2vid_two_stage among them. one_stage is unaffected and pays nothing, which is why it is currently the only full-model arm that reaches generation.

Not claimed

The wall-clock figures above are a rate measured over a 10.4-minute window and extrapolated, not a completed pass. No two-stage full-model render has completed, so the end-to-end number remains open. The rate, the thread count and the stack attribution are measured.

Credit: found while attempting the first full-model render; the design comment predicted exactly this and asked for the number.