Ltx2FuseLoraIntoTensor fuses a LoRA delta with a scalar single-threaded triple loop, measured at ~0.53 GFLOP/s. On the full 21.004 B LTX-2.5 DiT this makes every LoRA-bearing pipeline kind unrunnable — the fusion alone projects to hours before a single denoise step.
Where
src/vllm/model_executor/models/ltx2_lora.cpp:321-334 — the (B * strength) @ A product:
for (int64_t o = 0; o < rows; ++o) {
const uint16_t* brow = bs.data() + o * pair->rank;
for (int64_t i = 0; i < cols; ++i) {
float acc = 0.0F;
for (int64_t k = 0; k < pair->rank; ++k) {
acc += vt::BF16ToF32(brow[k]) * vt::BF16ToF32(pair->a[k * cols + i]);
}
agg[o * cols + i] = vt::F32ToBF16(acc);
}
}One thread, no blocking, no SIMD, and a non-inlined vt::BF16ToF32 call per multiply. The inner operand pair->a[k * cols + i] strides by cols so every one of the rank loads in the innermost loop is a separate cache line.
Called from ltx2_loader.cpp:537 via FuseLorasInto, which runs per tensor at load from both PlanDit arms (:681, :746).
Measured
On dgx (GB10, 20 cores) loading the full/dev transformer with the distilled adapter:
- three
gdb stacks, all identical: vt::BF16ToF32 <- Ltx2FuseLoraIntoTensor <- Ltx2LoadDitFromSafetensors <- Ltx2VideoEngine::Load
- one thread at 99.9% of one core; the other 19 idle
- over 300-629 s the f32 working set grew 9.432 -> 10.235 GiB: 2.3% of one pass in 10.4 minutes
- cross-check: sum of
out*in*rank over the 1660 targeted modules = 8.53e12 MAC, which at the observed rate is hours of wall
Why it matters
This is not a slow path that costs a few seconds. It sits between Load and any generation, so it blocks:
- every pipeline kind that carries the distilled LoRA on the full model
- the IC-LoRA kinds
The only full-model arm that renders today is one_stage, and it works precisely because upstream marks it Full with no adapter, so it never enters this function.
What it should be
The operation is a rank-r GEMM ([rows, rank] x [rank, cols]), which is exactly what vt:: already does well elsewhere. Route it through the existing GEMM seam rather than hand-rolling it, and keep the bf16 rounding pattern the surrounding comments justify (fuse_loras.py:103-116): B * strength rounds to bf16 before the product, accumulation is f32, the store is bf16. A correct port of the arithmetic is already here; only the execution strategy is wrong.
Whatever replaces it must keep the Fail on a shape mismatch and the second-adapter refusal, both of which have tests.
Verification
Ltx2FuseLoraIntoTensor is bit-exact against the recorded goldens today, so the replacement has an exact oracle: fuse the same adapter both ways and assert byte equality, not a tolerance.
Ltx2FuseLoraIntoTensorfuses a LoRA delta with a scalar single-threaded triple loop, measured at ~0.53 GFLOP/s. On the full 21.004 B LTX-2.5 DiT this makes every LoRA-bearing pipeline kind unrunnable — the fusion alone projects to hours before a single denoise step.Where
src/vllm/model_executor/models/ltx2_lora.cpp:321-334— the(B * strength) @ Aproduct:One thread, no blocking, no SIMD, and a non-inlined
vt::BF16ToF32call per multiply. The inner operandpair->a[k * cols + i]strides bycolsso every one of therankloads in the innermost loop is a separate cache line.Called from
ltx2_loader.cpp:537viaFuseLorasInto, which runs per tensor at load from bothPlanDitarms (:681,:746).Measured
On
dgx(GB10, 20 cores) loading the full/dev transformer with the distilled adapter:gdbstacks, all identical:vt::BF16ToF32<-Ltx2FuseLoraIntoTensor<-Ltx2LoadDitFromSafetensors<-Ltx2VideoEngine::Loadout*in*rankover the 1660 targeted modules = 8.53e12 MAC, which at the observed rate is hours of wallWhy it matters
This is not a slow path that costs a few seconds. It sits between
Loadand any generation, so it blocks:The only full-model arm that renders today is
one_stage, and it works precisely because upstream marks itFullwith no adapter, so it never enters this function.What it should be
The operation is a rank-
rGEMM ([rows, rank] x [rank, cols]), which is exactly whatvt::already does well elsewhere. Route it through the existing GEMM seam rather than hand-rolling it, and keep the bf16 rounding pattern the surrounding comments justify (fuse_loras.py:103-116):B * strengthrounds to bf16 before the product, accumulation is f32, the store is bf16. A correct port of the arithmetic is already here; only the execution strategy is wrong.Whatever replaces it must keep the
Failon a shape mismatch and the second-adapter refusal, both of which have tests.Verification
Ltx2FuseLoraIntoTensoris bit-exact against the recorded goldens today, so the replacement has an exact oracle: fuse the same adapter both ways and assert byte equality, not a tolerance.