From 3e6fb648279011f0a0e4c95a742334e204c335c1 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Sun, 16 Aug 2026 15:59:21 +0000 Subject: [PATCH 1/4] spec(LTX25-DECODE-THREADS): the decode has 20 cores and uses one, and the seam it needs is already in the tree Lever 3 of the `LTX25-DECODE-SPEED` investigation (#1006, PR #1018). Issue #1009. `ParallelForRows` (`src/vt/cpu/cpu_threadpool.cpp:413`) is synchronous and used by 10+ CPU kernels in this tree. Zero of them are in the LTX-2.5 video VAE decode, whose 42 convolutions carry ~7.25 TFLOP at 448x256/25f and take 2681 s on one core of twenty. The spec commits to the axis before the code exists, because the axis is the whole risk. The sibling dtype row (#1008, `d1b0ea3a8`) had to change this convolution's summation ORDER to a blocked one to stay inside a 5e-06 tolerance, and parallelism is the second thing that can change a summation order. The partition taken is the output line `(oc, ti, hi)`: the `ci * kernel^3` reduction stays entirely inside one output element's body, so a worker executes exactly the serial instruction sequence for every element it owns, and the result does not depend on the worker count or on which worker stole which chunk. Splitting the reduction axis `ic` into per-thread partials is the alternative, and it is rejected in writing: it would make the summation order a function of the thread count. Two gate cases are declared rather than one, because a determinism A/B is green on a serial implementation too. The second case observes the dispatch itself through the pool's public work-stealing cursor, which is the case that is RED before the change. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code] --- .agents/specs/ltx25-decode-threads.md | 285 ++++++++++++++++++++++++++ 1 file changed, 285 insertions(+) create mode 100644 .agents/specs/ltx25-decode-threads.md diff --git a/.agents/specs/ltx25-decode-threads.md b/.agents/specs/ltx25-decode-threads.md new file mode 100644 index 000000000..3e7581a8c --- /dev/null +++ b/.agents/specs/ltx25-decode-threads.md @@ -0,0 +1,285 @@ +# LTX25-DECODE-THREADS — the decode runs on one core of twenty, and the seam it needs already exists + +Row: `LTX25-DECODE-THREADS`, under the `ROAD-V1-LTX25` campaign +([`roadmap_v1.md`](../roadmap_v1.md), [`ltx-2-5.md`](ltx-2-5.md)). +Issue: [#1009](https://github.com/mudler/vllm.cpp/issues/1009). +Parent: lever 3 of the `LTX25-DECODE-SPEED` investigation +([#1006](https://github.com/mudler/vllm.cpp/issues/1006)), which filed this +issue and lists it under `## Owed`. That spec is +`.agents/specs/ltx25-decode-speed.md` on [PR +#1018](https://github.com/mudler/vllm.cpp/pull/1018) and is **not yet on +`main`**, so it is cited by pull request rather than by relative link, exactly as +the sibling dtype row ([`ltx25-decode-dtype.md`](ltx25-decode-dtype.md)) does. + +Sibling, and the reason this row is riskier than it looks: +[`ltx25-decode-dtype.md`](ltx25-decode-dtype.md) (#1008) landed at `d1b0ea3a8` +and changed the convolution's **summation order** to a blocked one. This row +adds parallelism on top of that, and parallelism is the second thing that can +change a summation order. + +## Now + +`ACTIVE`. + +## 0. Scope + +**In scope.** Route the LTX-2.5 conv video VAE's convolution loops through +`vt::cpu::ParallelForRows`, the synchronous row-chunked parallel-for that 10+ +CPU kernels in this tree already use and that no line of the video VAE uses +today. + +**Not in scope, and deliberately so.** + +* The device arm ([#1007](https://github.com/mudler/vllm.cpp/issues/1007)). + There is no `vt::` conv3d op on any backend; that is a much larger change and + needs NDHWC first. +* NDHWC / memory format ([#1008](https://github.com/mudler/vllm.cpp/issues/1008) + §5 records the verdict and the blocker, `MiniMaxH3GroupNorm3d`'s signature). +* `memory_efficient_decode.py` ([#1011](https://github.com/mudler/vllm.cpp/issues/1011)). +* SIMD. A vectorised inner tap loop is a separate change with a separate + summation-order question, and mixing the two would make an order regression + unattributable. +* The **audio** VAE and the video **encoder**'s non-convolution paths. The + encoder shares `CausalConv3d`, so it inherits the change; nothing else in + either file is touched. + +**No end-to-end render number.** `dgx.casa` is unreachable and this box has no +GPU, so this row claims no render speedup and no ratio against any oracle. What +it can measure, and does, is a same-binary wall-clock A/B of the decode itself +at fixed thread counts on 20 local cores (§6). + +## 1. Why there is no upstream to mirror here + +Every oracle runs this decoder on an accelerator and none of them has a +host-parallel arm to port: + +* Lightricks LTX-2 @ `fd4ded7f2` builds the decoder onto a device + (`packages/ltx-pipelines/src/ltx_pipelines/utils/blocks.py:1139`, + `packages/ltx-core/src/ltx_core/loader/single_gpu_model_builder.py:267-288`); + the whole decoder's convolution work is one `nn.Conv3d` call at + `model/video_vae/convolution.py:312`. +* SGLang @ `f63458b5b` moves the latents to the local torch device + (`python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/ltx_2/decoding_av.py:71`). +* vLLM-Omni @ `a4ea67a21` states the contract outright — *"VAE(s) (always on + GPU)"*, `vllm_omni/diffusion/models/interface.py:92`. +* `diffusers` @ `3a2f35d4e` ships no CPU decode path at all + (`ltx2_diffusion_decoder.py:208-209`, *"No CPU path"*). + +So this row is a **local seam**, not an upstream mirror, and +[`ltx25-decode-speed.md`](ltx25-decode-speed.md) §6 lever 3 records it as such. +What it does mirror is *this tree's own* CPU convolution: +`src/vt/cpu/cpu_conv2d.cpp:75-78 @ d1b0ea3a8` partitions a 2-D convolution over +`n * cout * hout` output lines through the same call, with the same comment this +row's change carries — *"independent outputs, so the partition can never change a +reduction"*. + +## 2. The axis, and why it is reduction-safe + +`CausalConv3d`'s output loop nest is `oc / ti / hi / wi` +(`src/vllm/model_executor/models/ltx2_video_vae.cpp:178-217 @ d1b0ea3a8`). The +`ci * kernel^3` reduction lives **entirely inside one `(oc, ti, hi, wi)` body**, +in the blocked order #1008 shipped: one `float tap` partial per input channel, +added into `float acc`. + +**The parallel axis is the output line `(oc, ti, hi)`, and each unit is +`out.w` contiguous output elements.** `Volume::At(oc, ti, hi, wi)` is +`((oc*t + ti)*h + hi)*w + wi`, so output line `r` is exactly the contiguous span +`[r*out.w, (r+1)*out.w)` of `out.data`. + +Three properties follow, and together they are the determinism argument: + +1. **No output element is written by more than one worker.** The partition is a + partition of `r`, and lines do not overlap. +2. **No reduction crosses a worker boundary.** Every accumulation — over `ic`, + over `a`, `b`, `d` — is inside one `wi` iteration of one line. A worker + executes exactly the instruction sequence the serial arm executes for that + element, in the same order, on the same values. +3. **The result therefore does not depend on the worker count**, and it does not + depend on which worker took which chunk either. That matters, because + `ParallelForRows` (`src/vt/cpu/cpu_threadpool.cpp:413-458`) **steals work** + through an atomic cursor, so the row-to-thread assignment is genuinely + non-deterministic run to run. Bit-identity has to survive that, and it does, + for reason 2. + +This is not a new contract. `src/vt/cpu/cpu_threadpool.h:39-43` already states +it for the whole CPU backend: *"parallelism partitions OUTPUT elements only ... +No atomic accumulation into shared outputs, no reduction-order changes — results +are bit-identical to n_threads==1 by construction."* This row's job is to stay +inside that contract, not to invent one. + +**What was rejected, and why.** Parallelising over the *reduction* axis `ic` +with per-thread partials and a final combine would also be a legal +convolution — and it would change the summation order as a function of the +thread count, which is exactly the defect #1008 spent its budget removing. It is +not taken, and no tolerance is widened anywhere in this row. + +**`ParallelForRows` is synchronous**, so the `[&]` capture of the local `padded`, +`weight`, `out` and the loop bounds is safe: `Run` returns only after every +worker has passed the closing `Barrier()` inside `ComputeThread` +(`cpu_threadpool.cpp:234`), whose exit is a seq-cst fence (`:206-212`). + +## 3. The sites + +| site | what it is | parallel unit | rows | +|---|---|---|---| +| `ltx2_video_vae.cpp` `CausalConv3d`, output nest | 42 convs, ~all of the decode's FLOPs (`ltx25-decode-speed.md` §1.1) | one output line `(oc, ti, hi)` | `out_channels * out.t * out.h` | +| `ltx2_video_vae.cpp` `CausalConv3d`, pad gather | the replicate/reflect pad materialisation | one padded line `(c, ti, hi)` | `ci * pt * ph` | +| `ltx2_video_vae.cpp` `Linear3d` | the 1x1x1 conv used as `conv_shortcut` | a contiguous span of `(oc, i)` | `out_channels * in.spatial()` | + +The pad gather has no reduction at all — it is a pure gather, one source element +per destination element — so it is trivially order-independent. It is included +because it is `O(ci * pt * ph * pw)` inside the same function and would otherwise +become a serial section that bounds the speedup by Amdahl's law. + +Sites **not** taken, each for a stated reason: + +* `PixelNorm`, `Silu`, `ApplyAdaLn`, `expand`, `drop_first_frame` — memory-bound + elementwise passes. They are candidates, but they are not where the 7.25 TFLOP + is, and each one added is another surface for a reviewer to check. Owed + (§7) rather than done silently. +* `FeedSpatialNoise` — **must not** be parallelised. It consumes + `Ltx2NoiseStream` in call order + (`include/vllm/model_executor/models/ltx2_video_vae.h:201-210`), and that call + order is the reproducibility contract with upstream's `torch.Generator`. The + draw itself is already outside the loop; the loop that applies the plane could + be partitioned, but the win is nil and the risk is a later edit moving the draw + inside. Left alone deliberately. +* `AttnBlock3d` — the shipped decoder cannot construct an attention block at all: + `attn_res_x` is refused by name (`ltx2_video_vae.cpp:10-14`), because upstream + at the pinned revision cannot construct it either. + +## 4. Risks + +* **A partition that changes the summation order.** The one risk that can change + the design, and the one that bound on the sibling row. Mitigation: §2's axis, + plus the golden margins measured before and after and required to be + **exactly equal**, not merely within tolerance. Any movement at all in a + recorded `max|diff|` means the order moved and the design is wrong. No + tolerance is widened; that is the stop condition (§8). +* **A result that depends on the thread count.** Mitigation: the determinism + case in §5, which decodes the same input at five different worker counts and + requires `memcmp == 0`. +* **A data race.** New concurrency in a file that had none. Mitigation: the + ThreadSanitizer lane over the LTX suites (§6), because a race in a parallel + reduction is precisely the defect this row could introduce and CI's sanitize + lane is unreliable — it was cancelled in 4 of the last 12 `main` runs. +* **Nested dispatch.** `Threadpool::Run` throws on a dispatch from inside a + parallel region (`cpu_threadpool.cpp:355`). The decode is called from + `Ltx2VideoDecodeStreaming`, which is called from + `src/vllm/multimodal/ltx2_video.cpp:3258` on the render path, and no caller in + that chain is inside a parallel region. Checked by reading the chain, and the + full gate would throw loudly if it were wrong. +* **A determinism test that measures nothing.** Two runs of a *serial* + implementation are also bit-identical, so the determinism case alone is green + before this row's change. That is why §5 ships a **second** case that observes + the dispatch itself. + +## 5. The gate + +Two new cases in `tests/vllm/models/test_ltx2_vae.cpp`, both entering through +the production entry point `Ltx2VideoDecodeStreaming` — the one +`src/vllm/multimodal/ltx2_video.cpp:3258` calls on the render path, reaching +`Ltx2ConvVideoDecode` through `ltx2_video_vae_tiled.cpp:113`. + +**Case A — the decode dispatches partitioned work to the CPU threadpool.** This +is the case that is RED before the change. A fresh `vt::cpu::Threadpool` is +installed with `SwapForTesting`, and its work-stealing cursor is read through +the public `ChunkAdd(0)`, which returns the current value and adds nothing. The +cursor is `0` on a fresh pool; `ParallelForRows` seeds it with `ChunkSet(nth)` +and every steal advances it (`cpu_threadpool.cpp:437-455`). So a non-zero cursor +after a decode is a direct observation that a multi-chunk partitioned dispatch +ran on that pool, and a zero cursor is the observation that none did. Before this +row the decode never touches a pool, so the case reads `0` and fails. + +The same case asserts the decoded output against an analytically derived value +rather than a recorded one, so a decode that never ran cannot pass it. This is +the trap the sibling row hit and recorded: a zero-filled stub satisfies an +expectation of zero. The fixture therefore offsets `conv_out.conv.bias` off +zero, exactly as the width case does. + +**Case B — the decode is bit-identical across thread counts.** The same latent +is decoded at worker counts 1, 2, 3, 5 and 8 and every result is `memcmp`-equal +to the 1-thread arm. Worker count 1 short-circuits `ParallelForRows` to +`body(0, nr)` on the caller (`cpu_threadpool.cpp:423-426`), so the 1-thread arm +*is* the pre-change code path, byte for byte. The counts are deliberately not +all powers of two: 3 and 5 do not divide the row counts, so the chunk boundaries +land in different places on every arm. + +Case B also asserts that the decoded volume is not degenerate — that it holds +more than one distinct value — because an all-equal buffer, which is what a +stubbed decode returns, would satisfy a pure A-equals-B comparison. + +**And the existing goldens become a threading gate for free.** The full suite +runs on the global pool, which is `hardware_concurrency` wide (20 here), so every +LTX-2.5 video golden already executes the threaded path. Their recorded +`max|diff|` values from #1008 are the before-picture, and §6 requires them to +come back **identical**. + +## 6. Gates and evidence + +```sh +cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DVLLM_CPP_CUDA=OFF +cmake --build build -j6 +ctest --test-dir build -j4 --output-on-failure +``` + +Reported: `CONFIGURE_EXIT`, `BUILD_EXIT`, the `: error:` count, `ctest -N`, +`CTEST_EXIT`, the full pass/fail line, `No space left` and `BFD assertion` with +positive controls, load average and free disk. + +**ThreadSanitizer**, because this row adds concurrency: + +```sh +cmake -S . -B build-san -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo \ + -DVLLM_CPP_CUDA=OFF -DVLLM_CPP_SANITIZE=thread +``` + +with the LTX suites run under it. + +**The wall-clock A/B.** Same binary, one decode driven through +`Ltx2VideoDecodeStreaming` at a fixed synthetic decoder configuration, at +`VLLM_CPP_CPU_THREADS` 1, 2, 4, 8, 16 and 20, repeated enough times to show the +spread rather than one number, with the host load average recorded beside it. +The configuration and the harness source are recorded in `## Outcome` so the +measurement is reproducible; the shape is synthetic and is stated as such, +because the shipped checkpoint's `decoder_blocks` list comes out of a checkpoint +header this box does not have. + +**What this row may not claim:** an end-to-end render speedup, any ratio against +any oracle, or a composition figure with #1008. There is no GPU here, no +large-render host, and no installed `ltx_core`. + +## 7. Owed + +| Item | Why it is not done here | +|---|---| +| The elementwise passes — `PixelNorm`, `Silu`, `ApplyAdaLn`, `expand`, `drop_first_frame` | Memory-bound and not where the FLOPs are. Each is order-independent and could be partitioned the same way; measuring whether it pays needs the A/B this row establishes first. | +| `AttnBlock3d` | Unreachable in the shipped decoder (`attn_res_x` is refused by name). Parallelising a path nothing can construct is dead code. | +| A SIMD inner tap loop | Separate summation-order question; see §0. | +| The composition of this row with #1008 | `ltx25-decode-speed.md` §6 warns that a threaded arm may become memory-bound where the scalar arm was ALU-bound. This row measures its own axis only. | +| An end-to-end render number | `dgx.casa` unreachable; no GPU here. | + +No `.agents/issue-index.md` row is appended for #1009. That row already exists at +`.agents/issue-index.md:275` on PR +[#1018](https://github.com/mudler/vllm.cpp/pull/1018), which filed the issue and +is unmerged. `.gitattributes` sets `merge=union` on that file and +`scripts/check-agent-record.py` refuses a duplicate issue number, so appending a +second copy here would turn `main` red for every branch the moment #1018 merges — +which is exactly what a duplicate #995 row did on 2026-08-16. The sibling dtype +row made the same call for #1008 and recorded it in its pull request body. + +## 8. Stop conditions + +* Report `NEEDS_DECISION` rather than widening `kLtx2GoldenTol`, or any other + tolerance, if a partition moves a golden. The answer to a moved golden is a + partition that does not move it. +* Report `NEEDS_DECISION` rather than shipping a decode whose output depends on + the worker count. A decode that gives different pixels at 1 thread and at 16 is + a defect even with every golden green. +* Report the ThreadSanitizer result as it comes back. `test_ltx2_video` already + carries a pre-existing LeakSanitizer leak under the `address,undefined` lane + ([#1037](https://github.com/mudler/vllm.cpp/issues/1037), in the Gemma-4 rope + cache via `DevicePool`); that one is not this row's and must not be allowed to + mask a new report. +* Claim no number that was not measured on this box, in this session, with the + load recorded. From d653f731901ee1795f537983ef71952653146d04 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Sun, 16 Aug 2026 16:14:49 +0000 Subject: [PATCH 2/4] perf(LTX25-DECODE-THREADS): route the video VAE convolutions through ParallelForRows (#1009) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lever 3 of the `LTX25-DECODE-SPEED` investigation (#1006, PR #1018). `ParallelForRows` (`src/vt/cpu/cpu_threadpool.cpp:413`) is synchronous and 10+ CPU kernels in this tree dispatch through it. Zero of them were in the LTX-2.5 conv video VAE decode, whose 42 convolutions carry ~7.25 TFLOP at 448x256/25f and ran on one core of twenty. Three sites now dispatch: `CausalConv3d`'s output nest, its padding gather, and `Linear3d`. The axis is the whole risk, so it is argued at the site. The partition is the output line `(oc, ti, hi)`, `out.w` contiguous elements: `Volume::At` makes row `r` exactly `[r*out.w, (r+1)*out.w)`, and the entire `ci * kernel^3` reduction stays inside one output element's body in the blocked order #1008 shipped. A worker therefore runs the serial arm's instruction sequence, in the serial arm's order, for every element it owns, and the result cannot depend on the worker count or on which worker stole which chunk — which matters, because `ParallelForRows` steals through an atomic cursor. Splitting the reduction axis `ic` into per-thread partials is the alternative and is rejected in the comment: it would make the summation order a function of the thread count. Two cases, because a thread-count A/B is green on a serial implementation too and would have measured nothing. "the decode DISPATCHES its convolutions to the CPU threadpool" reads the pool's public work-stealing cursor, which is 0 on a fresh pool and non-zero after a partitioned dispatch; it fails `CHECK( 0 > 0 )` before this change. "the decode is BIT-IDENTICAL across thread counts" decodes the same latent at 1, 2, 3, 5 and 8 workers and memcmps every arm against the 1-worker one, which short-circuits to the pre-change serial path. Both enter through `Ltx2VideoDecodeStreaming`, and both assert an analytically derived value of exactly 7 rather than a recorded one, so a stubbed decode's zeros cannot pass. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code] --- docs/BENCHMARKS.md | 2 +- docs/FEATURES.md | 1 + docs/USAGE.md | 14 +- .../model_executor/models/ltx2_video_vae.cpp | 174 +++++++++------ tests/CMakeLists.txt | 4 + tests/vllm/models/test_ltx2_vae.cpp | 210 ++++++++++++++++++ 6 files changed, 339 insertions(+), 66 deletions(-) diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md index 85839309f..2a18777dc 100644 --- a/docs/BENCHMARKS.md +++ b/docs/BENCHMARKS.md @@ -462,7 +462,7 @@ built on it rather than keeping the flattering one. | Kimi-Linear-48B-A3B (KDA+MLA+MoE) | **RUNNER FOLD LANDS (ROW 7, §21, #122): engine==CLI 128/128 byte-identical; vs golden 122/128 (near-tie profile); FA2 MLA default-ON; SACRED green.** Server 19.0 tok/s wall; CLI 18.93 reproduced | vLLM ~21 (#111 floor; in-session re-measure ABORTED by GB10 reboot at util 0.82, §21): **~0.90×**, >= vLLM NOT met; residual = KDA host islands + grouped MoE + decode graph | | vLLM 0.26 re-benchmark | Pending | Re-run the binding grids on the advanced pin | | MiniMax-H3 FP4 speed (W-FP4a) | **Measured GB10 (`row/H3-FP4-GPU-E2E`).** Marlin W4A16 byte-exact vs bf16; fp4 a memory win, 0.8x bf16/forward. Real-ckpt fp4-resident e2e RUNS (mp4/wav) | fp4 speed CLOSED. bf16-vs-quant A/B: ENCODER half MEASURED (§8.15), DiT half NOT (no bf16 render exists). Detail: benchmark-record + spec §8 | -| LTX-2.5 axes | Speed `PENDING` (vllm-omni#6066 has no native 2.5), binding oracle too. **SIZE: 320x192/25f completes on GB10, 448x256 does not**; that render was REGISTER-conditioned, not prompted | Wall is the HOST VAE decode, not the pool: drain returns 0.11 GiB, byte-inert. 2 baselines UNRESOLVED (lock). A real-checkpoint PROMPTED render is OWED | +| LTX-2.5 axes | Speed `PENDING` (vllm-omni#6066 has no native 2.5), binding oracle too. **SIZE: 320x192/25f completes on GB10, 448x256 does not**; that render was REGISTER-conditioned, not prompted | Wall is the HOST VAE decode, not the pool: drain returns 0.11 GiB, byte-inert. 2 baselines UNRESOLVED (lock). A real-checkpoint PROMPTED render is OWED. Decode now THREADED: 9.14x at 20 CPU workers, bit-identical (#1009) | | MiniMax-Music3 (`MiniMaxMusic3ForConditionalGeneration`) | **Every axis `PENDING`, now OWED.** The pipeline runs end to end, so a forward pass exists to time; every gate was taken on CPU with `dgx.casa` down, and a CPU number against a graphed denominator is dishonest | Denominator: SGLang-Omni `748a0b43` in its production configuration (both CUDA graphs, compiled DIT and DAV, batched seeded sampling) | | MiniMax-H3 render coherence (`row/H3-RENDER-CLOSE` #77) | **CLOSED: a COHERENT scene on GB10.** #70/#74 white was wrong-PARTITION usage (t2va on the ref2va ckpt); t2va on the FL2VA GGUF renders a prompt-matched orange cat (adj-cos 0.95 vs 0.06, no patch-grid) | Verified first: t2va inputs byte-exact vs upstream; CUDA device==host at seq 1920. Follow-up `H3-TASK-PARTITION-GUARD`: the task/partition mismatch now RAISES 1:1 with `_resolve_task` (spec §8.6-8.7) | | MiniMax-H3 image conditioning (`row/H3-CONDITIONED-E2E`, `row/H3-VISION-SCATTER`, `row/H3-REF2VA-ASSEMBLY`) | **fl2va COHERENT; ref2va assembly bug FIXED+gated.** vision→cond scatter gated; ref2va block-dim double-division fixed + RED-first gated (128 vs 512) + a permanent ref2va DiT-forward rung (§8.10) | grid RE-ATTRIBUTED: with the fix ref2va grids in fp4 AND bf16, and t2va with no refs on the ref2va NVFP4 also grids while FL2VA-GGUF renders, so it is the **NVFP4 checkpoint/loader**, NOT assembly/fp4 (§8.10) | diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 1163a0764..0e5e546cc 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -169,6 +169,7 @@ in `ltx2_text_encoder.cpp` is the call that would have to change. | LTX-2.5 DFR base + generated keyframe slots | LTX-2.5 (21.00B video+audio) | gated vs EXECUTED upstream `dfr_layout` + 3 `dfr_pipeline` helpers @ `fd4ded7f` (`test_ltx2_dfr` 11/11, 652 assertions); canvas, tiles, stitch, carry-forward as EXACT index vectors, since each defect is plausible| `--pipeline-kind dfr`. Canvas PADS 9 to 25 then trims back; slots on the x8 grid, MARKED, read back BEFORE the trim. `num_generated_keyframes` SERVED elsewhere. Temporal ROUNDS refused (#986); detail LoRA refused (#975)| | LTX-2.5 tiled + streaming Conv VAE decode | LTX-2.5 video VAE | gated vs executed upstream `ltx_core` @ `fd4ded7f` (`test_ltx2_tiling` 10/10, 915 assertions); one-tile and untiled-spatial controls BIT-EXACT vs untiled on both causality arms; an untiled frames axis is REFUSED | Streams temporal chunks through upstream's AUTO layout (768/64 px, 80/24 frames); above one tile the pixel volume is never materialized. NO-OP below 768px and 81 frames; 81-120 IS tiled, differing 6.70% of range | | LTX-2.5 Conv VAE decode arithmetic width | LTX-2.5 video VAE | `test_ltx2_vae` "the decode's convolution accumulates in f32", entering through `Ltx2VideoDecodeStreaming`; widening the accumulator to `double`, or deleting the production call site, each turns it RED | **f32**, the width `F.conv3d` uses at f32 AND bf16 (MEASURED). Was f64 at 8 sites ([#1008](https://github.com/mudler/vllm.cpp/issues/1008)). Conv sums BLOCKED per input channel, as torch's. STORAGE stays f32; bf16 owed | +| LTX-2.5 Conv VAE decode threading | LTX-2.5 video VAE | `test_ltx2_vae` "the decode DISPATCHES its convolutions to the CPU threadpool" and "...BIT-IDENTICAL across thread counts", through `Ltx2VideoDecodeStreaming`; 34 golden margins UNCHANGED; TSan clean | **Parallel** over output lines via `vt::cpu::ParallelForRows` ([#1009](https://github.com/mudler/vllm.cpp/issues/1009)). 9.14x at 20 workers, 9.67x at c=128. Bit-identical at any count; elementwise still serial | | LTX-2.5 retake (`RetakePipeline`, regenerate a time window) | LTX-2.5 DiT + video VAE encoder | `test_ltx2_retake` 4/4 (69 assertions) and 4 `test_ltx2_video` cases entering through `Generate`; mask, conform and the four-way plan pinned to upstream `fd4ded7f` | `--pipeline-kind retake` on `ltx2-gen`. Source is a `frame_%06d.ppm` DIRECTORY; a container is REFUSED (no demuxer). Geometry comes from the clip. A folder has no audio, so the soundtrack is generated | | MTP speculator | Qwen3.6-27B, Qwen3.6-35B-A3B | token-identical to vLLM `mtp` at c1 | ~4% faster c1; +16% output tput (MoE) | | DFlash block-diffusion | Qwen3 (DFlash draft) | near-tie e2e 27/27 vs vLLM | 2.9x over spec-off, 1.003x vs vLLM DFlash-on | diff --git a/docs/USAGE.md b/docs/USAGE.md index 85a2273cf..2287bc811 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -870,8 +870,18 @@ and has to be stopped. The denoise itself is flat at either size. Unified memory makes those host bytes and this class of box reboots rather than OOM-killing, so start small and grow, and put a memory watchdog in front of anything larger. The recipe default (1024x1536 at 121 frames) is far beyond what one GB10 holds today. -Expect minutes, not seconds: most of a 320x192/25f render is spent single-threaded -in the host VAE decode at 0% GPU. +Expect minutes, not seconds: most of a 320x192/25f render is spent in the host +VAE decode at 0% GPU, because that decode has no device arm +([#1007](https://github.com/mudler/vllm.cpp/issues/1007)). + +It is no longer *single-threaded*, which is what this paragraph used to say. The +decode's convolutions now dispatch across `VLLM_CPP_CPU_THREADS` workers +(default `hardware_concurrency`), bit-identical at every worker count — +[#1009](https://github.com/mudler/vllm.cpp/issues/1009), measured at **9.14x on +20 workers** against one. Read that as a decode figure and not a render one: the +wall above was recorded on GB10 before the change and has not been re-measured, +and the 9.14x was taken on a synthetic decode shape on a 20-core x86 host. Set +`VLLM_CPP_CPU_THREADS` lower if the render has to share the box. *The render behind those numbers was NOT prompted, and it renders a scene without rendering YOUR scene.* It was the EMBEDS path — `--prompt-embeds` with diff --git a/src/vllm/model_executor/models/ltx2_video_vae.cpp b/src/vllm/model_executor/models/ltx2_video_vae.cpp index dafc39b9c..31b8e6aec 100644 --- a/src/vllm/model_executor/models/ltx2_video_vae.cpp +++ b/src/vllm/model_executor/models/ltx2_video_vae.cpp @@ -60,6 +60,18 @@ // TimestepEmbedding frequency table, which is a constant precompute rather than // a data path. // +// ─── AND IT IS PARALLEL, WHICH IT ALSO USED NOT TO BE (#1009) ──────────────── +// The convolutions dispatch through `vt::cpu::ParallelForRows`, the synchronous +// row-chunked parallel-for 10+ CPU kernels in this tree already use and that no +// line of this file used before. The partition is over OUTPUT lines only: the +// whole `ci * kernel^3` reduction stays inside one output element's body, so the +// blocked f32 order above is untouched and the result is bit-identical at any +// thread count and under any work-stealing assignment. Splitting the reduction +// axis `ic` instead would make the summation order a function of the thread +// count; it is rejected at the site. "the decode DISPATCHES its convolutions to +// the CPU threadpool" and "the decode is BIT-IDENTICAL across thread counts" in +// tests/vllm/models/test_ltx2_vae.cpp are the two instruments. +// // PHASE L6 OWES THE PRODUCTION ARM — the bf16/NVFP4 decode that inherits the // checkpoint dtype the way upstream does. Until it lands, this file is a // correctness reference, not the shipping path, and no memory or throughput @@ -75,6 +87,7 @@ #include #include "vllm/model_executor/models/minimax_h3.h" +#include "vt/cpu/cpu_threadpool.h" #include "vt/dtype.h" namespace vllm { @@ -149,23 +162,30 @@ Volume CausalConv3d(const Volume& in, int64_t out_channels, int64_t kernel, bool const int64_t pw = in.w + 2 * pad_spatial; std::vector padded(static_cast(ci * pt * ph * pw), 0.0f); - for (int64_t c = 0; c < ci; ++c) { - for (int64_t ti = 0; ti < pt; ++ti) { - // Temporal padding REPLICATES the edge frame, never zeros. - const int64_t st = std::max(0, std::min(in.t - 1, ti - pad_front)); - for (int64_t hi = 0; hi < ph; ++hi) { - bool zero_h = false; - const int64_t sh = SpatialIndex(hi - pad_spatial, in.h, mode, &zero_h); - for (int64_t wi = 0; wi < pw; ++wi) { - bool zero_w = false; - const int64_t sw = SpatialIndex(wi - pad_spatial, in.w, mode, &zero_w); - if (zero_h || zero_w) continue; - padded[static_cast(((c * pt + ti) * ph + hi) * pw + wi)] = - in.data[in.At(c, st, sh, sw)]; + // One "row" is one padded line (c, ti, hi) of `pw` elements. A pure GATHER — + // one source element per destination element, no reduction at all — so the + // partition cannot change any arithmetic. It is parallel because it is + // O(ci * pt * ph * pw) inside the hot function, and a serial section here + // would bound the convolution's speedup by Amdahl's law (#1009). + vt::cpu::ParallelForRows( + vt::cpu::CurrentThreadpool(), ci * pt * ph, [&](int64_t r0, int64_t r1) { + for (int64_t r = r0; r < r1; ++r) { + const int64_t hi = r % ph; + const int64_t ti = (r / ph) % pt; + const int64_t c = r / (ph * pt); + // Temporal padding REPLICATES the edge frame, never zeros. + const int64_t st = std::max(0, std::min(in.t - 1, ti - pad_front)); + bool zero_h = false; + const int64_t sh = SpatialIndex(hi - pad_spatial, in.h, mode, &zero_h); + for (int64_t wi = 0; wi < pw; ++wi) { + bool zero_w = false; + const int64_t sw = SpatialIndex(wi - pad_spatial, in.w, mode, &zero_w); + if (zero_h || zero_w) continue; + padded[static_cast(((c * pt + ti) * ph + hi) * pw + wi)] = + in.data[in.At(c, st, sh, sw)]; + } } - } - } - } + }); Volume out; out.channels = out_channels; @@ -175,46 +195,67 @@ Volume CausalConv3d(const Volume& in, int64_t out_channels, int64_t kernel, bool VT_CHECK(pt >= kernel && ph >= kernel && pw >= kernel && out.t > 0 && out.h > 0 && out.w > 0, "ltx2 conv3d: empty output"); out.data.resize(static_cast(out_channels * out.spatial())); - for (int64_t oc = 0; oc < out_channels; ++oc) { - for (int64_t ti = 0; ti < out.t; ++ti) { - for (int64_t hi = 0; hi < out.h; ++hi) { - for (int64_t wi = 0; wi < out.w; ++wi) { - // f32, because that is the width `nn.Conv3d` accumulates in — MEASURED, - // not assumed: F.conv3d returns 0.0 on the separable reduction in - // tests/vllm/models/test_ltx2_vae.cpp for f32 AND for bf16 tensors, - // where an f64 accumulator returns 2.5 (#1008). - float acc = bias != nullptr ? (*bias)[static_cast(oc)] : 0.0f; - for (int64_t ic = 0; ic < ci; ++ic) { - // BLOCKED, one partial sum per input channel, and this is the ORDER - // as well as the width. A single naive serial f32 sum over all - // `ci * kernel^3` taps accumulates error with sqrt of the whole - // length; splitting it into `ci` blocks of `kernel^3` accumulates - // with sqrt of each. That is not a local optimisation — torch's f32 - // convolution is a blocked GEMM and sums exactly this way, which is - // why `torch.sum` on the separable reduction returns 2.0999999 where - // a naive serial f32 sum returns 0.0. Narrowing the width alone, - // with the naive order kept, pushed the non-causal tiled golden to - // 5.00679e-06 against a 5e-06 tolerance — MEASURED, and the reason - // this loop is shaped this way. - float tap = 0.0f; - for (int64_t a = 0; a < kernel; ++a) { - for (int64_t b = 0; b < kernel; ++b) { - for (int64_t d = 0; d < kernel; ++d) { - tap += padded[static_cast( - ((ic * pt + ti * stride_t + a) * ph + hi * stride_h + b) * pw + - wi * stride_w + d)] * - weight[static_cast( - (((oc * ci + ic) * kernel + a) * kernel + b) * kernel + d)]; - } + // THE PARALLEL AXIS IS THE OUTPUT LINE (oc, ti, hi), AND THAT CHOICE IS THE + // WHOLE OF THIS ROW'S RISK (#1009, .agents/specs/ltx25-decode-threads.md). + // + // `Volume::At(oc, ti, hi, wi)` is `((oc*t + ti)*h + hi)*w + wi`, so row `r` is + // exactly the contiguous span [r*out.w, (r+1)*out.w) of `out.data`: no output + // element is written by more than one worker. And the entire `ci * kernel^3` + // reduction below stays inside one `wi` iteration of one row, so a worker + // executes precisely the serial arm's instruction sequence, in the serial + // arm's order, for every element it owns. The result is therefore bit-identical + // at any thread count AND under any row-to-thread assignment — which matters, + // because ParallelForRows STEALS work through an atomic cursor and the + // assignment is genuinely non-deterministic run to run. + // + // Splitting the REDUCTION axis `ic` into per-thread partials would also be a + // legal convolution, and it is deliberately not taken: it would make the + // summation order a function of the thread count, which is the defect #1008 + // spent its whole budget removing. `cpu_conv2d.cpp:75-78` partitions 2-D + // convolution the same way, and `cpu_threadpool.h:39-43` states the contract + // for the whole CPU backend. + const int64_t rows = out_channels * out.t * out.h; + vt::cpu::ParallelForRows(vt::cpu::CurrentThreadpool(), rows, [&](int64_t r0, int64_t r1) { + for (int64_t r = r0; r < r1; ++r) { + const int64_t hi = r % out.h; + const int64_t ti = (r / out.h) % out.t; + const int64_t oc = r / (out.h * out.t); + for (int64_t wi = 0; wi < out.w; ++wi) { + // f32, because that is the width `nn.Conv3d` accumulates in — MEASURED, + // not assumed: F.conv3d returns 0.0 on the separable reduction in + // tests/vllm/models/test_ltx2_vae.cpp for f32 AND for bf16 tensors, + // where an f64 accumulator returns 2.5 (#1008). + float acc = bias != nullptr ? (*bias)[static_cast(oc)] : 0.0f; + for (int64_t ic = 0; ic < ci; ++ic) { + // BLOCKED, one partial sum per input channel, and this is the ORDER + // as well as the width. A single naive serial f32 sum over all + // `ci * kernel^3` taps accumulates error with sqrt of the whole + // length; splitting it into `ci` blocks of `kernel^3` accumulates + // with sqrt of each. That is not a local optimisation — torch's f32 + // convolution is a blocked GEMM and sums exactly this way, which is + // why `torch.sum` on the separable reduction returns 2.0999999 where + // a naive serial f32 sum returns 0.0. Narrowing the width alone, + // with the naive order kept, pushed the non-causal tiled golden to + // 5.00679e-06 against a 5e-06 tolerance — MEASURED, and the reason + // this loop is shaped this way. The partition above does not touch it. + float tap = 0.0f; + for (int64_t a = 0; a < kernel; ++a) { + for (int64_t b = 0; b < kernel; ++b) { + for (int64_t d = 0; d < kernel; ++d) { + tap += padded[static_cast( + ((ic * pt + ti * stride_t + a) * ph + hi * stride_h + b) * pw + + wi * stride_w + d)] * + weight[static_cast( + (((oc * ci + ic) * kernel + a) * kernel + b) * kernel + d)]; } } - acc += tap; } - out.data[out.At(oc, ti, hi, wi)] = acc; + acc += tap; } + out.data[out.At(oc, ti, hi, wi)] = acc; } } - } + }); return out; } @@ -228,19 +269,26 @@ Volume Linear3d(const Volume& in, int64_t out_channels, const std::vector out.w = in.w; out.data.resize(static_cast(out_channels * in.spatial())); const int64_t n = in.spatial(); - for (int64_t oc = 0; oc < out_channels; ++oc) { - for (int64_t i = 0; i < n; ++i) { - // f32: this is an `nn.Conv3d` upstream too (make_linear_nd's dims==3 - // branch, convolution.py:84-85), so it accumulates at the same width as - // every other conv on the path. - float acc = bias[static_cast(oc)]; - for (int64_t ic = 0; ic < in.channels; ++ic) { - acc += in.data[static_cast(ic * n + i)] * - weight[static_cast(oc * in.channels + ic)]; - } - out.data[static_cast(oc * n + i)] = acc; - } - } + // One "row" is one output ELEMENT (oc, i), and the `in.channels` reduction + // stays inside it — the same partition-the-outputs discipline CausalConv3d + // above uses, for the same reason (#1009). `out.data[oc * n + i]` means a + // chunk of consecutive `r` is a contiguous span of the output. + vt::cpu::ParallelForRows( + vt::cpu::CurrentThreadpool(), out_channels * n, [&](int64_t r0, int64_t r1) { + for (int64_t r = r0; r < r1; ++r) { + const int64_t oc = r / n; + const int64_t i = r - oc * n; + // f32: this is an `nn.Conv3d` upstream too (make_linear_nd's dims==3 + // branch, convolution.py:84-85), so it accumulates at the same width as + // every other conv on the path. + float acc = bias[static_cast(oc)]; + for (int64_t ic = 0; ic < in.channels; ++ic) { + acc += in.data[static_cast(ic * n + i)] * + weight[static_cast(oc * in.channels + ic)]; + } + out.data[static_cast(oc * n + i)] = acc; + } + }); return out; } diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index d8ef596cd..5c2230d99 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -311,6 +311,10 @@ vllm_cpp_add_test(test_ltx2_device vllm/models/test_ltx2_device.cpp) # dimensions. ltx2_vae_goldens.inc lives next to the test source. vllm_cpp_add_test(test_ltx2_vae vllm/models/test_ltx2_vae.cpp) target_include_directories(test_ltx2_vae PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/vllm/models) +# LTX25-DECODE-THREADS (issue #1009): the two threading cases reach +# vt::cpu::Threadpool::SwapForTesting and the pool's work-stealing cursor, which +# live under src/ like every other CPU-threading A/B in tests/vt/. +target_include_directories(test_ltx2_vae PRIVATE ${CMAKE_SOURCE_DIR}/src) # LTX25-TILED-DECODE (issue #644): the tiling algebra, the AUTO layout and the # STREAMING tiled decode, gated against upstream ltx_core executed at reduced # dimensions. Its own target so the decode arms — which run the reduced decoder 18 diff --git a/tests/vllm/models/test_ltx2_vae.cpp b/tests/vllm/models/test_ltx2_vae.cpp index b3f688d41..f8dd8b9f1 100644 --- a/tests/vllm/models/test_ltx2_vae.cpp +++ b/tests/vllm/models/test_ltx2_vae.cpp @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -23,6 +24,10 @@ #include "doctest/doctest.h" #include "support/max_abs_diff.h" +// LTX25-DECODE-THREADS (issue #1009): Threadpool::SwapForTesting and the pool's +// work-stealing cursor, reached via -I src the way every other threading A/B in +// this tree does (tests/vt/test_ops_conv2d.cpp:25). +#include "vt/cpu/cpu_threadpool.h" #include "vllm/model_executor/models/ltx2_audio_vae.h" #include "vllm/model_executor/models/ltx2_audio_vae_encoder.h" #include "vllm/model_executor/models/ltx2_conditioning.h" @@ -1184,6 +1189,211 @@ TEST_CASE("ltx2 vae: the decode's convolution accumulates in f32, the width torc CHECK(err < 1.0); } +// --------------------------------------------------------------------------- +// LTX25-DECODE-THREADS (issue #1009): the decode had 20 cores and used one. +// .agents/specs/ltx25-decode-threads.md +// --------------------------------------------------------------------------- + +// The fixture both threading cases decode. Deliberately shared, so the case that +// proves the dispatch happens and the case that proves the result does not +// depend on it are looking at the SAME work. +// +// It is the accumulator-width fixture's derivation at a size where the +// convolution has many output lines: `decoder_blocks` empty, patch_size 1, +// replicate padding and an all-ones latent, so every conv tap reads exactly 1.0 +// at every output voxel including the borders and one reduction is repeated +// everywhere. conv_in channel 0 carries the f32/f64 separable reduction, channel +// 1 is all-zero weights with bias 1, conv_out selects channel 0's centre tap and +// adds a bias of 7. +struct Ltx2ThreadFixture { + vllm::Ltx2ConvVideoDecoderConfig cfg; + vllm::Ltx2VaeWeights weights; + std::vector latent; + int64_t lt = 0, lh = 0, lw = 0; +}; + +Ltx2ThreadFixture MakeLtx2ThreadFixture() { + Ltx2ThreadFixture f; + f.cfg.prefix = "ltx2.videodec.threads."; + f.cfg.in_channels = 1; + f.cfg.out_channels = 1; + f.cfg.patch_size = 1; + // 24 output channels, so conv_in's parallel row count is 24 * out.t * out.h and + // no thread count under test degenerates to one chunk. + f.cfg.base_channels = 24; + f.cfg.causal = false; + f.cfg.timestep_conditioning = false; + f.cfg.norm_layer = vllm::Ltx2NormLayer::kPixelNorm; + f.cfg.spatial_padding_mode = vllm::Ltx2PaddingMode::kReplicate; + f.cfg.decoder_blocks = {}; + + const std::string p = f.cfg.prefix; + f.weights.tensors[p + "per_channel_statistics.std-of-means"] = {1.0f}; + f.weights.tensors[p + "per_channel_statistics.mean-of-means"] = {0.0f}; + + // conv_in is [out=24, in=1, 3, 3, 3]. Channel 0 carries the separable + // reduction in the exact order CausalConv3d walks it; channels 1..23 are + // all-zero weights with bias 1, which gives PixelNorm 23 unit channels to + // divide by and keeps channel 0 at zero on an f32 accumulator. + constexpr float kBig = 1e8f; + constexpr float kSmall = 0.1f; + std::vector conv_in(static_cast(24 * 1 * 27), 0.0f); + conv_in[0] = kBig; + for (size_t i = 1; i < 26; ++i) conv_in[i] = kSmall; + conv_in[26] = -kBig; + f.weights.tensors[p + "conv_in.conv.weight"] = conv_in; + std::vector conv_in_bias(24, 1.0f); + conv_in_bias[0] = 0.0f; + f.weights.tensors[p + "conv_in.conv.bias"] = conv_in_bias; + + // conv_out is [out=1, in=24, 3, 3, 3]; it selects channel 0's centre tap and + // nothing else. THE BIAS IS 7 for the reason #1008 recorded: with a bias of 0 + // the expected value is zero, and a decode that never ran hands back a + // zero-filled buffer, so the case would pass while measuring nothing. + std::vector conv_out(static_cast(1 * 24 * 27), 0.0f); + conv_out[13] = 1.0f; // ic = 0, a = b = d = 1 + f.weights.tensors[p + "conv_out.conv.weight"] = conv_out; + f.weights.tensors[p + "conv_out.conv.bias"] = {7.0f}; + + f.lt = 3; + f.lh = 5; + f.lw = 4; + f.latent.assign(static_cast(f.lt * f.lh * f.lw), 1.0f); + return f; +} + +// Untiled, so the ONE decode call the streaming entry point makes is the whole +// measurement. +vllm::Ltx2TileSizeConfig Ltx2ThreadUntiled() { + vllm::Ltx2TileSizeConfig tiling; + tiling.frames = vllm::Ltx2DimensionSizeConfig{10000, 0}; + tiling.height = vllm::Ltx2DimensionSizeConfig{10000, 0}; + tiling.width = vllm::Ltx2DimensionSizeConfig{10000, 0}; + return tiling; +} + +vllm::Ltx2VideoFrames Ltx2ThreadDecode(const Ltx2ThreadFixture& f, int64_t* chunks) { + // ENTERS THROUGH THE PRODUCTION ENTRY POINT. `Ltx2VideoDecodeStreaming` is what + // the render path calls (src/vllm/multimodal/ltx2_video.cpp:3258) and it reaches + // Ltx2ConvVideoDecode at ltx2_video_vae_tiled.cpp:113. A case that called + // Ltx2ConvVideoDecode directly would prove the function works, never that the + // shipping path reaches the threaded one. + vllm::Ltx2VideoFrames got; + *chunks = 0; + vllm::Ltx2VideoDecodeStreaming( + vllm::Ltx2VideoDecoderKind::kConv, f.cfg, f.weights, f.latent, f.cfg.in_channels, f.lt, f.lh, + f.lw, /*noise=*/nullptr, Ltx2ThreadUntiled(), [&](const vllm::Ltx2VideoChunk& chunk) { + ++*chunks; + got = chunk.frames; + }); + return got; +} + +TEST_CASE("ltx2 vae: the decode DISPATCHES its convolutions to the CPU threadpool") { + // THE CASE THAT IS RED BEFORE #1009, and the reason the determinism case below + // is not enough on its own: two runs of a SERIAL decode are also bit-identical, + // so a thread-count A/B is green on an implementation that never threads + // anything. This case observes the dispatch itself. + // + // THE INSTRUMENT. `ParallelForRows` (src/vt/cpu/cpu_threadpool.cpp:413-458) + // partitions its rows through the pool's shared work-stealing cursor: worker 0 + // seeds it with `ChunkSet(nth)` and every steal advances it with `ChunkAdd(1)`. + // `ChunkAdd(0)` is a public non-mutating read of that cursor — `fetch_add(0)` + // returns the current value. A fresh pool reads 0; a pool that has run at least + // one multi-chunk partitioned dispatch reads at least `nth`. So the read is a + // direct observation of "partitioned work ran on THIS pool", and the assertion + // before the decode is its own positive control: the instrument demonstrably + // reads zero when nothing has dispatched, which is exactly the state this row + // removes. + const Ltx2ThreadFixture f = MakeLtx2ThreadFixture(); + + vt::cpu::Threadpool tp(4); + REQUIRE(tp.NThreads() == 4); + // Positive control: the cursor reads zero on a pool nothing has dispatched to. + REQUIRE(tp.ChunkAdd(0) == 0); + + vt::cpu::Threadpool* prev = vt::cpu::Threadpool::SwapForTesting(&tp); + int64_t chunks = 0; + vllm::Ltx2VideoFrames got; + try { + got = Ltx2ThreadDecode(f, &chunks); + } catch (...) { + vt::cpu::Threadpool::SwapForTesting(prev); + throw; + } + const int cursor = tp.ChunkAdd(0); + vt::cpu::Threadpool::SwapForTesting(prev); + + REQUIRE(chunks == 1); + REQUIRE(got.data.size() == static_cast(f.lt * f.lh * f.lw)); + + // The decode ran, and it ran through the pool. + CHECK(cursor > 0); + + // AND IT PRODUCED THE RIGHT PIXELS. The derivation, so a reader can check the + // number rather than trust it: conv_in channel 0 accumulates to 0 in f32, + // channels 1..23 to their bias of 1; PixelNorm leaves channel 0 at 0; SiLU(0) + // is 0; conv_out forwards channel 0 and adds its bias. Every output element is + // therefore exactly 7. A stubbed or deleted decode returns zeros and fails + // here; an f64 accumulator keeps channel 0's 2.5 and fails here too. + const std::vector want(got.data.size(), 7.0f); + const double err = MaxAbsDiff(got.data, want.data(), got.data.size()); + INFO("threaded decode max|out - 7| = " << err); + CHECK(err <= kLtx2GoldenTol); +} + +TEST_CASE("ltx2 vae: the decode is BIT-IDENTICAL across thread counts") { + // The determinism half of #1009. `ParallelForRows` steals work through an + // atomic cursor, so which worker takes which output line is genuinely + // non-deterministic run to run; the partition is over OUTPUT lines only and the + // whole `ci * kernel^3` reduction stays inside one output element's body, so + // every element is produced by the same instruction sequence on the same values + // whatever the worker count is. That is the contract cpu_threadpool.h:39-43 + // states for the whole CPU backend, and this case holds the decode to it. + // + // A decode that returns different pixels at 1 thread and at 8 is a defect even + // with every golden green, and no golden here would see it: the goldens run at + // one thread count, the global pool's. + // + // Worker count 1 short-circuits ParallelForRows to `body(0, nr)` on the caller + // (cpu_threadpool.cpp:423-426), so the 1-thread arm IS the pre-#1009 serial code + // path byte for byte, and every other arm is compared against it. The counts are + // not all powers of two on purpose: 3 and 5 do not divide the row counts, so the + // chunk boundaries land in different places on every arm. + const Ltx2ThreadFixture f = MakeLtx2ThreadFixture(); + + std::vector base; + for (int nth : {1, 2, 3, 5, 8}) { + vt::cpu::Threadpool tp(nth); + vt::cpu::Threadpool* prev = vt::cpu::Threadpool::SwapForTesting(&tp); + int64_t chunks = 0; + vllm::Ltx2VideoFrames got; + try { + got = Ltx2ThreadDecode(f, &chunks); + } catch (...) { + vt::cpu::Threadpool::SwapForTesting(prev); + throw; + } + vt::cpu::Threadpool::SwapForTesting(prev); + + REQUIRE(chunks == 1); + REQUIRE(got.data.size() == static_cast(f.lt * f.lh * f.lw)); + if (base.empty()) { + // NOT DEGENERATE, so a stubbed decode cannot satisfy the comparison below. + // An all-zero buffer is bit-identical to another all-zero buffer, so + // "every arm agrees" is a vacuous statement about a decode that never ran. + // This fixture's answer is 7 everywhere, which no absent computation + // produces. + const std::vector want(got.data.size(), 7.0f); + REQUIRE(MaxAbsDiff(got.data, want.data(), got.data.size()) <= kLtx2GoldenTol); + base = got.data; + } else { + INFO("worker count " << nth); + CHECK(std::memcmp(base.data(), got.data.data(), base.size() * sizeof(float)) == 0); + } + } +} + TEST_CASE("ltx2 vae: the video decoder's norm_eps is gated where it BINDS") { // THE ARM THAT MAKES `Ltx2ConvVideoDecoderConfig::norm_eps` NUMERICALLY // REACHABLE, and the correction of a record that said it was not. From 249418305086f48816930bc44b065b5d75c46beb Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Sun, 16 Aug 2026 16:40:35 +0000 Subject: [PATCH 3/4] =?UTF-8?q?record(LTX25-DECODE-THREADS):=20the=20outco?= =?UTF-8?q?me=20=E2=80=94=209.14x=20on=2020=20workers,=20and=2034=20golden?= =?UTF-8?q?=20margins=20that=20did=20not=20move?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The row's `## Outcome`, written after the measurements rather than before them. The risk this spec was built around did not bind. Both suites were rebuilt with `kLtx2GoldenTol` set to 0.0 before the change and again after, and all 34 recorded margins — 23 in `test_ltx2_vae`, 11 in `test_ltx2_tiling` — came back byte-for-byte identical, compared by diffing the two sorted lists rather than by eye. Nothing was within tolerance; nothing moved. Those before-values also match what `ltx25-decode-dtype.md` §8.1 recorded on its own host, so this box reproduces the sibling row rather than reporting a local artefact. CPU A/B, same binary, `VLLM_CPP_CPU_THREADS` the only variable, 14 runs per count across an ascending and a descending sweep: 1.93x at 2, 3.68x at 4, 6.78x at 8, 9.15x at 16, 9.14x at 20, and 9.67x at the checkpoint's real channel width of 128. Spread is under 7% at every count up to 8 and 21-23% at 16 and 20, on a box holding one non-agent process at ~1.07 cores throughout, at one-minute load 4.03 to 6.77. The flattening above 16 is NOT called a ceiling: the implied serial fraction is 6.3%, which is the right order for the elementwise passes this row deliberately left serial, and §7 owns them. The output checksum was bit-identical across all 84 A/B decodes at both shapes, which is a second determinism proof on pseudo-random weights beside the engineered fixture's memcmp. ThreadSanitizer is clean on all three LTX suites — after two instrument problems were settled first. The binaries would not start at all (`unexpected memory mapping`, exit 66, a verdict-shaped ASLR failure that `setarch -R` fixes), and a silent sanitizer is indistinguishable from an absent one, so a deliberate race was compiled into the same lane and produced 87 warnings before being reverted. Recorded honestly: reverting any ONE of the three dispatch sites is detected by nothing, because the single work-stealing cursor stays non-zero while the other two dispatch. T0 holds the conjunction; the per-site gap is written up and owed. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code] --- .agents/specs/ltx25-decode-threads.md | 260 +++++++++++++++++++++++++- 1 file changed, 255 insertions(+), 5 deletions(-) diff --git a/.agents/specs/ltx25-decode-threads.md b/.agents/specs/ltx25-decode-threads.md index 3e7581a8c..784c7f8cc 100644 --- a/.agents/specs/ltx25-decode-threads.md +++ b/.agents/specs/ltx25-decode-threads.md @@ -19,7 +19,10 @@ change a summation order. ## Now -`ACTIVE`. +`DONE`, pending review. The three convolution sites dispatch, the numerics are +byte-identical to the serial arm, and the CPU A/B is in `## Outcome`: **9.14x at +20 workers and 9.67x at the checkpoint's real channel width**, measured, on this +box, at a recorded load. No end-to-end render number, and none is claimed. ## 0. Scope @@ -51,7 +54,14 @@ at fixed thread counts on 20 local cores (§6). ## 1. Why there is no upstream to mirror here Every oracle runs this decoder on an accelerator and none of them has a -host-parallel arm to port: +host-parallel arm to port. The four anchors below are the parent investigation's +([`ltx25-decode-speed.md` on PR #1018](https://github.com/mudler/vllm.cpp/pull/1018) §2), read there at the +revisions named; none of these repositories is checked out on this box, so this +row cites them rather than re-deriving them. `ltx_core` has no +`.agents/oracles/` file at all — that pin is owed by +[#655](https://github.com/mudler/vllm.cpp/issues/655) and +[#1012](https://github.com/mudler/vllm.cpp/issues/1012), and the parent spec +lists both. * Lightricks LTX-2 @ `fd4ded7f2` builds the decoder onto a device (`packages/ltx-pipelines/src/ltx_pipelines/utils/blocks.py:1139`, @@ -66,7 +76,7 @@ host-parallel arm to port: (`ltx2_diffusion_decoder.py:208-209`, *"No CPU path"*). So this row is a **local seam**, not an upstream mirror, and -[`ltx25-decode-speed.md`](ltx25-decode-speed.md) §6 lever 3 records it as such. +[`ltx25-decode-speed.md` on PR #1018](https://github.com/mudler/vllm.cpp/pull/1018) §6 lever 3 records it as such. What it does mirror is *this tree's own* CPU convolution: `src/vt/cpu/cpu_conv2d.cpp:75-78 @ d1b0ea3a8` partitions a 2-D convolution over `n * cout * hout` output lines through the same call, with the same comment this @@ -256,7 +266,7 @@ large-render host, and no installed `ltx_core`. | The elementwise passes — `PixelNorm`, `Silu`, `ApplyAdaLn`, `expand`, `drop_first_frame` | Memory-bound and not where the FLOPs are. Each is order-independent and could be partitioned the same way; measuring whether it pays needs the A/B this row establishes first. | | `AttnBlock3d` | Unreachable in the shipped decoder (`attn_res_x` is refused by name). Parallelising a path nothing can construct is dead code. | | A SIMD inner tap loop | Separate summation-order question; see §0. | -| The composition of this row with #1008 | `ltx25-decode-speed.md` §6 warns that a threaded arm may become memory-bound where the scalar arm was ALU-bound. This row measures its own axis only. | +| The composition of this row with #1008 | the parent spec §6 warns that a threaded arm may become memory-bound where the scalar arm was ALU-bound. This row measures its own axis only. | | An end-to-end render number | `dgx.casa` unreachable; no GPU here. | No `.agents/issue-index.md` row is appended for #1009. That row already exists at @@ -268,7 +278,247 @@ second copy here would turn `main` red for every branch the moment #1018 merges which is exactly what a duplicate #995 row did on 2026-08-16. The sibling dtype row made the same call for #1008 and recorded it in its pull request body. -## 8. Stop conditions +## 8. Outcome — what was measured + +Everything below was measured on the shared 20-core development box on +2026-08-16, at branch head `dac85969c`. No GPU was involved and none was +available; `dgx.casa` was unreachable for this row's whole duration. + +### 8.1 The numerics did not move at all, and that is the point + +The one risk that could have changed the design did not bind. Both suites were +built with `kLtx2GoldenTol` temporarily set to `0.0` so every golden reports its +`max|diff|` rather than its verdict, **before** the change and **again after**, +and both tolerances were restored. + +**All 34 recorded margins — 23 in `test_ltx2_vae`, 11 in `test_ltx2_tiling` — +came back byte-for-byte identical.** The comparison was a `diff` of the two +sorted value lists, not an eyeball: `VAE_MARGINS_IDENTICAL (23 values)` and +`TILING_MARGINS_IDENTICAL (11 values)`. Nothing was within tolerance; nothing +moved. + +| golden arm | before (serial) | after (20-thread global pool) | tol | +|---|---|---|---| +| Conv video decoder | 1.72853e-06 | 1.72853e-06 | 5e-06 | +| non-causal Conv video decoder | 2.08616e-06 | 2.08616e-06 | 5e-06 | +| norm_eps-binding video decoder | 1.54972e-06 | 1.54972e-06 | 5e-06 | +| tiled decode, untiled control A | 2.74181e-06 | 2.74181e-06 | 5e-06 | +| tiled decode, untiled control B | 2.80142e-06 | 2.80142e-06 | 5e-06 | +| video encoder (`*_res`) | 4.76837e-07 | 4.76837e-07 | 5e-06 | +| video encoder (strided convs) | 8.34465e-07 | 8.34465e-07 | 5e-06 | +| cropped video encoder | 4.76837e-07 | 4.76837e-07 | 5e-06 | +| causal-arm video encoder | 4.17233e-07 | 4.17233e-07 | 5e-06 | +| every other arm in both suites | unchanged | unchanged | — | + +Those "before" values are also the ones +[`ltx25-decode-dtype.md`](ltx25-decode-dtype.md) §8.1 recorded on its own host, +which is an independent check that this box reproduces the sibling row's +measurement rather than a local artefact. **No tolerance was touched.** + +That table is a threading gate in its own right and worth naming as one: the +full suite runs on the global pool, `hardware_concurrency` wide, so every LTX-2.5 +video golden after this change executes on 20 workers. The "Conv video decoder" +fixture carries a `res_x_y` block, so `Linear3d` and its `conv_shortcut` are on +that path too. + +### 8.2 Determinism, proven twice and at two scales + +* **Case B**, `test_ltx2_vae` "the decode is BIT-IDENTICAL across thread + counts": the same latent decoded at 1, 2, 3, 5 and 8 workers, every arm + `memcmp`-equal to the 1-worker arm, which short-circuits to the pre-change + serial path. +* **The A/B harness**, independently: across **84 decodes** spanning worker + counts 1, 2, 4, 8, 16 and 20, two sweep directions and two tensor shapes, the + output checksum was **bit-identical every time** — `763841.709997177` at + `c=64` and `973177.818164825` at `c=128`, on pseudo-random weights and a + pseudo-random latent rather than the engineered fixture. + +### 8.3 The wall-clock A/B + +Same binary throughout; `VLLM_CPP_CPU_THREADS` selects the pool width and +nothing else changes. One decode driven through `Ltx2VideoDecodeStreaming` at a +**synthetic** decoder configuration — stated as synthetic because the shipped +checkpoint's `decoder_blocks` list lives in a checkpoint header this box does not +have. 14 runs per thread count: 7 on an ascending sweep and 7 on a descending +one, so an ordering drift would show as a spread rather than hide in a mean. + +Configuration: `in_channels = base_channels = 64`, `out_channels = 3`, +`patch_size = 1`, one `res_x` block of 2 layers, non-causal, no timestep +conditioning, `PixelNorm`, replicate padding, latent `64 x 5 x 40 x 40`. Six +convolutions, 8.930 GFLOP. + +| threads | runs | min s | median s | max s | spread | speedup | parallel efficiency | +|---|---|---|---|---|---|---|---| +| 1 | 14 | 1.9859 | **2.0418** | 2.1172 | 6.4% | 1.00x | 100% | +| 2 | 14 | 1.0266 | **1.0552** | 1.0843 | 5.5% | **1.93x** | 96.7% | +| 4 | 14 | 0.5464 | **0.5555** | 0.5674 | 3.8% | **3.68x** | 91.9% | +| 8 | 14 | 0.2920 | **0.3013** | 0.3072 | 5.0% | **6.78x** | 84.7% | +| 16 | 14 | 0.2129 | **0.2232** | 0.2597 | 21.0% | **9.15x** | 57.2% | +| 20 | 14 | 0.2024 | **0.2234** | 0.2534 | 22.8% | **9.14x** | 45.7% | + +A second shape at the checkpoint's real `base_channels`, 3 runs each: +`c = 128`, latent `128 x 5 x 32 x 32`, 22.755 GFLOP — 5.1015 s at one thread +against 0.5276 s at twenty, **9.67x**. + +**The load this was taken at.** One-minute load average 4.03 to 6.77 across both +sweeps, on a box whose one-minute average had been between 2 and 52 earlier the +same day. It was not idle and no measurement here claims it was: one non-agent +process (`minimax-music3-`, PID 2291593, running for 9h57m) held ~1.07 cores for +the entire measurement. That process alone accounts for part of the gap at 16 and +20 threads, and it is also why the 16- and 20-thread spreads are 21-23% where +every count at or below 8 is under 7%. + +**No ceiling is declared.** The curve flattens at ~9.15x from 16 threads, and the +next traceable hypothesis is named rather than the flattening being called a +limit: Amdahl on the passes this row deliberately did **not** parallelise. From +the measured 9.14x at 20 workers the implied serial fraction is 6.3% +(`1/(s + (1-s)/20) = 9.14` gives `s = 0.063`), which is the right order for +`PixelNorm`, `Silu`, `ApplyAdaLn`, the residual add and `expand` — every one of +them still serial, every one of them listed under §7. Memory bandwidth is the +second candidate and is not separated here. Whether the flattening is Amdahl, +bandwidth, or the ~1.07 cores another process was holding is **not resolved by +this measurement**, and §7 owns the follow-up. + +### 8.4 What this row does NOT claim + +* **No end-to-end render speedup.** No GPU here, no large-render host, and this + row never ran a render. The 2681 s figure for a 448x256/25f decode is + [`ltx25-tiled-decode.md`](ltx25-tiled-decode.md)'s measurement on GB10 and + nothing here divides into it. +* **No ratio against any oracle.** `ltx_core` is not installed on this box and + has no pin; §1 and the parent spec's §7 both say why there is no denominator. +* **No composition figure with #1008.** The dtype row landed unmeasured for + speed, and separating the two contributions needs the f64 arm rebuilt and + re-timed. Not done. +* **Nothing about the shipped checkpoint's shape.** The harness configuration is + synthetic. What generalises from it is the *scaling*, not the absolute wall. + +### 8.5 ThreadSanitizer, with the instrument positive-controlled first + +`cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DVLLM_CPP_CUDA=OFF +-DVLLM_CPP_SANITIZE=thread`, three suites. + +| suite | exit | cases | `WARNING: ThreadSanitizer` | +|---|---|---|---| +| `test_ltx2_vae` | 0 | 42/42 | 0 | +| `test_ltx2_tiling` | 0 | 10/10 | 0 | +| `test_ltx2_video` | 0 | 57/57 | 0 | + +**Two things had to be settled before that table meant anything.** + +First, the binaries would not start: `FATAL: ThreadSanitizer: unexpected memory +mapping`, exit **66**, before a single case ran. That is the kernel's ASLR +entropy against TSan's fixed shadow layout, not a defect in this change, and it +is a verdict-shaped instrument failure — a `&&` chain would have read that +non-zero exit as a race. `setarch x86_64 -R` fixes it and every run above uses +it. + +Second, a sanitizer that reports nothing is indistinguishable from one that is +not instrumenting. A deliberate race — one unsynchronised `static int64_t` +incremented from inside `CausalConv3d`'s parallel body — was compiled into the +same lane, and TSan reported **87** `WARNING: ThreadSanitizer: data race` with +`EXIT=66`, naming the `CausalConv3d` lambda by its full signature and the line +the race sat on in the mutated tree. No anchor is cited for that line, because +that tree no longer exists. The mutation was then reverted, rebuilt, and rerun: +back to 0 warnings and `EXIT=0`. The clean table is a measured clean, not a +silent one. + +### 8.6 Mutations, each with three facts + +`git diff --numstat`, whether it BUILT with its `: error:` count, and the exit +code captured directly rather than through a pipe. + +| mutation | numstat | built | exit | detected by | +|---|---|---|---|---| +| **T0** — all three dispatches reverted to serial | 13/8 | yes, 0 errors | **1** | Case A, `CHECK( 0 > 0 )` on the cursor | +| T1 — `CausalConv3d`'s OUTPUT loop alone reverted | 3/2 | yes, 0 errors | **0** | **nothing. 42/42 and 10/10 pass** | +| T2 — the padding gather alone reverted | 5/3 | yes, 0 errors | **0** | **nothing. 42/42 and 10/10 pass** | +| T3 — `Linear3d` alone reverted | 5/3 | yes, 0 errors | **0** | **nothing. 42/42 and 10/10 pass** | +| D1 — chunk-boundary-dependent value, visible at 1 worker too | 1/0 | yes, 0 errors | **1** | 10 cases in `test_ltx2_vae` + 2 in `test_ltx2_tiling`, including Case B | +| **D2** — the same defect made INVISIBLE to the 1-worker arm | 1/0 | yes, 0 errors | **1** | Case B's `memcmp`, on all four of the 2/3/5/8-worker arms | +| **R** — the production `Ltx2ConvVideoDecode` call site deleted | 17/2 | yes, 0 errors | **1** | Case A on the cursor AND on the value; Case B's non-degeneracy `REQUIRE` | +| T1, first attempt | 4/2 | **NO, 45 errors** | — | **nothing — a mutation that does not build establishes nothing** | + +**T1's first attempt is in the table on purpose.** One unbalanced brace closed +the anonymous namespace early and produced 45 `-Werror` errors that read as +unrelated `unused-function` complaints hundreds of lines away. The runner +refused to draw a verdict, printed the errors and restored the tree. Had it run +the stale binary instead, it would have printed a plausible 42/42. + +**T1, T2 and T3 are an honest gap, and it is owed.** Case A observes one +work-stealing cursor, and the cursor is shared: reverting any single site leaves +the other two dispatching, so the case reads non-zero and passes. It gates *"at +least one of the three sites dispatches partitioned work"*, not each site +individually, and T0 is what holds the conjunction. T3 additionally cannot be +seen by this fixture at all, because `decoder_blocks` is empty and `Linear3d` is +only reached through a `res_x_y` block. What does bound each site is §8.1's +golden table — the "Conv video decoder" arm reaches all three at 20 workers and +its margin did not move — and §8.3's wall-clock, which is what a serial +convolution would actually cost. Closing the gap properly needs one dispatch +observation per site, which needs an instrument the pool does not have today. + +**D1 is in the table beside D2 because it is the weaker of the two.** D1 +perturbs the first row of every chunk including the first, so the 1-worker arm +moves as well and the case fails on its value assertion before reaching the +`memcmp`. D2 perturbs only chunks that do not start at row 0, which is invisible +at one worker — `ParallelForRows` short-circuits to `body(0, nr)` there — so the +`memcmp` across worker counts is the only thing that can report it. It does, on +every one of the four non-base arms. That is the determinism guarantee mutated +rather than read. + +### 8.7 The gate + +```sh +cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DVLLM_CPP_CUDA=OFF +cmake --build build -j6 +ctest --test-dir build -j4 --output-on-failure +``` + +Run twice: once on the implementation commit, and once again at the branch head +so that a green gate is chained directly to the push. + +| | first run | head run | +|---|---|---| +| `CONFIGURE_EXIT` | 0 | 0 | +| `BUILD_EXIT` | 0 | 0 | +| `: error:` count | **0** | **0** | +| `ctest -N` | **492** | **492** | +| `CTEST_EXIT` | **0** | **0** | +| result | **100% passed, 0 failed of 492** | **100% passed, 0 failed of 492** | +| total test time | 308.99 s | 316.87 s | +| one-minute load | 32 to 52 | **82 to 94** | + +Two skips in both runs, both pre-existing and unrelated: +`test_modelopt_mixed_precision_checkpoint` and `test_voxtral_e2e`. + +`No space left` **0** and `BFD` internal-error/assertion **0** across every build +and ctest log, both greps positive-controlled against a synthetic file carrying +the real message forms — 1 and 2 hits respectively there, 0 in the real logs. + +None of the load-dependent suites flaked in either run, and the head run passed +at a one-minute load of 82-94 on a 20-core box, which is four times +oversubscribed. Free disk 21-30 GiB of 447 GB throughout; it dipped to 19 GiB +mid-run under other agents' builds. The sanitizer tree was 834 MiB and was +removed after §8.5. + +### 8.8 The harness, recorded so the measurement is reproducible + +Not shipped — a scratch developer tool, and the row deliberately does not add a +benchmark surface to the tree for it. Built against the gate's own `libvllm.a`: + +```sh +g++ -O3 -DNDEBUG -std=c++20 -I include -I third_party ltx2_decode_bench.cpp \ + build/libvllm.a build/libblake3_vendored.a -lpthread -o bench +VLLM_CPP_CPU_THREADS= ./bench +``` + +It builds `Ltx2VaeWeights` from a fixed LCG, decodes through +`Ltx2VideoDecodeStreaming` with an untiled `Ltx2TileSizeConfig`, and prints wall +seconds, the derived GFLOP/s and a full-output checksum per run. The checksum is +what makes it a determinism instrument as well as a timer: it is printed at every +thread count and must not move. + +## 9. Stop conditions * Report `NEEDS_DECISION` rather than widening `kLtx2GoldenTol`, or any other tolerance, if a partition moves a golden. The answer to a moved golden is a From 15e5556b7c18e6cb7adabfe8e8ae3372bf52b9b2 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Sun, 16 Aug 2026 18:00:12 +0000 Subject: [PATCH 4/4] fix(LTX25-DECODE-THREADS): the review's five non-blocking findings, and the one that would have made a later agent weaken the test (#1044) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review of [PR #1041](https://github.com/mudler/vllm.cpp/pull/1041) returned PASS with five non-blocking findings. Each was re-verified before it was repaired, because a reviewer's finding is a hypothesis; none of the five was rejected. **F1 — the T1/T2/T3 gap now has an owner.** §8.6 measured that reverting any single one of the three dispatch sites is detected by nothing, and argued it in prose only. It is now [#1044](https://github.com/mudler/vllm.cpp/issues/1044), an entry in §7's `## Owed` table, and an index row owned by `LTX25-DECODE-THREADS`. The issue carries the closing test the reviewer supplied: a per-dispatch `Threadpool::RunCount()` and an EXACT expected count rather than `> 0`, plus a `res_x_y` fixture because `Linear3d` is unreachable with `decoder_blocks` empty. It is NOT implemented here — a new gate needs its own red-before evidence and its own review. The index row names the owning row rather than leaning on `## Owed`, and that is deliberate: `owed_issues()` in `scripts/check-agent-record.py` splits on a bare `\n## Owed`, this spec's heading is `## 7. Owed`, so nothing listed there is visible to the unowned ratchet. Measured: the unowned count is 33 before and after, against `UNOWNED_HIGH_WATER = 33`. **F2 — the public records carried three significant figures a 21-23% spread does not support.** `9.14x`/`9.67x` appeared bare in `FEATURES.md` and `USAGE.md` while the spec disclosed the load and the spread; the projection is what lost them. Both now carry `~9x at 16-20 workers` with the conditions, and the bare `9.67x at c=128` — n=3, no min/median/max, same contended box — is gone from `FEATURES.md` rather than restated. `MAX_CELL_CHARS = 220` binds and the `BENCHMARKS.md` LTX-2.5 cell sat at exactly 220: it is now 212, and the `FEATURES.md` cell 210 -> 204, both measured with the checker's own parser. **F3 — the #1009 index row was cited at `:275` on PR #1018.** #1018 is CLOSED and superseded by [#1038](https://github.com/mudler/vllm.cpp/pull/1038); the row lives at line **279** on `row/LTX25-DECODE-SPEED-R2`. The decision not to append a second #1009 row is correct and unchanged — under `merge=union` a duplicate reds `main` for every branch the moment #1038 lands. Only the citation moved, here and in the pull request body, along with the spec's two other pointers at the closed pull request. **F4 — the evidence SHA did not resolve.** §8 cited `dac85969c`, which is not an ancestor of the head (`git merge-base --is-ancestor` exits 1) and would not exist in a fresh clone. §8 now cites `d653f7319` and states why the measurement transfers: `dac85969c:src` and `d653f7319:src` are both `7444ffa171b0c2868c505b5b9ea1113fa39c5477`, both `:tests` are `f0e5eac268119e9fe94da478c50e2d668a2e64b3`, and the diff between them touches only the spec and three `docs/` files. **F5 — the determinism test's stated reason was arithmetically false, and that is the finding that mattered.** The comment said 3 and 5 were chosen because they "do not divide the row counts". Both conv row counts are 360 (`conv_in`, 24*3*5) and 15 (`conv_out`, 1*3*5), and 3 and 5 divide each of them. The choice works for a different reason: `nchunk` derives from `nth * 4`, so at `nr = 360` the stride is 45 at 2 workers, 30 at 3, 18 at 5 and 12 at 8 — four DIFFERENT partitions, which is the property the `memcmp` needs. Verified by replicating `cpu_threadpool.cpp:428-443` rather than by reading it. A later agent "fixing" the row counts to satisfy the stated rationale would have weakened the test while believing it was strengthening it, so the comment now says so outright. **Out of scope and untouched:** the parallelisation, the three dispatch sites, `kLtx2GoldenTol` (5e-06 on both arms), the new cases' assertions, and the `## Owed` ratchet beyond F1's entry. Nothing was re-measured: `dgx.casa` is down and the A/B harness is deliberately not in the tree, so every wall-clock figure stands as the implementer recorded it. Gate at this tree: `CONFIGURE_EXIT=0`, `BUILD_EXIT=0`, `: error:` count 0 on a full 1449-target build, `ctest -N` 492, `CTEST_EXIT=0`, **100% tests passed, 0 tests failed out of 492** in 165.47 s, the same two pre-existing skips. `No space left` 0 and `BFD`/internal-error 0 across both logs, each grep positive-controlled against a synthetic file carrying the real message forms (2, 1 and 2 hits there). One-minute load 10 to 26; free disk 40 GiB falling to 21 GiB. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code] --- .agents/issue-index.md | 1 + .agents/specs/ltx25-decode-threads.md | 65 ++++++++++++++++++++------- docs/BENCHMARKS.md | 2 +- docs/FEATURES.md | 2 +- docs/USAGE.md | 11 +++-- tests/vllm/models/test_ltx2_vae.cpp | 17 +++++-- 6 files changed, 74 insertions(+), 24 deletions(-) diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 90c07ad1b..bb466dce5 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -273,3 +273,4 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#999](https://github.com/mudler/vllm.cpp/issues/999) | — | `scripts/check-commit-style.py` `validate_range` still raises `range base must be an ancestor of range head`, so the merge-base repair `GATE-FORK-ANCESTRY` (#773) applied to `check-commit-trailers.py` never reached it: the checker aborts before reading a commit on any branch cut before the last merge of `main`. Found auditing the guards for #998, and it is why that row reports a SKIP rather than dropping the guard and letting both checkers run. Owed by [`gate-preflight-skip-report.md`](specs/gate-preflight-skip-report.md) | bug | | [#1000](https://github.com/mudler/vllm.cpp/issues/1000) | `ENG-EXPERT-STREAM` | `main` is RED on `check-env-doc` at `3ce1cf7c7`: `VT_MOE_EXPERT_STREAM`, `VT_MOE_EXPERT_STREAM_SLOTS` and `VT_MOE_EXPERT_STREAM_SLOT_BYTES` arrived with `3005447f8` (#993) and are neither in `docs/ENVIRONMENT.md` nor on `scripts/env-doc-allowlist.txt`, so `check-env-doc` and `test_check_env_doc` fail on every branch cut from current main. Measured in a clean worktree at that SHA with no local edits. Found running the preflight as the gate for #998, and NOT repaired there: choosing documented knob versus internal tuning switch for each var belongs to the row that added them | bug | | [#1023](https://github.com/mudler/vllm.cpp/issues/1023) | `ENG-EXPERT-STREAM` | The IQ1_S (ggml 19) / IQ1_XXXS (ggml 66) decode landed in [#946](https://github.com/mudler/vllm.cpp/pull/946) with every parameter EXCEPT the grid pinned only by self-consistency. The grid seal (FNV-1a digest + lane census) works and stops one table short: `ReferenceDotF64` (`tests/vt/test_ops_quant_dot.cpp:231-236`) and the G3 NMSE reference (`:915`) both decode the weight with `vt::cpu::BlockToFloat`, the function under test, so they are independent only in the SUMMATION. Three injected defects, each applied and compiled, left the suite green with an UNCHANGED assertion count: `kIq1sDelta` `0.125F`->`0.25F` (`cpu_quant_iq_tables.h:422`, affects BOTH encodings), the IQ1_S delta sign inverted in dequant AND vec_dot, and the IQ1_S scale read from `qh` bits 13-15 instead of 12-14 in both paths. Second defect, same PR: `gguf_dequant.cpp:107-116` lists no `case 19` and no `case 66`, so the expansion path throws `unsupported ggml type` for the two encodings the target checkpoints are 96.92 % made of, and `RouteGgufTensor` (`gguf_keep_quant.cpp:122`) sends a tensor there whenever `VT_CPU_REF` is on, keep-quant is off, K is ragged, or the role is not verbatim — a refusal to load on the reference lane. **ggml 18 (IQ3_XXS) carried the same omission**, pre-existing since the DeepSeek-V4 UD-IQ2_XXS port, and all three are one shared `switch` branch. Repaired by golden vectors whose EXPECTED values come from the ORACLES themselves (`ggml_get_type_traits(type)->to_float` built from `ggml-org/llama.cpp @ 237ad9b96` for 18/19 and `unslothai/llama.cpp @ 36fe8e1cc` for 66) over REAL checkpoint bytes, which is the first reference for this encoding family that is not this tree's own decoder. Also: the `2e-3` NMSE ceiling passed the doubled-delta defect that `6e-4` fails (measured 5.240e-4 unmutated, 6.967e-4 mutated on `iq1_s`), and on `iq1_xxxs` that defect moves the statistic the WRONG WAY (3.109e-4 -> 1.420e-4), so no ceiling can catch it and only the goldens can; and both new census cases claimed TOTAL coverage of 1702 records while summing to 864, F32's 838 tensors omitted. Spec [`expert-streaming.md`](specs/expert-streaming.md) | bug | +| [#1044](https://github.com/mudler/vllm.cpp/issues/1044) | `LTX25-DECODE-THREADS` | The three parallel dispatch sites [#1009](https://github.com/mudler/vllm.cpp/issues/1009) added to the LTX-2.5 conv video VAE share ONE work-stealing cursor, so reverting any SINGLE one of them is detected by nothing: `CausalConv3d`'s padding gather (`src/vllm/model_executor/models/ltx2_video_vae.cpp:170 @ 249418305`), its output nest (`:218`) and `Linear3d` (`:276`). The instrument, the case "the decode DISPATCHES its convolutions to the CPU threadpool", reads `Threadpool::ChunkAdd(0)` and that cursor is seeded once per pool (`src/vt/cpu/cpu_threadpool.cpp:438 @ 249418305`, advanced at `:455`), so it gates "at least ONE of the three dispatches", never each site. MEASURED as T1/T2/T3 in [`ltx25-decode-threads.md`](specs/ltx25-decode-threads.md) §8.6 and reproduced independently by that row's reviewer: each single-site revert BUILT with 0 errors and left ctest at exit 0 and 42/42 + 10/10 green; only reverting all three (T0) goes red. NOT a correctness hole -- 34 golden margins were byte-identical, the bit-identity case `memcmp`s five worker counts, and ThreadSanitizer is clean against an 84-race positive control -- but a site can silently go serial again and only a wall-clock nobody runs in CI would notice. Closing it needs a per-dispatch `Threadpool::RunCount()` bumped in `Run()` (`src/vt/cpu/cpu_threadpool.h:112 @ 249418305`; no such counter exists) and an EXACT expected count rather than `> 0`, plus a fixture carrying a `res_x_y` block, because `MakeLtx2ThreadFixture` sets `decoder_blocks = {}` and `Linear3d` is unreachable without one. A new gate needs its own red-before evidence and its own fresh review, so it is a row rather than an in-flow repair. Listed under `## 7. Owed` in [`ltx25-decode-threads.md`](specs/ltx25-decode-threads.md) | verification | diff --git a/.agents/specs/ltx25-decode-threads.md b/.agents/specs/ltx25-decode-threads.md index 784c7f8cc..644474b9c 100644 --- a/.agents/specs/ltx25-decode-threads.md +++ b/.agents/specs/ltx25-decode-threads.md @@ -7,9 +7,12 @@ Parent: lever 3 of the `LTX25-DECODE-SPEED` investigation ([#1006](https://github.com/mudler/vllm.cpp/issues/1006)), which filed this issue and lists it under `## Owed`. That spec is `.agents/specs/ltx25-decode-speed.md` on [PR -#1018](https://github.com/mudler/vllm.cpp/pull/1018) and is **not yet on -`main`**, so it is cited by pull request rather than by relative link, exactly as -the sibling dtype row ([`ltx25-decode-dtype.md`](ltx25-decode-dtype.md)) does. +#1038](https://github.com/mudler/vllm.cpp/pull/1038), branch +`row/LTX25-DECODE-SPEED-R2`, and is **not yet on `main`**, so it is cited by pull +request rather than by relative link, exactly as the sibling dtype row +([`ltx25-decode-dtype.md`](ltx25-decode-dtype.md)) does. It was PR #1018 while +this row was implemented; that pull request is now **closed** and #1038 +supersedes it, so every citation here names the open one. Sibling, and the reason this row is riskier than it looks: [`ltx25-decode-dtype.md`](ltx25-decode-dtype.md) (#1008) landed at `d1b0ea3a8` @@ -20,9 +23,11 @@ change a summation order. ## Now `DONE`, pending review. The three convolution sites dispatch, the numerics are -byte-identical to the serial arm, and the CPU A/B is in `## Outcome`: **9.14x at -20 workers and 9.67x at the checkpoint's real channel width**, measured, on this -box, at a recorded load. No end-to-end render number, and none is claimed. +byte-identical to the serial arm, and the CPU A/B is in `## Outcome`: **~9x at 16 +to 20 workers**, medians 9.15x and 9.14x, on a box that was not idle and where +those two counts spread 21-23% run to run. A second channel width corroborates at +9.67x on n=3. Read the band, not the decimals; §8.3 carries both with the load +they were taken at. No end-to-end render number, and none is claimed. ## 0. Scope @@ -55,7 +60,7 @@ at fixed thread counts on 20 local cores (§6). Every oracle runs this decoder on an accelerator and none of them has a host-parallel arm to port. The four anchors below are the parent investigation's -([`ltx25-decode-speed.md` on PR #1018](https://github.com/mudler/vllm.cpp/pull/1018) §2), read there at the +([`ltx25-decode-speed.md` on PR #1038](https://github.com/mudler/vllm.cpp/pull/1038) §2), read there at the revisions named; none of these repositories is checked out on this box, so this row cites them rather than re-deriving them. `ltx_core` has no `.agents/oracles/` file at all — that pin is owed by @@ -76,7 +81,7 @@ lists both. (`ltx2_diffusion_decoder.py:208-209`, *"No CPU path"*). So this row is a **local seam**, not an upstream mirror, and -[`ltx25-decode-speed.md` on PR #1018](https://github.com/mudler/vllm.cpp/pull/1018) §6 lever 3 records it as such. +[`ltx25-decode-speed.md` on PR #1038](https://github.com/mudler/vllm.cpp/pull/1038) §6 lever 3 records it as such. What it does mirror is *this tree's own* CPU convolution: `src/vt/cpu/cpu_conv2d.cpp:75-78 @ d1b0ea3a8` partitions a 2-D convolution over `n * cout * hout` output lines through the same call, with the same comment this @@ -268,21 +273,45 @@ large-render host, and no installed `ltx_core`. | A SIMD inner tap loop | Separate summation-order question; see §0. | | The composition of this row with #1008 | the parent spec §6 warns that a threaded arm may become memory-bound where the scalar arm was ALU-bound. This row measures its own axis only. | | An end-to-end render number | `dgx.casa` unreachable; no GPU here. | +| A per-SITE dispatch gate — [#1044](https://github.com/mudler/vllm.cpp/issues/1044) | §8.6's T1/T2/T3 measured it: reverting any ONE of the three sites is detected by nothing, because Case A reads one cursor the whole pool shares. Correctness stays gated; what is ungated is a site silently going serial again. Closing it needs a per-dispatch `Threadpool::RunCount()` and an EXACT expected count, plus a `res_x_y` fixture for `Linear3d`. A new gate needs its own red-before evidence and its own review, so it is a row rather than an in-flow repair. | No `.agents/issue-index.md` row is appended for #1009. That row already exists at -`.agents/issue-index.md:275` on PR -[#1018](https://github.com/mudler/vllm.cpp/pull/1018), which filed the issue and -is unmerged. `.gitattributes` sets `merge=union` on that file and +`.agents/issue-index.md:279` on PR +[#1038](https://github.com/mudler/vllm.cpp/pull/1038), branch +`row/LTX25-DECODE-SPEED-R2`, which is the open successor to the closed PR #1018 +and is unmerged. `.gitattributes` sets `merge=union` on that file and `scripts/check-agent-record.py` refuses a duplicate issue number, so appending a -second copy here would turn `main` red for every branch the moment #1018 merges — +second copy here would turn `main` red for every branch the moment #1038 merges — which is exactly what a duplicate #995 row did on 2026-08-16. The sibling dtype row made the same call for #1008 and recorded it in its pull request body. +**#1044 is different and IS appended.** It is a new issue this change filed, it +is not one of the ids #1038 appends (#1006-#1012, #1014-#1016, #1021, #1024, +#1040), and its index row names `LTX25-DECODE-THREADS` as the owning row. The +owner has to be named in the row rather than left to this section, because +`owed_issues()` in `scripts/check-agent-record.py` splits on a bare `\n## Owed` +and this spec's heading is numbered, so nothing listed here is visible to that +ratchet. + ## 8. Outcome — what was measured Everything below was measured on the shared 20-core development box on -2026-08-16, at branch head `dac85969c`. No GPU was involved and none was -available; `dgx.casa` was unreachable for this row's whole duration. +2026-08-16, at **`d653f7319`**, the implementation commit on this branch. No GPU +was involved and none was available; `dgx.casa` was unreachable for this row's +whole duration. + +**The commit the measurement ran on was `dac85969c`, and that SHA is deliberately +not the citation.** It was rewritten out of the branch and is not an ancestor of +the head (`git merge-base --is-ancestor dac85969c HEAD` exits **1**), so it does +not resolve in a fresh clone and citing it would name evidence nobody can reach. +The measurement transfers because the two commits are **byte-identical in +`src/` and `tests/`**: `git rev-parse dac85969c:src d653f7319:src` both give +`7444ffa171b0c2868c505b5b9ea1113fa39c5477` and both `:tests` give +`f0e5eac268119e9fe94da478c50e2d668a2e64b3`, and +`git diff --stat dac85969c d653f7319` touches only +`.agents/specs/ltx25-decode-threads.md`, `docs/BENCHMARKS.md`, +`docs/FEATURES.md` and `docs/USAGE.md`. Every number below came out of a binary +built from the code `d653f7319` carries. ### 8.1 The numerics did not move at all, and that is the point @@ -360,6 +389,11 @@ A second shape at the checkpoint's real `base_channels`, 3 runs each: `c = 128`, latent `128 x 5 x 32 x 32`, 22.755 GFLOP — 5.1015 s at one thread against 0.5276 s at twenty, **9.67x**. +**That 9.67x is the weakest number this row produced, and it is labelled as +such wherever it is projected.** `n = 3` against the table's 14, no min/median/max +recorded, and the same contended box — so it corroborates the table's shape at a +second channel width and is not independently a three-significant-figure result. + **The load this was taken at.** One-minute load average 4.03 to 6.77 across both sweeps, on a box whose one-minute average had been between 2 and 52 earlier the same day. It was not idle and no measurement here claims it was: one non-agent @@ -445,7 +479,8 @@ unrelated `unused-function` complaints hundreds of lines away. The runner refused to draw a verdict, printed the errors and restored the tree. Had it run the stale binary instead, it would have printed a plausible 42/42. -**T1, T2 and T3 are an honest gap, and it is owed.** Case A observes one +**T1, T2 and T3 are an honest gap, and it is owed as +[#1044](https://github.com/mudler/vllm.cpp/issues/1044)** (§7). Case A observes one work-stealing cursor, and the cursor is shared: reverting any single site leaves the other two dispatching, so the case reads non-zero and passes. It gates *"at least one of the three sites dispatches partitioned work"*, not each site diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md index 2a18777dc..00d6edb80 100644 --- a/docs/BENCHMARKS.md +++ b/docs/BENCHMARKS.md @@ -462,7 +462,7 @@ built on it rather than keeping the flattering one. | Kimi-Linear-48B-A3B (KDA+MLA+MoE) | **RUNNER FOLD LANDS (ROW 7, §21, #122): engine==CLI 128/128 byte-identical; vs golden 122/128 (near-tie profile); FA2 MLA default-ON; SACRED green.** Server 19.0 tok/s wall; CLI 18.93 reproduced | vLLM ~21 (#111 floor; in-session re-measure ABORTED by GB10 reboot at util 0.82, §21): **~0.90×**, >= vLLM NOT met; residual = KDA host islands + grouped MoE + decode graph | | vLLM 0.26 re-benchmark | Pending | Re-run the binding grids on the advanced pin | | MiniMax-H3 FP4 speed (W-FP4a) | **Measured GB10 (`row/H3-FP4-GPU-E2E`).** Marlin W4A16 byte-exact vs bf16; fp4 a memory win, 0.8x bf16/forward. Real-ckpt fp4-resident e2e RUNS (mp4/wav) | fp4 speed CLOSED. bf16-vs-quant A/B: ENCODER half MEASURED (§8.15), DiT half NOT (no bf16 render exists). Detail: benchmark-record + spec §8 | -| LTX-2.5 axes | Speed `PENDING` (vllm-omni#6066 has no native 2.5), binding oracle too. **SIZE: 320x192/25f completes on GB10, 448x256 does not**; that render was REGISTER-conditioned, not prompted | Wall is the HOST VAE decode, not the pool: drain returns 0.11 GiB, byte-inert. 2 baselines UNRESOLVED (lock). A real-checkpoint PROMPTED render is OWED. Decode now THREADED: 9.14x at 20 CPU workers, bit-identical (#1009) | +| LTX-2.5 axes | Speed `PENDING` (vllm-omni#6066 has no native 2.5), binding oracle too. **SIZE: 320x192/25f completes on GB10, 448x256 does not**; that render was REGISTER-conditioned, not prompted | Wall is the HOST VAE decode, not the pool: drain returns 0.11 GiB, byte-inert. 2 baselines UNRESOLVED (lock). A real-checkpoint PROMPTED render is OWED. Decode THREADED ~9x at 16-20 workers, contended box (#1009) | | MiniMax-Music3 (`MiniMaxMusic3ForConditionalGeneration`) | **Every axis `PENDING`, now OWED.** The pipeline runs end to end, so a forward pass exists to time; every gate was taken on CPU with `dgx.casa` down, and a CPU number against a graphed denominator is dishonest | Denominator: SGLang-Omni `748a0b43` in its production configuration (both CUDA graphs, compiled DIT and DAV, batched seeded sampling) | | MiniMax-H3 render coherence (`row/H3-RENDER-CLOSE` #77) | **CLOSED: a COHERENT scene on GB10.** #70/#74 white was wrong-PARTITION usage (t2va on the ref2va ckpt); t2va on the FL2VA GGUF renders a prompt-matched orange cat (adj-cos 0.95 vs 0.06, no patch-grid) | Verified first: t2va inputs byte-exact vs upstream; CUDA device==host at seq 1920. Follow-up `H3-TASK-PARTITION-GUARD`: the task/partition mismatch now RAISES 1:1 with `_resolve_task` (spec §8.6-8.7) | | MiniMax-H3 image conditioning (`row/H3-CONDITIONED-E2E`, `row/H3-VISION-SCATTER`, `row/H3-REF2VA-ASSEMBLY`) | **fl2va COHERENT; ref2va assembly bug FIXED+gated.** vision→cond scatter gated; ref2va block-dim double-division fixed + RED-first gated (128 vs 512) + a permanent ref2va DiT-forward rung (§8.10) | grid RE-ATTRIBUTED: with the fix ref2va grids in fp4 AND bf16, and t2va with no refs on the ref2va NVFP4 also grids while FL2VA-GGUF renders, so it is the **NVFP4 checkpoint/loader**, NOT assembly/fp4 (§8.10) | diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 0e5e546cc..6a3af7f17 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -169,7 +169,7 @@ in `ltx2_text_encoder.cpp` is the call that would have to change. | LTX-2.5 DFR base + generated keyframe slots | LTX-2.5 (21.00B video+audio) | gated vs EXECUTED upstream `dfr_layout` + 3 `dfr_pipeline` helpers @ `fd4ded7f` (`test_ltx2_dfr` 11/11, 652 assertions); canvas, tiles, stitch, carry-forward as EXACT index vectors, since each defect is plausible| `--pipeline-kind dfr`. Canvas PADS 9 to 25 then trims back; slots on the x8 grid, MARKED, read back BEFORE the trim. `num_generated_keyframes` SERVED elsewhere. Temporal ROUNDS refused (#986); detail LoRA refused (#975)| | LTX-2.5 tiled + streaming Conv VAE decode | LTX-2.5 video VAE | gated vs executed upstream `ltx_core` @ `fd4ded7f` (`test_ltx2_tiling` 10/10, 915 assertions); one-tile and untiled-spatial controls BIT-EXACT vs untiled on both causality arms; an untiled frames axis is REFUSED | Streams temporal chunks through upstream's AUTO layout (768/64 px, 80/24 frames); above one tile the pixel volume is never materialized. NO-OP below 768px and 81 frames; 81-120 IS tiled, differing 6.70% of range | | LTX-2.5 Conv VAE decode arithmetic width | LTX-2.5 video VAE | `test_ltx2_vae` "the decode's convolution accumulates in f32", entering through `Ltx2VideoDecodeStreaming`; widening the accumulator to `double`, or deleting the production call site, each turns it RED | **f32**, the width `F.conv3d` uses at f32 AND bf16 (MEASURED). Was f64 at 8 sites ([#1008](https://github.com/mudler/vllm.cpp/issues/1008)). Conv sums BLOCKED per input channel, as torch's. STORAGE stays f32; bf16 owed | -| LTX-2.5 Conv VAE decode threading | LTX-2.5 video VAE | `test_ltx2_vae` "the decode DISPATCHES its convolutions to the CPU threadpool" and "...BIT-IDENTICAL across thread counts", through `Ltx2VideoDecodeStreaming`; 34 golden margins UNCHANGED; TSan clean | **Parallel** over output lines via `vt::cpu::ParallelForRows` ([#1009](https://github.com/mudler/vllm.cpp/issues/1009)). 9.14x at 20 workers, 9.67x at c=128. Bit-identical at any count; elementwise still serial | +| LTX-2.5 Conv VAE decode threading | LTX-2.5 video VAE | `test_ltx2_vae` "the decode DISPATCHES its convolutions to the CPU threadpool" and "...BIT-IDENTICAL across thread counts", through `Ltx2VideoDecodeStreaming`; 34 golden margins UNCHANGED; TSan clean | **Parallel** over CONV output lines via `vt::cpu::ParallelForRows` ([#1009](https://github.com/mudler/vllm.cpp/issues/1009)). ~9x at 16-20 workers, contended box, 21-23% spread. Bit-identical at any count | | LTX-2.5 retake (`RetakePipeline`, regenerate a time window) | LTX-2.5 DiT + video VAE encoder | `test_ltx2_retake` 4/4 (69 assertions) and 4 `test_ltx2_video` cases entering through `Generate`; mask, conform and the four-way plan pinned to upstream `fd4ded7f` | `--pipeline-kind retake` on `ltx2-gen`. Source is a `frame_%06d.ppm` DIRECTORY; a container is REFUSED (no demuxer). Geometry comes from the clip. A folder has no audio, so the soundtrack is generated | | MTP speculator | Qwen3.6-27B, Qwen3.6-35B-A3B | token-identical to vLLM `mtp` at c1 | ~4% faster c1; +16% output tput (MoE) | | DFlash block-diffusion | Qwen3 (DFlash draft) | near-tie e2e 27/27 vs vLLM | 2.9x over spec-off, 1.003x vs vLLM DFlash-on | diff --git a/docs/USAGE.md b/docs/USAGE.md index 2287bc811..5fd0ab7b8 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -877,10 +877,13 @@ VAE decode at 0% GPU, because that decode has no device arm It is no longer *single-threaded*, which is what this paragraph used to say. The decode's convolutions now dispatch across `VLLM_CPP_CPU_THREADS` workers (default `hardware_concurrency`), bit-identical at every worker count — -[#1009](https://github.com/mudler/vllm.cpp/issues/1009), measured at **9.14x on -20 workers** against one. Read that as a decode figure and not a render one: the -wall above was recorded on GB10 before the change and has not been re-measured, -and the 9.14x was taken on a synthetic decode shape on a 20-core x86 host. Set +[#1009](https://github.com/mudler/vllm.cpp/issues/1009), measured at **roughly +9x on 16 to 20 workers** against one. Take the band rather than a decimal: the +medians are 9.15x at 16 and 9.14x at 20, but those two counts spread 21-23% run +to run on a box that was not idle, where every count at or below 8 spreads under +7%. Read it as a decode figure and not a render one: the wall above was recorded +on GB10 before the change and has not been re-measured, and the ~9x was taken on +a synthetic decode shape on a contended 20-core x86 host. Set `VLLM_CPP_CPU_THREADS` lower if the render has to share the box. *The render behind those numbers was NOT prompted, and it renders a scene without diff --git a/tests/vllm/models/test_ltx2_vae.cpp b/tests/vllm/models/test_ltx2_vae.cpp index f8dd8b9f1..a4c39eb76 100644 --- a/tests/vllm/models/test_ltx2_vae.cpp +++ b/tests/vllm/models/test_ltx2_vae.cpp @@ -1357,9 +1357,20 @@ TEST_CASE("ltx2 vae: the decode is BIT-IDENTICAL across thread counts") { // // Worker count 1 short-circuits ParallelForRows to `body(0, nr)` on the caller // (cpu_threadpool.cpp:423-426), so the 1-thread arm IS the pre-#1009 serial code - // path byte for byte, and every other arm is compared against it. The counts are - // not all powers of two on purpose: 3 and 5 do not divide the row counts, so the - // chunk boundaries land in different places on every arm. + // path byte for byte, and every other arm is compared against it. + // + // WHY 3 AND 5, STATED CORRECTLY. It is NOT that they fail to divide the row + // counts — they divide both of this fixture's conv row counts exactly. conv_in + // partitions 24*3*5 = 360 output lines and conv_out 1*3*5 = 15, and 3 and 5 + // divide each of those. The mechanism is that a chunk boundary is a function of + // `nth * 4`, not of `nth`: ParallelForRows takes four chunks per thread + // (cpu_threadpool.cpp:428-431), so `dr = ceil(nr / nchunk)` (:443) with + // `nchunk = ceil(nr / ceil(nr / (nth*4)))`. At nr = 360 that is a stride of 45 + // at 2 workers, 30 at 3, 18 at 5 and 12 at 8 — four DIFFERENT partitions of the + // same output, which is what the memcmp needs. Do NOT "fix" the fixture's row + // counts to make them indivisible by 3 and 5: that would change the shape for a + // reason that was never true, and 4 distinct strides is the property that + // matters. const Ltx2ThreadFixture f = MakeLtx2ThreadFixture(); std::vector base;