feat(LTX25-KEYFRAME-INTERP): the interpolation pipeline, on the conditioning builder that has no frame-0 branch (#1096)

[localai-bot]

Closes #1096. Closes #1191.

pipeline_kind = keyframe_interpolation resolves KeyframeInterpolationPipeline
(keyframe_interpolation.py:55 @ fd4ded7f) on all four generations the table
keys. Asking for it used to get the generic table refusal. It was the last
unported LTX-2.5 pipeline that is neither hardware- nor artifact-blocked.

Structurally it is ti2vid_two_stage: the same parser, the same guided half-res
stage 1 on the UNADAPTED model, the same Euler stepper (derived — neither
self.stage_1(...) at :231 nor self.stage_2(...) at :271 passes one, so
utils/blocks.py:524-527 applies), the same frozen three-sigma stage 2, and the
same schedule_tokens = kSchedulerDefault because :200 calls
execute(steps=...) with no latent. Two fields differ, and both of them render
either way.

The conditioning builder, which is what this pipeline is named after

:211 and :260 call image_conditionings_by_adding_guiding_latent
(helpers.py:343-367). Every other pipeline calls combined_image_conditionings
(:272-308). The two differ by one branch:

	combined_image_conditionings	..._by_adding_guiding_latent
frame_idx == 0	VideoConditionByLatentIndex (:295-300)	VideoConditionByKeyframeIndex
any other frame_idx	VideoConditionByKeyframeIndex (:301-305)	VideoConditionByKeyframeIndex
what frame 0 does	REPLACES latent frame 0's clean tokens; the count never moves (latent_cond.py:38-39)	APPENDS a latent frame of tokens (keyframe_cond.py:79-82)

The second has no branch at all. The first image is a keyframe the model
interpolates from, not a frame it overwrites. This engine hard-coded the
other arm, and both conditioning primitives were already ported and gated — what
was missing is the SELECTION, now Ltx2PipelineRecipe::image_conditioning,
defaulting to today's behaviour so nothing landed moves, read in exactly one
place.

Nothing about a render can see this. The wrong builder returns a clip of the
right size, the right frame count and the right sample rate with the image
visibly present. It is conditioned; it is conditioned as a different pipeline.
The only observable is the sequence length the DiT ran over, and the gate renders
the same image at the same geometry on both kinds plus a bare control that
pins the target grid — without which the comparison passes on a tree where both
arms append. Measured: video_tokens 12 on keyframe_interpolation against 8 on
ti2vid_two_stage and 8 bare, with image_tokens 4 on both.

audio_output_phase is 1, and on ti2vid_two_stage it is 0

:271 binds video_state, audio_state = self.stage_2(...) and :293 decodes
that name. ti2vid_two_stages.py:289 binds video_state, _ under its own
comment at :287-288, "Stage 2 refines video only; discard its audio", and
decodes the name :247 bound. Neither file argues the point, so the binding is
the statement — and copying the neighbour ships a soundtrack one refinement stage
stale, at the right length and the right sample rate. The recipe case asserts
both polarities beside a2vid_two_stage's 1 and res2s_two_stage's 0, so it
cannot pass because every two-stage recipe happens to agree.

Two of #1096's three blockers were stale, and the real difference was in none

.agents/issue-index.md:321 named a multi-keyframe request surface, a per-sigma
guided denoiser, and two missing checkpoints. All three were re-derived at the
pin rather than inherited:

• The per-sigma denoiser is stale on this pipeline's default path. main()
  passes plain MultiModalGuiderParams (:325-340), never a factory, so
  create_multimodal_guider_factory takes its last line,
  MultiModalGuiderFactory.constant (guiders.py:360), which builds
  _params_by_sigma = ((inf, params),) (:312-315) — ONE bin, so
  build_from_sigma returns the same guider at every sigma and
  FactoryGuidedDenoiser delegates to _guided_denoise, which is
  Ltx2GuidedDenoise (landed daeff67f2). The sigma-BINNED arm,
  MultiModalGuiderFactory.from_dict, is real, is reachable only by a caller
  who constructs a factory, and is now The image-conditioning request surface is two fixed slots, against upstream's repeatable --image with a per-item frame index, strength and CRF #1187.
• Both checkpoints are on the NAS, byte-verified, and The bf16 DiT is refused at load, so EVERY LTX-2.5 arm that needs the Full model is unreachable -- and the refusal's advice is unreachable too #1148 closed the
  pure-BF16 DiT refusal at 40a796aa9. What is owed is the RUN.
• The multi-keyframe surface is true and is not the blocker. Two pinned
  keyframes at the two ends is what interpolation means at its default
  configuration. An interior frame_idx is The image-conditioning request surface is two fixed slots, against upstream's repeatable --image with a per-item frame index, strength and CRF #1187.

--last-frame on ltx2-gen (#1191, in flow)

vllm_video_params has carried last_frame and the engine has served it since
#930; ltx2-gen parsed --first-frame and never read the field. That only
starts to bite on a pipeline whose whole job is the motion between two pinned
frames, so this row is the first caller it narrows. One flag, parsed and assigned
beside --first-frame, sharing the --image-crf and strength the two slots
already share.

The gate

Whole binaries, never a --test-case filter. Focused, at this head:

Binary	cases	assertions	exit
test_ltx2_pipeline	56	3316	0
test_ltx2_video	87	2713	0

RED before the recipe landed, on the same binaries: test_ltx2_pipeline 56 / 54
passed / 3183 assertions, Status: FAILURE!, exit 1, both new cases throwing
Unsupported LTX pipeline kind/version: 'keyframe_interpolation'/'2.5';
test_ltx2_video 87 / 83 passed / 2593, exit 1.

Full gate at 251f76b5e, after merging origin/main: CONFIGURE_EXIT=0,
BUILD_EXIT=0, : error: count 0, ctest -N 515, CTEST_EXIT=0,
100% tests passed, 0 tests failed out of 515. No space left and BFD are
each 0 in the build and ctest logs, against injected controls that returned 1.
Load average 50 to 76 across the run and 24 GiB free; none of the four
load-dependent gates (#618, #294, #1052, #428) went red at that load. The same
gate ran green at 1d880bcc5 before the merge, at 511 registered tests.

The READER ANCHORS instrument in ltx2_video.cpp was ARMED rather than assumed
correct: inserting one line above kKnownLoadExtras shifted every derived anchor
by one and the gate went RED naming both lists. On the real tree recorded and
derived agree at [823 833 834 896 992 1008 1056 1147 1172 1277 1318 1360 1362],
because this row's edits are below the last anchor. The tree was restored
byte-for-byte after every mutation, with os.utime and a confirmed ninja
recompile each time; git status is clean at the pushed head.

Nine mutations, each printing four facts

git diff --stat, BUILT, the : error: count, and the exit code captured
directly — with the expected OLD line content asserted unique before every edit.

#	mutation	diff	built	errors	result
M1	recipe.image_conditioning assignment deleted	1 deletion	YES	0	video rc=1 (1 case), pipeline rc=1 (2 cases)
M2	reader forced to the REPLACE arm	1+/2-	YES	0	video rc=1 (1 case)
M3	audio_output_phase 1 -> 0	1+/1-	YES	0	pipeline rc=1 (2 cases)
M4	stage1.schedule_tokens deleted	1 deletion	YES	0	video rc=1, pipeline rc=1
M5	stage1.loras = kNoAdapters deleted	1 deletion	YES	0	pipeline rc=1
M6	requires_distilled_lora deleted	1 deletion	YES	0	video rc=1, pipeline rc=1
M7	the dispatch key made UNREACHABLE	1+/1-	YES	0	all six new cases RED, by name
M8	the COND arm left in velocity space	3+/1-	YES	0	video rc=1, 7 cases / 13 assertions
M9	the anchor case's gen.steps 3 -> 2	1+/1-	YES	0	video rc=1 (2 assertions)

M7 was run twice and the first run is recorded rather than dropped. Deleting
the whole dispatch arm left KeyframeInterpolationRecipe unreferenced and
-Werror killed the build — BUILT=NO, 1 error, no test result at all, which is
exactly the shape that reads as a passing test. Renaming the dispatch key keeps
every line compiled and referenced while making the recipe unselectable through
the request surface, which is what the reachability mutation is there to measure.

M9 is the mutation ltx25-ti2vid-recipe.md's first head passed. The step count
comes back OUT of the render lambda and the trajectory recomputation runs at it,
with rendered_steps > 2 asserted by name, because at two steps stretch pins
both non-zero sigmas and the schedule is {1, 0.1, 0} for every token count.
Measured here: keyframe: 4096 / 4096 res2s: 2 / 8.

What is NOT verified

No real-weights render. Upstream marks this arm Full + distilled LoRA
(packages/ltx-pipelines/CLAUDE.md:24), so stage 1's identity is CFG on the
UNADAPTED model and the checkpoint it needs is
ltx-2.5-22b-dev-transformer-bf16.safetensors. It is on the NAS and loadable.
What is owed is a GPU lease and the two renders; another agent holds dgx:gpu0
and no GPU work is in this row's scope. Running the arm against a distilled
checkpoint instead would be worse than not running it — the distilled scales are
trained into those weights, so a CFG-guided stage 1 samples a trajectory they
were never trained for and renders a plausible clip with no diagnostic (#1137).

The reach claim rests on the ABI path — LoadVideoEngine + Generate, which is
what ltx2-gen drives. #928 does not exclude the HTTP route here, because all
three knobs are LOAD extras and requires_audio_input is false; that is a claim
about the request surface, and no case here drives HTTP end to end.

Review repair

A fresh review of 251f76b5e returned eight findings. Six are repaired in
0e4357707; the other two are recorded rather than silently absorbed.

The blocking one was a bug this row introduced two hundred lines from the code
that carries it. The last-frame arm located its own appended tokens at
positions[target_tokens * 2], the first token past the fixed target grid --
which is its own token only while this arm owns the first append. This row put a
second appending item in front of it, so with both ends pinned the index named
the FIRST frame's keyframe at temporal 0 and the arm threw. That is
docs/USAGE.md's worked example for this kind and what ltx2-gen --help tells
the reader to do, so the documented headline command did not run.

Neither assertion is weakened. Each arm now captures the sequence length at the
moment of its own append and locates its tokens from that, so no arm depends on
being first. The generated-keyframe-slot arm carried the same derivation and is
repaired the same way.

Three guarantees the review found ungated are addressed, and one of the three
turned out to be a different problem than reported. frame_idx = 0 and the
trace's tail slice are now gated. causal_fix = true at that call site is
INERT rather than merely ungated, measured on a probe: at num_pixel_frames = 1,
which both production arms pass, flipping it moves 0 of 48 position values,
because the temporal start clamps to 0 either way and the num_pixel_frames == 1
narrow overwrites the end the fix moved. No call-site check can detect that flip.
The risk lives in the frame_idx == 0 gate the argument passes through, and that
is gated in test_ltx2_vae at num_pixel_frames != 1, where it shows.

Two records are corrected: ti2vid_two_stages.py:211 is blank and the real
combined_image_conditionings calls are :231 and :276; and the claim that
this row moved #1150's owed count from six to five was wrong on both numbers and
on the premise, since the keyframe arm was unported rather than divergent. Both
of those corrections stand.

Closes #1219.

#1220 is filed and NOT fixed: the two schedule-anchor cases return the request
step count where their comment claims to read the render. It still catches the
mutation it was built for, so it is a weakened guard rather than a vacuous one,
and re-deriving it changes what a landed case measures on both pipelines. Listed
under the row spec's ## Owed.

The scoped re-review: two off-by-N anchors, deferred whole

A scoped re-review of 87e9f0e37 returned two findings with one root cause. An
earlier shape of the repair corrected two inherited off-by-N upstream anchors on
the seven new lines that restated them. Those corrections are now REVERTED and
the whole question is deferred to
#1230.

Both readings are right, and that is not the point. Re-derived here at the
LTX-2 pin fd4ded7f by reading the pinned files rather than inheriting the
citation: latent_cond.py:38 is latent_state = latent_state.clone() and :39
is blank, so the two writes are :40-41; schedulers.py:31 is the return
annotation ) -> torch.FloatTensor:, so the
tokens = math.prod(latent.shape[2:]) read is :32. Correcting seven of the
twenty-two citations was still wrong, because a partial correction is strictly
worse than none.

A uniformly wrong anchor is one grep from being right, and a whole tree citing
:38-39 is a single mechanical edit for a single reviewer. A file citing BOTH
forms is not. src/vllm/multimodal/ltx2_video.cpp read latent_cond.py:38-39
at :2208 and :3174 and latent_cond.py:40-41 a hundred lines later at
:3392, with nothing in the tree recording which one to believe -- so the one
file a reader of this change opens was the one file stating both. The same split
ran through schedulers.py:31 in tests/vllm/multimodal/test_ltx2_video.cpp,
whose two near-identical schedule-anchor comments at :7890 and :8485
disagreed after the correction and agree again now.

It also cost a gate, which is the blocking half.
include/vllm/model_executor/models/ltx2_pipeline.h:754 was one of the seven,
and USER_USAGE_PREFIXES in scripts/check-doc-checkpoint.py:99 is a pure path
match on include/vllm/ with no content analysis. A one-line comment edit in a
public header therefore reads as a usage change and demands a docs/USAGE.md
edit. .github/workflows/ci.yml:448 runs that gate per commit over the range, so
no follow-up commit could clear it -- the commit itself had to change. There is
no user-visible usage change here, so writing a docs/USAGE.md edit to turn the
gate green is the move AGENTS.md forbids. Removing the header edit is the honest
fix, and it retires the mixed file in the same stroke. The repair commit no
longer touches include/ at all.

Measured, on the rewritten repair commit 263f82b67:

range	check-doc-checkpoint.py
--base 5af6e7631 --head 87e9f0e37	exit 1, commit 0e4357707: changed user_usage but did not update docs/USAGE.md
--base 5af6e7631 --head 853384a8b	exit 0, OK: public documents match the claims this change makes

The old head is kept as the control so the instrument is shown armed rather than
assumed: the checker still reds on the pre-rewrite range from the same working
tree that returns 0 on the new one. It is also 0 against today's origin/main.

The RECORDS go the other way on purpose and are not reverted. The spec's
port-map table and #1219's index row state :40-41 and :32, because a record's
job is to say what is true, and the index is append-only, so a wrong anchor
written there could never be swept. What the record now says, and what #1230
carries, is: the anchors are :40-41 and :32, the source cites them uniformly
short in twenty-two places, and one row corrects all of them at once. The tree
already carried eight citations of the CORRECT form before this row existed --
include/vllm/multimodal/ltx2_video.h:625 among them -- so the mixture is older
than this change and outlives it either way.

#1230 is filed, listed under ## Owed in the row spec and appended to the issue
index, and stays OPEN.

Gate rerun by the operator

Rerun at 87e9f0e37, not taken from the implementer's report.
CONFIGURE_EXIT=0, BUILD_EXIT=0, : error: 0, no No space left.
ctest -N = 516 and ctest -j3 = 100% tests passed, 0 tests failed out
of 516, CTEST_EXIT=0.

Focused, with ANSI stripped and Status: read rather than inferred from the exit
code: test_ltx2_pipeline 56 cases / 3316 assertions, test_ltx2_video 88 /
2755, test_ltx2_vae 43 / 3125, all Status: SUCCESS! at exit 0. The video
case count moved from the reviewed head's 87 / 2713, which is what proves the new
both-keyframes case ran rather than matching nothing -- a doctest filter that
selects zero cases prints SUCCESS! and exits 0.

windows-msvc-cpu and windows-msvc-vulkan are red. They are red on every PR in
this repo -- confirmed on the unrelated #1186 and #1178 -- because those jobs are
PR-only with no main baseline. Not attributable to this change.

Gate rerun after the anchor revert

Rerun by the repairing session at 853384a8b, on the merged tree, in its own
worktree. CONFIGURE_EXIT=0, BUILD_EXIT=0, : error: count 0, No space left and BFD each 0.
ctest -N = 516 and ctest -j3 = 100% tests passed, 0 tests failed out of 516, CTEST_EXIT=0.

Focused, whole binaries with no --test-case filter, ANSI stripped and Status:
read rather than inferred from the exit code:

Binary	cases	assertions	Status:	exit
test_ltx2_pipeline	56	3316	SUCCESS!	0
test_ltx2_video	88	2755	SUCCESS!	0
test_ltx2_vae	43	3125	SUCCESS!	0

Unchanged from the operator's rerun at 87e9f0e37, which is the expected result:
the revert touches eight comment lines and no executable statement.
test_ltx2_video holding at 88 rather than falling back to 87 is the
load-bearing one, because a lost case is how a history rewrite drops work
silently.

Record gates on the rewritten range: check-commit-trailers OK,
check-commit-style OK, check-issue-index-append-only OK, check-now-current
OK, check-agent-record OK.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]