Skip to content

LTX-2.5: A2VidPipelineTwoStage has no recipe row, so audio-to-video rides the distilled two-stage trajectory #1117

Description

@localai-bot

A2VidPipelineTwoStage (a2vid_two_stage.py:53 @ fd4ded7f) has no recipe row. Its audio conditioning landed with #922 and is reachable today through the audio_path per-generation extra; its recipe — the two-stage schedule, the guiders and the required audio input — does not exist, so pipeline_kind = a2vid_two_stage gets the generic table refusal at src/vllm/model_executor/models/ltx2_pipeline.cpp:1328-1332 naming the pair rather than the missing machinery. Measured at daeff67f2: git grep -n '"a2vid' over src include tests docs examples returns TWO hits, both upstream anchors inside Fail message assertions (tests/vllm/multimodal/test_ltx2_video.cpp:4363,:4427), against a control of 4 hits for "one_stage" in include/ alone. A closed conditioning issue is not a recipe.

What a supplied take rides today is distilled_two_stage, which is a DIFFERENT trajectory and the a2v spec says so under ## Owed (ltx25-a2v-audio-input.md:438-446). Four differences, each read at the pin:

  1. Stage 1 is guided and caller-configured. GuidedDenoiser with MultiModalGuider(params=video_guider_params, negative_context=v_context_n) (a2vid_two_stage.py:230-240), whose six fields come from the params table through the CLI defaults (utils/args.py:947-996, --a2v-guidance-scale defaulting to video_guider.modality_scale at :987-996, which is 3.0 at utils/constants.py:54,:64). distilled_two_stage fixes allow_guidance_override = false and carries the default positive-only guider on both phases.
  2. Stage 1's schedule is scheduler-derived, self._scheduler.execute(steps=num_inference_steps) (:225-227), against our fixed DistilledSigmas() (ltx2_pipeline.cpp:1163).
  3. Stage 1 is plain Euler. a2vid_two_stage.py:229-258 passes no stepper, so DiffusionStage.__call__'s EulerDiffusionStep() default applies (utils/blocks.py:524-527); our distilled_two_stage selects kEulerAncestral for stage 1 on generation 2.5 (distilled.py:76-84).
  4. The audio stream's guider is the DEFAULT one, MultiModalGuiderParams() at :237-239 — cfg 1.0 / stg 0.0 / modality 1.0 (guiders.py:199-211) — and NOT the params table's audio row, which carries cfg 7.0.

Two things that are not schedule and must not be guessed:

  • The audio file is REQUIRED. --audio-path is required=True (:312-317). Nothing in this tree makes a pipeline_kind demand a per-generation extra today, so an a2vid load with no take renders unconditioned video and looks like the feature working.
  • The distilled LoRA rides stage 2 ALONE. stage_2_loras = (*loras, *distilled_lora) at :114, against loras=tuple(loras) for stage 1 at :107; --distilled-lora is required=True (utils/args.py:1140-1153). This engine fuses at LOAD into ONE weight set (src/vllm/multimodal/ltx2_video.cpp:816-820, the only dit_options.loras.push_back in the tree), so the per-phase adapter cannot be expressed. Same seam blocks #1093 (ti2vid_two_stages.py:140,:151) and #921 (ti2vid_two_stages_hq.py:154,:165, two strengths). distilled.py:131 builds ONE stage set, which is why no shipped recipe has needed it until now.

The blocker this row waited on is gone: Ltx2GuidedDenoise landed at daeff67f2 (#1092/#1102) with four production callers in src/vllm/multimodal/ltx2_video.cpp, and .agents/specs/ltx25-guided-video.md §1 lists a2vid_two_stage under ## Owed as "its own row; none is blocked on this seam any more". ltx-2.5-22b-distilled-lora-450-bf16.safetensors — the artifact #1093's control could not find — is now on the NAS.

Scope of this row: the recipe row, the required-audio refusal, the required-LoRA refusal mirroring --distilled-lora, and a gate that the frozen take is CONSUMED on the a2vid arm rather than merely carried. NOT in scope and owed: the per-phase adapter, and any real-checkpoint render (the GPU is out of bounds for this row).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions