Repository navigation
Wan I2V expand_timesteps: why hard-clamp only first frame (no last-frame clamp)? #13167
Description
I have a question on expand_timesteps option within WanImageToVideoPipeline. I see that the mechanism hard-clamps only the first frame. For FLF (first+last) conditioning, is there a reason we can’t (or shouldn’t) apply the same mechanism to both endpoints?
The related code is in:
diffusers/src/diffusers/pipelines/wan/pipeline_wan_i2v.py
In prepare_latents, when expand_timesteps is enabled, video_condition ignores last_image and first_frame_mask is defined only for frame 0:
# prepare_latents
if self.config.expand_timesteps:
video_condition = image
...
if self.config.expand_timesteps:
first_frame_mask = torch.ones(1, 1, num_latent_frames, latent_height, latent_width, ...)
first_frame_mask[:, :, 0] = 0
return latents, latent_condition, first_frame_maskIn the denoising loop, the clamp/mix uses only first_frame_mask:
if self.config.expand_timesteps:
latent_model_input = (1 - first_frame_mask) * condition + first_frame_mask * latents
temp_ts = (first_frame_mask[0][0][:, ::2, ::2] * t).flatten()Question
Is it a limitation of the Wan2.2 I2V checkpoint training (only first-frame conditioning)?
Or a design choice because per-token timestep masking doesn’t extend cleanly to a last-frame constraint?
I see that the corresponding part was written by @yiyixuxu, thanks!
Activity
chinmayirm commentedon Sep 22, 2026
Hi @emirks, the first-frame-only clamp is what that branch is doing, and the last frame is handled on the other path.
Root cause
prepare_latentsforks onself.config.expand_timesteps.When it is true,
video_conditionis onlyimage,last_imageis not encoded, andfirst_frame_mask[:, :, 0] = 0. The denoising loop then buildslatent_model_inputfrom that mask and sets per-token timesteps withtemp_ts = (first_frame_mask[0][0][:, ::2, ::2] * t). Frame 0 stays at timestep 0. Every other frame, including the last, stays att. That is the image-to-video schedule this checkpoint was trained with.When it is false and
last_imageis set, the condition is[first, zeros, last]and the mask keeps both endpoints. First-and-last-frame conditioning already goes through that branch.Clamping the last frame inside
expand_timestepswould change the timestep layout of a model that was not trained that way. The two schedules are not interchangeable.Proposed approach
Leave
expand_timestepsas first-frame-only. A docstring onprepare_latentsshould say that this flag ignoreslast_image, and that first-and-last-frame conditioning is theexpand_timesteps=Falsepath. I would not add a last-frame clamp to this branch.I would be glad to open a PR for that docstring if it would help.