Skip to content

feat: Video generation - #9163

Merged
lstein merged 213 commits into
invoke-ai:mainfrom
lstein:lstein/feature/wan-video-support
Jul 28, 2026
Merged

feat: Video generation#9163
lstein merged 213 commits into
invoke-ai:mainfrom
lstein:lstein/feature/wan-video-support

Conversation

@lstein

@lstein lstein commented May 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

This PR adds basic support for AI-based video generation using the Wan 2.2 family of text-to-video and image-to-video models. Currently it can only be used through the workflow editor.

invoke-videos.mp4

This PR adds the following support:

  • A new Video type, with support for upload, thumbnail generation, and metadata storage.
  • New movie player functionality in the viewer.
  • A Wan Denoise Video node for denoising Wan 5D tensors into latents
  • A Latents To Video node for decoding the denoiser latents
  • A few image nodes for selecting frames from videos and concatenating videos together
  • Template workflows that show typical text-to-video and image-to-video graphs.

Important notes

Testing plan

  • Smoke-test T2V Lightning (832×480, 81 frames, 4 steps, CFG 1.0)
  • Smoke-test I2V Lightning with a 16:9 source image
  • Verify auto-switch to new video on completion, full-resolution first-frame preview, inline playback
  • Upload an MP4 from disk; verify it appears in the gallery and plays
  • Drag a gallery video onto a Video Primitive; verify the field populates
  • Exercise the video concatenate node by joining two videos using the various transition options
  • Run on Windows / macOS to confirm the imageio FFmpeg path works on those platforms

Getting Started Hints

Run uv pip install to pick up the new dependency on the ffmpeg library, and of course rebuild the front end.

Install the following from the starter model collection:

  • Wan 2.2 TI2V-5B (Q4_K_M) -- a very small, low-quality video generator suitable for 12 GB VRAM or less

  • Wan 2.2 T2V A14B High Noise (Q4_K_M) - text-to-video transformer, rough phase

  • Wan 2.2 T2V A14B Low Noise (Q4_K_M) - text-to-video transformer, refiner phase

  • Wan 2.2 T2V Lightning High Noise (4-step, V1.1) - turbo LoRA, rough phase

  • Wan 2.2 T2V Lightning Low Noise (4-step, V1.1) - turbo LoRA, refiner phase

The corresponding models for image-to-video are:

  • Wan 2.2 I2V A14B High Noise (Q4_K_M)
  • Wan 2.2 I2V A14B Low Noise (Q4_K_M)
  • Wan 2.2 I2V Lightning High Noise (4-step, V1.1)
  • Wan 2.2 I2V Lightning Low Noise (4-step, V1.1)

The encoder and VAE should download as dependencies.

Give it a spin! There are several working templates in the Workflow library that you can start with:

  • Text to Video - Wan 2.2 -- basic text to video. Select the TI2V-5B model if you are low on VRAM and leave "Transformer (Low Noise) empty. If you have >=16 GB, you can use the high-quality A14B models, apply the high noise transformer model to the Main Model "Transformer" field, and the low noise transformer model to the "Transformer (Low Noise)" field. Select the models you downloaded for the standalone VAE and T5 Encoder fields. Type in a prompt, and use the default values for CFG, image dimensions, frames and steps.
  • Text to Video - Wan 2.2 Lightning -- This is the same as above, but has loaders for the high and low noise lightning LoRAs which will reduce the number of required steps to 4.
  • Image to Video - Wan 2.2 -- basic reference image to video. You need to provide a reference image and a prompt describing what to do with it. The reference image will be the first frame of the image. The image should have the same aspect ratio as the desired video, but doesn't need to be exactly the same dimensions.
  • Image to Video - Wan 2.2 Lightning -- as above, but with nodes for the two lightning LoRAs for dramatic speedups.

The models work best across a limited series of dimensions. The ones I've tested are:

  • 720 x 480
  • 480 x 720
  • 832 x 480
  • 480 x 832

On my RTX 5060Ti it takes 3-4 minutes to generate a video when using the two Lightning LoRAs.

If you have lots of VRAM you can try increasing the frame count, but these videos get big fast. Alternatively, you can create a workflow that captures the last frame of the original video, generates a new video on top of it, and concatenates the two together. I've got a workflow that works well for this, but haven't added it to the branch yet.

🤖 Generated with help from Claude Code

lstein and others added 30 commits May 9, 2026 10:33
@lstein @claude
feat(model): add Wan 2.2 image generation support (Phases 0-2)
5725f7a 
Foundation + TI2V-5B MVP + A14B dual-expert MoE for Wan 2.2 image
generation. Wan was trained on video but is competitive with leading
open-source image models when run at num_frames=1; this commit wires
that path into InvokeAI.

Phase 0 — Foundation:
- BaseModelType.Wan + WanVariantType {T2V_A14B, TI2V_5B}
- SubModelType.Transformer2 for the dual-expert MoE
- MainModelDefaultSettings per variant
- step_callback Wan branch (16-channel preview; 48-channel TI2V-5B
  falls back to slicing first 16 channels until proper factors land)
- Frontend enums + node colour

Phase 1 — TI2V-5B Diffusers MVP:
- Main_Diffusers_Wan_Config probe (variant from transformer_2/ +
  vae/config.json::z_dim, with filename heuristic fallback)
- WanDiffusersModel loader (subclasses GenericDiffusersLoader)
- WanT5EncoderField, WanTransformerField (with dual-expert slots),
  WanConditioningField, WanConditioningInfo
- New invocations: wan_model_loader, wan_text_encoder, wan_denoise,
  wan_image_to_latents, wan_latents_to_image
- FlowMatchEulerDiscreteScheduler integration with on-disk config load
- RectifiedFlowInpaintExtension reused for inpaint
- 5D <-> 4D shape juggling: latents stay 4D in InvokeAI's pipeline,
  re-add T=1 only inside the transformer call / VAE encode-decode

Phase 2 — A14B dual-expert MoE:
- Probe reads boundary_ratio from model_index.json
- Loader emits both transformer (high-noise) and transformer_low_noise
  (low-noise expert at transformer_2/) for A14B
- _ExpertSwapper in wan_denoise drives GPU residency between experts:
  high-noise for t >= boundary_ratio * num_train_timesteps, low-noise
  below. Only one expert locked at a time so the cache can evict the
  other - relies on existing CachedModelWithPartialLoad to handle
  oversized models on lower-VRAM GPUs.
- guidance_scale_low_noise field for separate low-noise CFG override

Tests:
- 24 passing tests covering probe variant detection, default settings,
  noise sampling, end-to-end denoise on a synthetic transformer (CPU),
  dual-expert boundary swap, CFG branch
- 1 heavy-test placeholder gated by INVOKEAI_HEAVY_TESTS=1 for the
  real-weights smoke test

Phase 3+ deferred: standalone VAE/encoder configs, GGUF, LoRA,
ControlNet, ref image, inpaint UI, frontend wiring, starter models.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(model): Wan 2.2 Phase 3 + tokenizer-load fix
ad4a31e 
Phase 3 adds standalone VAE and UMT5-XXL encoder configs so users can run
GGUF-quantized Wan transformers (Phase 4) without installing the full
~30 GB Diffusers pipeline.

VAE configs:
- VAE_Checkpoint_Wan_Config + VAE_Diffusers_Wan_Config (16-channel A14B
  vs 48-channel TI2V-5B, distinguished by decoder.conv_in z_dim).
- 16-channel files share the AutoencoderKLWan architecture with Qwen
  Image; disambiguated via filename heuristic ("wan" in name -> Wan,
  otherwise -> Qwen Image). Mirror exclusion in QwenImage's probe.
- VAELoader gets a Wan branch that builds AutoencoderKLWan(z_dim=...)
  via init_empty_weights, mirroring the QwenImage single-file pattern.
- Existing standard VAE probe excludes both QwenImage- and Wan-style
  state dicts.

UMT5-XXL encoder:
- New ModelType.WanT5Encoder + ModelFormat.WanT5Encoder.
- WanT5Encoder_WanT5Encoder_Config probes the diffusers folder layout
  (text_encoder/config.json with model_type=umt5, or flat layout with
  config.json at root). Refuses full Wan pipelines.
- WanT5EncoderLoader handles both layouts and loads UMT5EncoderModel +
  AutoTokenizer.

Component-source plumbing:
- WanModelLoaderInvocation now exposes wan_t5_encoder_model and
  component_source pickers (mirrors QwenImage pattern). Resolution
  order: standalone > main (if Diffusers) > component_source. Required
  when the main model is a single-file format in Phase 4.

Bug fix in wan_text_encoder:
- Tokenizer was loading via AutoTokenizer.from_pretrained(<root>)
  directly, which fails for nested layouts where files live in
  <root>/tokenizer/. Now routed through the model cache so the
  registered loaders handle layout differences correctly.

Frontend:
- New type guards (isWanVAEModelConfig, isWanT5EncoderModelConfig,
  isWanMainModelConfig, isWanDiffusersMainModelConfig) and hooks/
  selectors (useWanVAEModels, useWanT5EncoderModels,
  useWanDiffusersModels). New zSubModelType / zModelType / zModelFormat
  enum entries for transformer_2 and wan_t5_encoder.

Tests:
- 16 new tests covering z_dim detection, VAE checkpoint/diffusers
  probes, the bidirectional Qwen-vs-Wan filename deferral, and the
  UMT5 encoder probe (nested + flat + T5 + full-pipeline rejection).
- Total Wan test count: 41 passing, 1 heavy-test placeholder skipped.
- Full config test suite (63 tests) still passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): unbreak frontend lint after Wan additions

Five issues turned up running `make frontend-lint`:

1. wan_denoise.py used `from __future__ import annotations`, which made
   the `invoke()` return annotation a string ('LatentsOutput'). The
   InvocationRegistry's `get_output_annotation()` returns the raw
   annotation, so OpenAPI generation crashed with
   `'str' object has no attribute '__name__'`. Removed the future-import
   and added `Any` to the typing imports.

2. ModelRecordChanges.variant didn't list WanVariantType, so the
   generated schema's install/update endpoints rejected `t2v_a14b` and
   `ti2v_5b`. Added it.

3. Regenerated frontend/web/src/services/api/schema.ts from the live
   backend so it now includes BaseModelType.wan, ModelType.wan_t5_encoder,
   SubModelType.transformer_2, ModelFormat.wan_t5_encoder, the Wan
   variants, all Wan invocation types and their conditioning/transformer
   field types.

4. modelManagerV2/models.ts: added `wan_t5_encoder` to the category map,
   `wan` to the base color/long-name/short-name maps, the two Wan
   variants to the variant-name map, and `wan_t5_encoder` to the
   format-name map.

5. ModelManagerPanel/ModelFormatBadge.tsx: added `wan_t5_encoder` to
   FORMAT_NAME_MAP and FORMAT_COLOR_MAP.

`make frontend-lint` now passes cleanly (tsc, dpdm, eslint, prettier).
All 41 Wan Python tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

chore(wan): drop unused FE exports flagged by knip

These were forward-compatibility wiring for Phase 9 (the FE graph
builder) that has no consumers yet; knip rightly flagged them. Removed
or de-exported. They'll come back when the graph builder lands and
needs them.

- common.ts: zWanVariantType drops `export` (still used internally by
  zAnyModelVariant).
- types.ts: drop isWanMainModelConfig, isWanDiffusersMainModelConfig,
  isWanVAEModelConfig (no callers). The remaining
  isWanT5EncoderModelConfig is used by models.ts. WanT5EncoderModelConfig
  type drops `export` (still used as the type guard's narrowing target).
- modelsByType.ts: drop the six unused useWan*/selectWan* hooks +
  selectors and their type-guard imports.

`make frontend-lint` (tsc + dpdm + eslint + prettier + knip) now green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

docs(wan): use *-Diffusers HF repo names in plan

The Wan-AI org publishes two flavours of each release:
  * Wan-AI/Wan2.2-{TI2V-5B,T2V-A14B,I2V-A14B}            ← upstream native
  * Wan-AI/Wan2.2-{TI2V-5B,T2V-A14B,I2V-A14B}-Diffusers  ← convertible

The native release has _class_name=WanModel in config.json and ships
weights flat at the repo root with no transformer/, vae/, text_encoder/
subdirs. It is not loadable by Diffusers' WanPipeline.from_pretrained.

Update plan doc to reference the -Diffusers repos throughout (probe
notes, starter-model entries) so the plumbing path matches what the
Diffusers loader actually expects.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): accept 0 as 'unset' sentinel for guidance_scale_low_noise

The frontend renders Optional[float] inputs with default 0 in the
numeric input rather than passing null/unset. Combined with ge=1.0,
this caused every wan_denoise invocation to fail Pydantic validation
with "Input should be greater than or equal to 1" until the user
manually entered a value (or knew to leave the field disconnected).

The validation error was rejected before invocation logging, so it
never showed up in the server log either - making the failure hard to
diagnose.

Relaxing the constraint to ge=0.0 and treating values below 1.0 as the
"fall back to primary Guidance Scale" sentinel. The user's natural FE
default (0) now works as expected.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): correct preview dimensions and colors for TI2V-5B

Two bugs in the Wan branch of the diffusion step callback:

1. Wrong dimensions. The reported preview size hardcoded `* 8` for the
   spatial downscale ratio, but TI2V-5B's Wan2.2-VAE uses 16x. A
   1024x1024 target was being announced to the FE as 512x512.

2. Wrong colors. The previous fallback for 48-channel TI2V-5B latents
   sliced the first 16 channels and applied the standard 16-channel
   Wan-VAE projection. Those channel layouts are unrelated, so the
   projection produced meaningless colors.

Adding the proper Wan2.2-VAE 48-channel RGB projection matrix (and
bias) from ComfyUI's Wan22 latent format, and selecting the right
matrix + spatial-scale by latent channel count: 16 → A14B (Wan VAE,
8x), 48 → TI2V-5B (Wan2.2-VAE, 16x).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): honor model's _class_name when building scheduler

TI2V-5B's scheduler_config.json declares _class_name=UniPCMultistepScheduler
with flow_shift=5.0. The previous code hardcoded
FlowMatchEulerDiscreteScheduler.from_pretrained(...), which silently
constructed a default-config FlowMatch instead of the UniPC the model
expects. The mismatched noise schedule manifests as soft / under-denoised
faces and global graininess in the final images.

Now: read scheduler_config.json, look up the named class on the diffusers
module, and instantiate that class via from_pretrained. UniPC and
FlowMatch share the same step()/set_timesteps()/sigmas/num_train_timesteps
interfaces, so the denoise loop works transparently for either.

A14B continues to use FlowMatchEulerDiscreteScheduler when its scheduler
config says so (its reference is FlowMatchEuler with shift=8.0). Falls
back to FlowMatchEulerDiscreteScheduler defaults when no on-disk config
is available.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): match diffusers WanPipeline tokenizer length and latent dtype

Two divergences from the Diffusers reference that were hurting image
quality (soft / grainy / distorted faces at default settings):

1. Tokenizer max_sequence_length was 226 in wan_text_encoder, but the
   model was trained with 512-token sequences. The upstream native
   config.json has text_len: 512, and Diffusers' WanPipeline.__call__
   default is 512 (overriding _get_t5_prompt_embeds's stale 226 default).
   Wan's cross-attention sees padded zeros past the prompt's actual
   length but expects to be looking at a 512-position context window.

2. Latents were stored in bf16 throughout the denoise loop. Diffusers'
   WanPipeline.prepare_latents explicitly uses dtype=torch.float32 and
   only casts to the transformer's dtype right at the forward call:
       latent_model_input = latents.to(transformer_dtype)
   Storing in bf16 between steps accumulates ~40 steps of bf16
   quantization on the scheduler's small per-step deltas. Now
   latent_dtype = torch.float32 throughout, with a per-step cast for
   the transformer forward pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

chore(wan): add diffusers reference comparison script

scripts/wan_diffusers_reference.py runs a Diffusers-format Wan 2.2
checkpoint directly via WanPipeline.from_pretrained, with the same
arguments InvokeAI's wan_denoise uses. Use to A/B against InvokeAI
output when image quality is questionable.

Defaults to enable_model_cpu_offload so the script fits on 16 GB cards
where the full pipeline (transformer + UMT5-XXL + VAE) would otherwise
OOM. --offload {model,sequential,none} controls the strategy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(model): Wan 2.2 Phase 4 - GGUF transformer support
f9a22c1 
Adds single-file GGUF support for Wan 2.2 transformers, the path that
makes A14B usable on consumer GPUs (~7 GB/expert at Q4_K_M instead of
~28 GB at bf16).

Probe (configs/main.py):
- New helpers: _has_wan_keys (Wan vs Qwen/FLUX/Z-Image fingerprint via
  condition_embedder.text_embedder.linear_1 + patch_embedding);
  _detect_wan_gguf_variant (16ch -> A14B, 48ch -> TI2V-5B from
  patch_embedding.weight.shape[1]); _detect_wan_gguf_expert (filename
  heuristic for high_noise / low_noise / none).
- Main_GGUF_Wan_Config(base=Wan, format=GGUFQuantized, variant, expert).
  Tolerates the ComfyUI 'model.diffusion_model.' / 'diffusion_model.'
  prefixes via _has_wan_keys' multi-prefix scan.
- Registered in factory.py.

Loader (model_loaders/wan.py):
- WanGGUFCheckpointModel mirrors the QwenImage GGUF pattern:
  gguf_sd_loader -> strip ComfyUI prefix -> auto-detect arch from state
  dict shapes (num_layers, inner_dim, ffn_dim, text_dim, in_channels,
  num_heads = inner_dim/128) -> init_empty_weights +
  load_state_dict(strict=False, assign=True).

Loader invocation (wan_model_loader.py):
- New 'Transformer (Low Noise)' picker: optional second GGUF for the
  A14B dual-expert MoE. Auto-swaps if the user wired the experts in
  the wrong order. Warns when an A14B GGUF is loaded without a paired
  low-noise expert (single-expert run, degraded quality).
- GGUF mains require either a standalone VAE+encoder or a Diffusers
  Component Source (which can also supply boundary_ratio).
- Diffusers main path unchanged (still pulls both experts from
  transformer/ + transformer_2/).

Tests (tests/.../test_wan_gguf_config.py):
- 14 tests across key fingerprint, variant detection, expert filename
  heuristic, and the full probe (A14B high/low, TI2V-5B, GGUF rejection,
  unrecognised state-dict rejection, explicit override).

Total Wan tests: 55 passing (no regressions). FE lint clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): support QuantStack-style GGUFs and standalone Diffusers VAE

The city96 Wan 2.2 GGUF repos have been removed from Hugging Face,
leaving QuantStack as the surviving distributor. QuantStack ships the
native upstream Wan key layout (text_embedding.0/2, self_attn/cross_attn,
ffn.0/2, head.head, head.modulation, ...) rather than the diffusers
naming city96 used; biases are stored as F16 rather than BF16; and the
standalone Wan VAE installs as a flat AutoencoderKLWan folder which the
generic loader rejects. Three fixes:

1. Probe now recognises both diffusers and native key layouts via a new
   _is_native_wan_layout helper; _has_wan_keys accepts either text-proj
   fingerprint.

2. GGUF loader converts native -> diffusers keys (mirroring diffusers'
   convert_wan_transformer_to_diffusers) and unwraps non-quantized
   GGMLTensors to plain tensors at compute_dtype. The unwrap is needed
   because conv3d isn't in GGMLTensor's dispatch table, so the F16
   patch_embedding bias would otherwise hit conv3d against bf16 latents.

3. VAELoader gains a VAE_Diffusers_Wan_Config branch that loads
   AutoencoderKLWan directly; the generic path can't handle a flat
   single-class folder when a submodel_type is provided.

Adds 12 tests covering the native layout (probe + converter + unwrap).
Verified end-to-end against Wan2.2-T2V-A14B-Q4_K_M from QuantStack:
1095 tensors round-trip key-for-key against WanTransformer3DModel.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(model): Wan 2.2 Phase 5 - LoRA support
11b7eb3 
Probe + config (LoRA_LyCORIS_Wan_Config):
  - Detects Wan LoRAs in three layouts: diffusers PEFT, native upstream PEFT
    (ComfyUI), and Kohya (both naming variants).
  - Anti-pattern guards prevent collisions with Anima (Cosmos DiT q_proj
    convention), QwenImage (transformer_blocks), Flux (double/single blocks),
    and Z-Image (diffusion_model.layers).
  - Optional ``expert: "high" | "low" | None`` field; auto-detected from
    filename (high_noise / low_noise / hyphenated / concatenated variants).

Key conversion (wan_lora_conversion_utils):
  - Native upstream keys (self_attn/cross_attn, ffn.0/2) -> diffusers
    (attn1/attn2, ffn.net.0.proj / ffn.net.2).
  - Strips ``transformer.``, ``diffusion_model.``, ``base_model.model.transformer.``
    prefixes from PEFT-style keys.
  - Kohya layer names mapped through an explicit longest-match table.
  - Output paths use diffusers naming so the LayerPatcher can resolve them
    against WanTransformer3DModel parameter paths.

Loader integration:
  - Adds BaseModelType.Wan branch to LoRALoader._load_model.

Invocation nodes (wan_lora_loader.py):
  - WanLoRALoaderInvocation: single LoRA with auto/both/high/low target field.
  - WanLoRACollectionLoader: list of LoRAs, auto-routed by each LoRA's
    recorded expert tag.
  - Output WanLoRALoaderOutput carries the WanTransformerField with updated
    ``loras`` / ``loras_low_noise`` lists.

Denoise integration:
  - _ExpertSwapper now manages both the model_on_device context and the
    LayerPatcher.apply_smart_model_patches context per expert. LoRA patches
    are entered after device load and exited before device release, with
    fresh iterators per swap.
  - GGUF (quantized) experts request sidecar patching so GGMLTensor weights
    aren't touched directly.
  - Low-noise expert falls back to the primary loras list when
    ``loras_low_noise`` is empty (matches WanTransformerField semantics).

Tests: 81 new tests covering probe accept/reject across formats, anti-pattern
guards on competing architectures, converter round-trips for all three
layouts, invocation target resolution + routing + duplicate guards, and the
_ExpertSwapper lifecycle (lora context opens/closes in the right order
around the device swap, quantized flag forwards, no-LoRA path skips the
patch context, re-entering the same label is a no-op).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): probe Wan LoRA before Anima in the config union

Native-PEFT Wan LoRAs (lightx2v's Lightning, most ComfyUI-trained Wan
LoRAs) carry keys like ``diffusion_model.blocks.X.cross_attn.k.lora_A.weight``.
Anima's probe matches on the bare ``cross_attn``/``self_attn`` substring —
it does not require the Anima-specific ``_proj`` suffix nor any of the
``mlp``/``adaln_modulation`` Cosmos DiT markers — so these Wan LoRAs were
classified as ``BaseModelType.Anima`` because Anima happened to run first.

Reorder the LyCORIS section of ``AnyModelConfig`` so Wan probes first.
Wan's probe is strictly more restrictive (it rejects Anima's ``_proj``
attention suffix via the anti-pattern guard added in the previous commit),
so Anima LoRAs are still correctly classified after this reorder.

Existing users with mis-tagged installs need to delete the affected LoRA
records and reinstall.

Adds two regression tests: a union-ordering assertion, and a sanity check
that demonstrates Anima's probe *would* match Wan native keys if asked
directly — pinning the constraint that motivates the ordering.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

chore(i18n): add Wan2.2 T5 Encoder model-manager label

The frontend source already references ``modelManager.wanT5Encoder``;
the locale key was added with a casing typo (``want5Encoder``). Fix
the key so the Wan T5 Encoder model type renders its display name
correctly in the model manager UI.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(model): Wan 2.2 Phase 7 - reference-image (I2V) conditioning
22799af 
Re-implementation after the first attempt — which used CLIP-vision
conditioning — was reverted. Wan 2.2 I2V-A14B does NOT use a CLIP-vision
encoder (the Diffusers repo ships ``image_encoder: [null, null]`` in
``model_index.json``); instead it conditions on a reference image by
VAE-encoding it and concatenating the resulting latents (plus a
first-frame mask) to the noise latents along the channel dim. The I2V
transformer therefore has ``in_channels=36`` (16 noise + 16 ref-image
latents + 4 mask) vs ``in_channels=16`` for T2V.

Taxonomy:
  - Re-adds ``WanVariantType.I2V_A14B``.

Probes:
  - Diffusers: ``_detect_wan_variant`` reads ``transformer/config.json::in_channels``;
    36 → I2V_A14B, 16 → T2V_A14B (both share the dual-expert layout).
  - GGUF: ``_detect_wan_gguf_variant`` recognises ``in_channels=36`` from the
    patch_embedding tensor shape and emits I2V_A14B.

Backend extension (``backend/wan/extensions/wan_ref_image_extension.py``):
  - ``preprocess_reference_image`` resizes + normalises to a 5D pixel tensor.
  - ``encode_reference_image_to_condition`` VAE-encodes the image and stacks
    a 4-channel first-frame mask on top, producing the
    ``[1, 20, 1, H/8, W/8]`` condition tensor the denoise loop consumes.
  - Mirrors diffusers ``WanImageToVideoPipeline.prepare_latents`` with
    ``num_frames=1`` and ``expand_timesteps=False``.

Invocation node (``wan_ref_image_encoder.py``):
  - "Reference Image - Wan 2.2": image + VAE + width/height pickers.
  - Output ``WanRefImageConditioningField`` carries the condition tensor
    name plus the dimensions used (so the denoise step can validate dim
    parity).

Denoise integration:
  - ``WanDenoiseInvocation`` gains an optional ``ref_image`` field.
  - Variant gate: rejects ref_image on T2V_A14B and TI2V-5B with a clear
    error before doing any work.
  - Dimension gate: rejects ref-image width/height mismatch vs denoise.
  - At every transformer call, concatenates the 20-channel condition
    tensor to the 16-channel noise latents along the channel dim before
    passing to the transformer (giving the 36-channel input I2V expects).

Tests: 14 new across the probe, the extension, and the denoise loop.
The synthetic ``_ZeroTransformer`` test stand-in now mirrors the real
I2V transformer's ``in_channels=36, out_channels=16`` asymmetry by
slicing its zero output back to 16 channels when the input is 36-wide.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): derive GGUF out_channels from proj_out shape (I2V support)

The GGUF loader was setting ``out_channels = in_channels`` which is wrong for
Wan 2.2 I2V-A14B: that variant has ``in_channels=36`` (16 noise + 16 ref-image
latents + 4 first-frame mask, concatenated by the denoise loop) but
``out_channels=16`` since the transformer only predicts the noise component
back. Loading an I2V GGUF would build a transformer with the wrong proj_out
shape and crash:

  RuntimeError: Error(s) in loading state_dict for WanTransformer3DModel:
    size mismatch for proj_out.weight: copying a param with shape
    torch.Size([64, 5120]) from checkpoint, the shape in current model is
    torch.Size([144, 5120]).

(144 = 36 * 4, 64 = 16 * 4 — patch_size=(1, 2, 2) → prod=4)

Read out_channels directly from the ``proj_out.weight`` shape in the state
dict. This is correct for all three Wan 2.2 variants without needing to know
the variant in advance.

Also tighten the num_layers fallback: T2V_A14B and I2V_A14B share 40 layers;
only TI2V-5B has 30. The fallback is rarely hit in practice (the per-block
count comes from the state dict scan), but the previous code would have
defaulted I2V_A14B to 30 layers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(model): make Anima LoRA probe mutually exclusive with Wan

InvokeAI's ``Config_Base.CONFIG_CLASSES`` is a Python ``set``, so iteration
order during model probing is non-deterministic across process restarts.
First-match-wins ordering in ``AnyModelConfig`` is documentation only — it
has no effect on which config is iterated first.

Anima's previous probe accepted any state dict containing the substring
``cross_attn`` or ``self_attn``, which collides with Wan's native LoRA key
layout (``diffusion_model.blocks.X.cross_attn.q.lora_down.weight``). Both
probes accepted Wan native LoRAs (including lightx2v's Lightning T2V and I2V
distillations), and the ``matches.sort_key`` tiebreaker only disambiguates
by ModelType, not within LoRA configs. So which config "won" depended on
dict hash order — sometimes Wan, sometimes Anima.

The previous mitigation reordered the AnyModelConfig union to put Wan
before Anima. That worked by luck and was inherently fragile.

Tighten Anima's probe to require Cosmos-DiT-exclusive subcomponents:
``mlp``, ``adaln_modulation``, or ``_proj``-suffixed attention names
(``q_proj``/``k_proj``/``v_proj``/``output_proj``) — none of which appear
in any Wan LoRA. Wan native uses bare ``.q``/``.k``/``.v``/``.o`` on
``self_attn``/``cross_attn``, and ``ffn.N``/``ffn.net.N`` instead of ``mlp``.

The new strict detectors live alongside the original loose ones so the
Anima conversion utility (which runs after probing) still works.

Regression tests in ``test_wan_lora_probe_independence.py`` cover:
- I2V Lightning V1 (the bug-triggering LoRA), T2V Lightning V2, Wan Kohya
  and Wan diffusers PEFT layouts — Wan probe accepts, Anima probe rejects.
- Anima PEFT and Kohya layouts — Anima accepts, Wan rejects.
- A meta-test that runs every LoRA config in CONFIG_CLASSES against the
  Lightning state dicts and asserts exactly one accepts — this catches
  ANY future probe collision, not just Wan vs Anima.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): defer expert model loading in _ExpertSwapper to avoid cache thrash

The swapper used to take pre-loaded ``LoadedModel`` handles at construction:

    high_info = context.models.load(self.transformer.transformer)
    low_info  = context.models.load(self.transformer.transformer_low_noise)
    swapper = _ExpertSwapper(high_info=high_info, low_info=low_info, ...)

With dual ~9 GB A14B GGUF experts plus the ~10 GB UMT5-XXL encoder competing
for the same RAM cache, the LRU policy frequently dropped one expert by the
time the denoise loop swapped into it. The model manager then emitted

    [MODEL CACHE] Locking model cache entry ... but it has already been
    dropped from the RAM cache. This is a sign that the model loading
    order is non-optimal in the invocation code (See ... invoke-ai#7513).

and reloaded the weights from disk (~1.2s extra per swap).

Refactor the swapper to take the ``ModelIdentifierField`` plus the
``InvocationContext`` and call ``context.models.load(model_id)`` lazily
inside ``get()``. Each swap obtains a fresh handle, the LRU window is
small, and the warning goes away.

Config metadata (used to compute ``is_quantized``) is read upfront via
``context.models.get_config()`` — that's metadata, not weights, so it
doesn't put pressure on the cache.

Tests: existing swapper lifecycle tests refactored to use a fake context
whose ``models.load`` is logged. A new ``test_lazy_load_per_swap_not_upfront``
pins the regression — it asserts ``models.load`` is NOT called at swapper
construction, only at first get() per expert.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
test(wan): add Phase 8 inpaint regression tests
5d5f8cb 
The denoise_mask wiring + RectifiedFlowInpaintExtension integration in
wan_denoise.py was put in place during Phase 2/3 alongside the rest of
the denoise loop. Phase 8 of the plan was about ensuring this path
worked and is locked in by tests.

Three new tests under TestWanDenoiseInpaint:

1. test_preserved_region_matches_init_exactly: builds a half/half mask
   (left = preserve, right = regenerate in user-side convention), runs
   full denoise with the synthetic zero-output transformer, and asserts
   the preserved half of the final latents equals the init exactly while
   the regenerated half does not. Pins the mask-inversion + per-step
   merge behavior.

2. test_inpaint_requires_init_latents: a mask without init latents must
   raise a clear ValueError — the merge has nothing to weld back to.

3. test_no_mask_path_is_unchanged: regression that adding the inpaint
   extension didn't perturb the non-inpaint codepath (with init latents
   + denoising_start=0.5 but no mask, the loop just runs img2img).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

feat(frontend): add I2V_A14B to Wan variant zod enum + manager label

Phase 7 added the I2V_A14B backend variant. The frontend's zod enum
(features/nodes/types/common.ts:zWanVariantType) and the model manager's
variant-label map (features/modelManagerV2/models.ts) were still on the
two-variant list, so:

  - ModelIdentifierField inputs with ui_model_variant filters on Wan
    couldn't list I2V models.
  - The model manager UI showed a raw 'i2v_a14b' string instead of the
    human label.

Phase 9 (full linear-view wiring — type guards, hooks, params slice,
graph builder, tab UI) is in progress on a follow-up commit; this lands
the two small enum fixes first so the I2V probe / install paths work
correctly end-to-end with the existing FE.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(wan): Phase 9 piece #1 - linear-view T2V txt2img graph builder
c595452 
Adds the minimum frontend wiring needed to generate Wan 2.2 images from
the linear view:

  - buildWanGraph.ts (new): text-to-image graph (model_loader →
    text_encoder × 2 → denoise → l2i). Diffusers main model only —
    transformer, VAE and UMT5 encoder all resolve from the same repo, so
    no Wan-specific params slice fields are required yet. CFG-skip
    branch when guidance_scale ≤ 1.0.
  - useEnqueueGenerate / useEnqueueCanvas dispatchers: route
    base === 'wan' to buildWanGraph.
  - graph/types.ts: add wan_l2i / wan_i2l / wan_denoise / wan_model_loader
    to the relevant node-type unions.
  - addTextToImage / addImageToImage: include wan_denoise / wan_l2i so
    width/height are wired correctly and the txt2img helper accepts the
    Wan l2i node.
  - isMainModelWithoutUnet: include wan_model_loader (Wan has no UNet,
    same as the other modern bases).
  - metadata.py: add wan_txt2img / wan_img2img / wan_inpaint to the
    generation_mode enum (img2img / inpaint pieces land next).
  - schema.ts: regenerated to pick up the metadata enum + new
    Wan invocations.

Pieces left in Phase 9: params slice (standalone VAE / T5 / GGUF
low-noise / LoRA / ref-image fields + selectors), img2img + I2V + inpaint
branches in the graph builder, and Wan-specific UI components.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

feat(wan): Phase 9 piece #2 - GGUF support and CFG-Low control in linear view

Adds the three Wan-specific params + UI controls that gate GGUF workflows
plus a separate low-noise CFG slider for A14B users.

Params slice:
  - wanTransformerLowNoise (the second-expert GGUF for A14B)
  - wanComponentSource (Diffusers Wan model providing VAE + UMT5-XXL
    when the main is a GGUF)
  - wanGuidanceScaleLowNoise (optional separate CFG for the low-noise
    expert; null = fall back to the primary CFG)

Plus a `selectIsWan` selector for accordion gating.

UI components:
  - ParamWanModelSelects.tsx (Advanced accordion): two model pickers —
    Transformer (Low Noise) filtered to Wan GGUF mains, and VAE/Encoder
    Source filtered to Wan Diffusers mains. Mirrors the
    ParamQwenImageComponentSourceSelect structure.
  - ParamWanGuidanceScaleLowNoise.tsx (Generation accordion): slider +
    number input with an "auto" indicator when cleared. Default 3.5
    matches the diffusers reference 4.0 / 3.0 split.

Wiring:
  - Generation accordion: ParamWanGuidanceScaleLowNoise shown when base
    is wan, scheduler excluded for wan (same pattern as Anima/Qwen).
  - Advanced accordion: ParamWanModelSelects shown when base is wan, and
    Wan excluded from the SD-family VAE/CFG-rescale blocks.
  - buildWanGraph.ts: forwards the three new params to the model loader
    and denoise nodes (transformer_low_noise_model, component_source,
    guidance_scale_low_noise) and adds them to the graph metadata.

Hooks/types:
  - useWanDiffusersModels + useWanGGUFModels in modelsByType.ts.
  - isWanDiffusersMainModelConfig + isWanGGUFMainModelConfig type guards.
  - Three new locale strings (wanComponentSource, wanTransformerLowNoise,
    wanGuidanceScaleLowNoise[Auto]).

GGUF workflow now works end-to-end in the linear view: pick a Wan GGUF
main, set Transformer (Low Noise) to the paired second-expert GGUF, set
VAE/Encoder Source to any Diffusers Wan repo (TI2V-5B is convenient at
~12 GB) — generate produces an image.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): UX polish on the Wan linear-view controls

Bundles four small fixes applied during a usability review of the Wan
linear-view section (piece #2):

1. **Filter Main vs Transformer (Low Noise) dropdowns by expert tag.**
   The Wan GGUF probe records each file's ``expert`` field
   (``"high"`` / ``"low"`` / ``"none"``) via filename heuristic.
   - ``MainModelPicker``: hides ``expert === 'low'`` Wan GGUFs so users
     can't accidentally wire a low-noise expert as the primary main.
   - Transformer (Low Noise) picker (``useWanGGUFLowNoiseModels``):
     shows ``expert === 'low'`` Wan GGUFs only.

   Diffusers Wan mains and TI2V-5B aren't affected — they don't carry
   the ``expert`` field on their config schema. The backend's auto-swap
   safety net stays in place.

2. **Match the primary CFG slider's range.** The Wan low-noise CFG
   slider was constrained to 1–10 while the primary CFG ranges 1–20.
   With the diffusers reference 4/3 split, the low-noise slider thumb
   sat noticeably further right than the primary — visually misleading.
   Both sliders now share the 1–20 range with marks at [1, 10, 20].

3. **Label fits the form column.** "CFG (Low Noise)" → "CFG (Low)" so
   the slider fits cleanly next to its label instead of overlapping.

4. **Indicator state for the low-noise CFG slider.** Replaced the inline
   "(auto)" / "(same as cfg)" text — which kept overlapping the slider
   regardless of how short the label got — with an X-only reset button
   that's only visible when the user has set an explicit value. Absence
   of the X conveys auto/fallback state without any text overhang.

5. **Friendlier Transformer (Low Noise) placeholder.** "Second-expert
   GGUF for A14B (pair with the high-noise main)" → "Add for full
   detail" — concise nudge for users who haven't paired the second
   expert yet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

feat(wan): Phase 9 piece #3 - linear-view img2img branch

Adds Wan 2.2 image-to-image to the linear view, mirroring the Qwen Image
pattern. The mode switches on the canvas state — pure-prompt runs go
through addTextToImage as before; canvas runs with an init image go
through addImageToImage which wires a fresh wan_i2l (Image to Latents -
Wan 2.2) node between the init image and the denoise's `latents` input,
honoring the existing denoise_start slider.

buildWanGraph:
  - Drops the txt2img-only guard, branches on generationMode.
  - img2img: spins up a wan_i2l node and hands it to addImageToImage
    alongside the existing denoise / l2i / modelLoader (as vaeSource).
  - inpaint / outpaint still fail loudly — pieces #4-#6.

graphBuilderUtils.getDenoisingStartAndEnd:
  - Adds 'wan' to the simple-linear case (denoising_start = 1 -
    denoisingStrength). Note: Wan's flow-matching schedule is "sticky"
    on the init compared to SDXL — users will likely need denoisingStrength
    ≥ 0.7 to see substantial change, matching the user-found 0.15-0.3
    denoising_start sweet spot from earlier img2img testing. We may
    revisit this with an exponent rescale (like FLUX uses) if the
    response curve feels off.

addImageToImage:
  - Adds 'wan_i2l' to the i2l-node-type union so the Wan i2l can be
    threaded through the shared helper.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): add wan_denoise to addImageToImage/addInpaint/addOutpaint type checks

Three sibling graph-helper utilities had the same modern-base list as
addTextToImage did, and the buildWanGraph img2img branch tripped one of
them at canvas-Generate time:

    error  [generation]: Failed to build graph
    {name: 'Error', message: 'Wrong assertion encountered'}

The else-branch in each helper assumes 'denoise_latents' (the SD1.5/SDXL
legacy path) and asserts that — failing for any modern base not listed
above the branch. addTextToImage was already updated in Phase 9 piece #1;
this catches the parallel cases that the img2img/inpaint/outpaint flows
go through.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

feat(wan): Phase 9 piece #4 - linear-view inpaint and outpaint branches

Wires Wan 2.2 inpaint and outpaint through the existing addInpaint /
addOutpaint helpers. The backend's RectifiedFlowInpaintExtension was
plumbed into wan_denoise.py back in Phase 8 (commit ab54617); this
just connects the FE.

buildWanGraph:
  - generationMode === 'inpaint' → spin up a wan_i2l, call addInpaint
    with denoise + l2i + modelLoader (used as both vaeSource and
    modelLoader since the Wan model loader carries the VAE).
  - generationMode === 'outpaint' → parallel branch with addOutpaint.

addInpaint:
  - i2l-node-type union now includes 'wan_i2l' (the addImageToImage and
    addOutpaint type unions already do — different union shapes).

metadata.py:
  - generation_mode literal adds "wan_outpaint" alongside the existing
    wan_txt2img / wan_img2img / wan_inpaint entries.

isMainModelWithoutUnet already includes wan_model_loader (Phase 9 piece
create_gradient_mask when Wan is the main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

feat(wan): Phase 9 piece #5 - linear-view I2V branch (raster as reference image)

Wan 2.2 I2V-A14B models condition on a reference image whose VAE-encoded
latents are concatenated to the noise along the channel dim each step
(in_channels=36 on the I2V transformer). In the linear view this maps
cleanly onto the existing canvas raster layer: pick an I2V model, drag
an image to raster, generate.

buildWanGraph:
  - Fetch the modelConfig early so the variant gate (i2v_a14b vs the
    rest) can drive the branch shape instead of being a post-hoc check.
  - I2V + txt2img: fail loudly ("Switch to the canvas tab and drag an
    image to the raster layer"). I2V models won't produce useful output
    without a reference, and the backend would crash trying to
    concatenate a missing condition tensor.
  - I2V + img2img: pull the raster image via the canvas compositor,
    wire it through a wan_ref_image_encoder (which VAE-encodes it and
    builds the 4-mask + 16-latent condition tensor backend-side), then
    feed the result into denoise.ref_image. Denoise runs from fresh
    noise (denoising_start=0, no init_latents) — the ref image is
    cross-attention/concat conditioning, not a noise-trajectory anchor.
  - I2V + inpaint/outpaint: fail clearly. Combining ref-image
    conditioning with a denoise mask is conceptually possible but the
    backend interaction hasn't been validated end-to-end.

metadata.py:
  - Adds "wan_i2v" to the generation_mode literal so the metadata field
    on I2V renders correctly.

T2V flows (txt2img / img2img / inpaint / outpaint) are unchanged for
non-I2V Wan variants (T2V-A14B and TI2V-5B).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): enforce multiple-of-16 dimensions to match transformer patch grid

Wan 2.2's transformer has ``patch_size=(1, 2, 2)``: it patch-embeds with
stride 2 then un-patches by 2. Combined with the VAE's 8x spatial scale,
canvas H/W must be a multiple of ``8 * 2 = 16`` — not just 8 — for the
patch round-trip to land exactly. Otherwise the latents and noise
prediction disagree by one in the spatial dim and the scheduler step
fails:

    RuntimeError: The size of tensor a (147) must match the size of
    tensor b (146) at non-singleton dimension 3

(here latent_w=147 → patch_w=73 → un-patched_w=146 ≠ 147)

This was silent for T2V at 1024x1024 (already a multiple of 16) but
fired for I2V at non-multiple-of-16 canvas sizes.

Fixes:

- ``optimalDimension.getGridSize``: Wan moves from the default 8 case to
  the multiple-of-16 case (alongside flux / sd-3 / qwen-image / z-image
  which have the same patch arithmetic). The canvas bbox UI now snaps
  Wan dimensions to multiples of 16.

- ``wan_denoise.py`` and ``wan_ref_image_encoder.py``: bump width/height
  ``multiple_of`` from 8 to 16. Defense-in-depth — workflow-editor
  users won't be able to send a non-16-aligned dim either.

Existing backend tests (23 passing) still hold — 1024 is divisible by 16
so the test fixtures didn't exercise the off-by-one path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): show negative prompt box in Wan linear-view

Wan was missing from SUPPORTS_NEGATIVE_PROMPT_BASE_MODELS, so the
linear-view negative-prompt input was hidden even though the Wan denoise
node already wires negative conditioning when CFG > 1
(buildWanGraph.ts:67-75). Adds 'wan' to the list.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

feat(wan): Phase 9 piece #6 - Wan LoRA collection in linear view

Adds Wan LoRA wiring to buildWanGraph, mirroring the Qwen Image pattern.
The shared LoRASelect / LoRAList UI in the linear view already filters
LoRAs by the selected main model's base, so Wan LoRAs surface
automatically when a Wan main is picked — no UI changes needed.

addWanLoRAs (new):
  - Filters state.loras.loras to enabled Wan LoRAs.
  - For each LoRA: spawns a ``lora_selector`` node and threads it
    through a single ``collect`` collector.
  - Routes the collector into a ``wan_lora_collection_loader`` which
    sits between modelLoader and denoise — modelLoader.transformer →
    loader, then loader.transformer → denoise (rerouting the original
    modelLoader → denoise edge).
  - Emits per-LoRA metadata so PNG metadata + workflow restore work.

The dual-expert routing (high-noise vs low-noise vs untagged) is
handled entirely on the backend by ``WanLoRACollectionLoader`` based on
each LoRA's recorded ``expert`` tag (set by the probe from the filename
heuristic in piece #5 of Phase 5). The FE just hands over the bag of
LoRAs; no per-list FE plumbing needed.

buildWanGraph:
  - Calls addWanLoRAs(state, g, denoise, modelLoader) after the base
    transformer edge is in place. The helper is a no-op when no Wan
    LoRAs are enabled, so it's safe to call unconditionally.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix(wan): detect LoRA variant and filter by main model

Wan 2.2 A14B (inner_dim=5120) and TI2V-5B (inner_dim=3072) LoRAs are not
interchangeable — applying one against the wrong main model crashes the
layer patcher with a tensor-shape error (e.g. A14B Lightning on TI2V-5B
mains produced ``shape '[3072, 3072]' is invalid for input of size 26214400``).

Probe Wan LoRAs' inner-dim at install time and record the family on a new
``variant`` field (``a14b`` / ``5b`` / null). The LoRA picker in the linear
view hides incompatible variants when the user selects a main, and the
graph builder filters any still-enabled mismatches at submit time with a
warning. Untagged LoRAs (probe couldn't identify) pass through so they
aren't silently hidden.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

feat(wan): ref-image panel, GGUF readiness, and auto-default sources

Wan 2.2 I2V now uses the global Reference Images panel (same UX as Qwen
Image Edit and FLUX.2 Klein) instead of pulling the conditioning image
from a canvas raster layer. Adds:

  - WanReferenceImageConfig zod type + isWanReferenceImageConfig guard;
    integrated into the ref-image discriminated union, settings panel,
    layer hooks, and validators.
  - 'wan' added to SUPPORTS_REF_IMAGES_BASE_MODELS, but the panel only
    shows for the i2v_a14b variant (T2V and TI2V-5B don't consume ref
    images, so the panel is hidden for them).
  - buildWanGraph I2V branch reads the first enabled wan_reference_image
    from refImagesSlice; the canvas-raster-as-ref path is removed. I2V
    now only supports txt2img mode (canvas img2img/inpaint/outpaint
    assert with a clear message).

GGUF Wan readiness check: GGUF mains carry only the transformer, so the
loader needs a Diffusers Component Source (or standalone VAE + UMT5-XXL
encoder) to resolve the VAE and text encoder. Without one, enqueue is
now blocked with a clear reason. The low-noise A14B partner expert
remains optional (loader falls back to the high-noise expert when it's
missing).

Adds standalone Wan VAE and Wan T5 Encoder selectors to the Advanced
accordion (Qwen pattern). Wires them as vae_model / wan_t5_encoder_model
on the wan_model_loader node — backend priority is standalone > diffusers
main > component source.

Auto-default on Wan selection (so GGUF users don't have to fiddle with
Advanced): when the new main is a Wan GGUF, fill the Component Source,
standalone VAE, and standalone T5 encoder with first available matches
if not already set. Component Source is matched by variant family
(A14B GGUF prefers an A14B Diffusers; TI2V-5B prefers a TI2V-5B
Diffusers) since the two families use different VAE channel counts
(16 vs 48); within A14B, T2V and I2V share VAE/encoder so they're
interchangeable as a source. Runs on every Wan selection (including
Diffusers -> GGUF switches), only fills empty slots.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(wan): add Wan 2.2 starter models and bundle
727eff9 
Wan 2.2 starter pack (selected when the user picks the Wan 2.2 bundle)
brings up the minimal-cost path to running A14B T2V end-to-end:

  - Standalone UMT5-XXL encoder and A14B VAE (so GGUF mains don't need
    a full Diffusers download for their VAE/encoder sources).
  - T2V A14B Q4_K_M and Q8_0 GGUF expert pairs (high + low noise).
  - T2V Lightning V1.1 Seko rank-64 LoRA pair (4-step inference).

Additional Wan 2.2 starter models browseable from the model manager:

  - Full Diffusers T2V A14B, I2V A14B, and TI2V-5B.
  - I2V A14B Q4_K_M and Q8_0 GGUF expert pairs + Lightning V1 LoRA pair.
  - TI2V-5B Q4_K_M and Q8_0 GGUFs + the 48-channel TI2V-5B VAE.

Each "high noise" GGUF lists its low-noise partner plus the shared VAE
and UMT5-XXL encoder as dependencies, so installing one of them pulls
in everything the loader needs. QuantStack's HighNoise/LowNoise file
naming and lightx2v's high_noise_model/low_noise_model.safetensors are
both picked up by the existing filename heuristic in the GGUF probe.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

docs(wan): add Wan 2.2 hardware requirements

Adds Wan 2.2 A14B (T2V/I2V) and TI2V-5B rows to the hardware
requirements table with rough VRAM/RAM guidance per quantization.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
fix(wan): recall low-noise transformer, component source, and standal…
6c076c5 
…one VAE/T5

Wan-specific metadata fields embedded by the graph builder
(wan_transformer_low_noise, wan_component_source, wan_vae_model,
wan_t5_encoder_model, wan_guidance_scale_low_noise) had no recall
handlers in features/metadata/parsing.tsx, so recalling an image's
parameters would leave these fields empty. Adds a handler for each
that dispatches the matching paramsSlice action and renders a row in
the metadata viewer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein
chore(frontend): typegen
b9b0665
@lstein @claude
feat(wan): add default Wan 2.2 T2V and I2V workflows
2d1add8 
Ships two default workflows in the library, tagged so they appear in
"Browse Workflows" under the wan2.2 / text to image / image to image
tags:

  - Text to Image - Wan 2.2: full T2V/TI2V-5B graph (model loader,
    positive + negative encoders, denoise, l2i). Exposes the five
    model slots, prompts, steps, dual CFG, and dimensions.
  - Image to Image - Wan 2.2: I2V A14B graph that adds a
    wan_ref_image_encoder. Exposes the reference image input plus
    the standard fields.

Both follow default-workflow rules: IDs prefixed with default_,
meta.category = "default", and no references to user-installed
resources.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(videos): Phase 1 - backend video storage, records, REST API
19d5c53 
Adds a parallel video pipeline alongside the existing image pipeline so the
gallery can host MP4 alongside PNGs. Implements:

- New service modules (parallel to image equivalents):
    video_records/    record store + sqlite impl
    video_files/      disk file store (mp4 + first-frame webp thumb)
    videos/           orchestrating service
    board_video_records/   board <-> video association
- migration_32 creates `videos` and `board_videos` tables
- /api/v1/videos/ router: upload, list, get DTO, /full (with HTTP Range
  so HTML5 <video> seek/scrub works), /thumbnail, /metadata, star/unstar,
  delete, batch delete, board add/remove
- LocalUrlService.get_video_url and SimpleNameService.create_video_name
- imageio[ffmpeg] dep for video encode (used in later phases)
- Wires all four new services into InvocationServices, dependencies.py,
  api_app.py, and three test fixtures

Verified end-to-end against an in-memory db + tmp output dir: upload,
probe, save (file + thumbnail + record), DTO build, list, delete.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(videos): Phase 2 - polymorphic gallery list endpoint
a0913ea 
Adds /api/v1/gallery/items/ and /api/v1/gallery/items/names returning a
unified time-sorted stream of images + videos so the frontend can render
them interleaved with a single virtualized query.

- gallery_common: GalleryItem discriminated union (kind + name + shared
  fields + nullable video duration/fps), GalleryItemRef, names result
- gallery_default: SqliteGalleryService implements UNION ALL across the
  images and videos tables, applying identical filters (origin/category/
  is_intermediate/board_id/search) to each half; pagination via outer
  ORDER BY + LIMIT/OFFSET; counts are summed across the two halves
- URLs are resolved at row -> DTO conversion time so each item routes to
  the correct /api/v1/images or /api/v1/videos endpoint
- Wired into InvocationServices, dependencies.py, api_app.py, and the
  three test fixtures

Existing /api/v1/images endpoints are unchanged so any non-gallery
consumers (queue, recall, metadata workflows) continue to work as-is.

Verified e2e: 2 images + 2 videos inserted in alternating order, both
list_items and list_item_names return the correct interleaved order;
category filter narrows to a single kind; starring an item bumps it to
the top when starred_first=True.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(videos): Phase 3 - frontend RTK endpoints + MP4 upload routing
c9aff11 
Adds the typed API surface and upload integration so videos can be
uploaded through the same gallery upload button that handles images.

Schema: re-ran pnpm typegen against the running backend to pick up
VideoDTO, VideoRecordChanges, GalleryItem, GalleryItemKind,
GalleryItemRef, GalleryItemNamesResult and the two new paginated
result types.

RTK Query (services/api/endpoints/videos.ts) - parallel to images.ts:
listVideos, getVideoDTO, getVideoMetadata, getVideoNames, uploadVideo,
deleteVideo / deleteVideos, changeVideoIsIntermediate, starVideos /
unstarVideos, addVideoToBoard / removeVideoFromBoard. Imperative helpers
(getVideoDTO, getVideoDTOSafe, uploadVideo, uploadVideos) and the
useVideoDTO convenience hook ride alongside, mirroring the image side.

Tag types and invalidation: added Video / VideoList / VideoMetadata /
VideoNameList / BoardVideosTotal / GalleryItemList / GalleryItemNameList
to the api root. Board-affecting mutations now invalidate the polymorphic
gallery list/name caches so videos and images stay coherent once the
gallery wiring lands in Phase 4. Added a sibling
getTagsToInvalidateForVideoMutation helper.

Upload UX: useImageUploadButton.tsx's dropzone now accepts video/mp4,
video/webm, video/quicktime alongside the existing image MIMEs. The
drop handler splits files into image/video sets and routes each through
its own mutation; a new onUploadVideo callback parallels the existing
onUpload. Existing image-only callers pass through unchanged.

Polymorphic gallery query endpoints + the useGalleryItemDTO hook will
land with Phase 4 where they have actual consumers; the schema types
they'll need are already in place under @knipignore tags.

Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green;
pnpm test 1103/1103 pass; live curl against the running dev server
uploads an MP4 and serves both the webp thumbnail and the MP4 with
a working HTTP Range response (206 + Content-Range).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(videos): Phase 4 - mixed gallery grid with video play badge
7cf98cb 
Videos now appear in the same gallery grid as images, interleaved by
created_at. Video thumbnails get a centered play-button badge so they
read as videos at a glance; everything else (selection, virtualization,
search, paged/virtual gallery views, keyboard nav) is unchanged.

Approach: selection state stays `string[]` of names. The kind is
recovered from the filename extension (.mp4 = video, anything else =
image), which is reliable because the backend's SimpleNameService
always emits `<uuid>.png` for images and `<uuid>.mp4` for videos. This
sidesteps a 32-file cross-cut from changing the selection shape to a
discriminated union, and selection is persist-denylisted so no
migration is needed.

Frontend:
- new isVideoName helper in features/gallery/store/types
- new endpoints/gallery.ts (deferred from Phase 3): useGetGalleryItemNamesQuery
- new ImageGrid/GalleryItemPlayBadge: centered triangular badge over thumbnail
- new ImageGrid/GalleryItemVideoStarIconButton: video-typed star toggle
- new ImageGrid/GalleryVideoItem: counterpart to GalleryImage; reuses
  galleryItemContainerSX, GalleryItemSizeBadge (width/height-only stand-in),
  selection handling (single/shift/ctrl/cmd); alt-click falls through to a
  normal select since comparison is image-only
- use-gallery-image-names now calls the polymorphic gallery names endpoint
  and exposes a mixed flat name list (existing callers - paged grid, search,
  navigation hotkeys - get the same shape)
- useRangeBasedImageFetching partitions visible names by extension; images
  bulk-fetch via the existing getImageDTOsByNames mutation, videos dispatch
  individual getVideoDTO queries (no batch endpoint yet)
- GalleryImageGrid's ImageAtPosition dispatches on isVideoName to render
  GalleryImage or GalleryVideoItem; star hotkey dispatches to the right
  star/unstar mutation based on kind
- pruned the now-unused useGetImageNamesQuery / isImageName exports

Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green;
pnpm test 1103/1103 pass; live curl of /api/v1/gallery/items returns
57 polymorphic items with video duration populated and image duration
null, /api/v1/gallery/items/names returns matching {kind, name} refs.

The useGalleryItemDTO hook is intentionally deferred to Phase 5 where
the polymorphic viewer is its first real consumer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(videos): Phase 5 - inline video player in the image viewer
50d9588 
Selecting a video now renders a polymorphic preview inside the existing
viewer panel: thumbnail with a centered play button by default; clicking
play swaps in an HTML5 <video controls autoplay>. Switching to a
different item drops the video element back to idle (auto-pauses) and
selecting an image again returns to the normal image preview.

New components (features/gallery/components/ImageViewer/):
- VideoPlayButtonOverlay: large centered play button with hover/shadow,
  used over the thumbnail in the idle state.
- CurrentVideoPreview: idle/playing state machine. Resets on
  video_name change. The <video> src points at /api/v1/videos/i/.../full
  which supports HTTP Range, so seek/scrub work natively in the browser.

New hook:
- common/hooks/useGalleryItemDTO: polymorphic DTO resolver that
  dispatches between useImageDTO and useVideoDTO based on filename
  extension (isVideoName). Centralizes the kind-dispatch the viewer
  and toolbar both need.

Wiring:
- ImageViewer dispatches on galleryItem.kind to render CurrentImagePreview
  or CurrentVideoPreview. The compare-image DnD drop target is hidden when
  a video is selected (comparison is image-only).
- ImageViewerToolbar hides the image-specific action row
  (CurrentImageButtons - load workflow, recall metadata, edit, etc.) and
  the metadata viewer toggle when a video is selected. The general-purpose
  ToggleProgressButton stays.

Out of scope (per the plan): video deletion from the viewer (use gallery
hover icons), video-specific metadata viewer, comparison-mode support
for videos.

Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green;
pnpm test 1103/1103 pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
fix(videos): accept MP4 (and other video) drops on the fullscreen dro…
1f62b44 
…pzone

The gallery-wide drag-and-drop target lives in FullscreenDropzone, not
in useImageUploadButton (which only powers the upload button). It had
its own hardcoded image-only zod allowlist that rejected MP4 files
with "File type / extension is not supported".

- Broaden the zod refines to accept video/mp4, video/webm,
  video/quicktime, video/x-matroska and the matching extensions
- Add isVideoFile helper, split dropped files into image/video sets,
  and route each set through its own uploader (uploadImages /
  uploadVideos). Both update their respective RTK caches and
  invalidate the polymorphic gallery list/names.
- Skip the canvas-paste fast-path for single-video drops — the canvas
  doesn't host videos as layers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(videos): right-click context menu on video items
c9ce8d6 
Adds a three-item context menu (delete, change board, download) on
right-click / long-press of any gallery video item. Mirrors the image
context menu's singleton-portal architecture so re-renders stay cheap.

New files:
- features/gallery/contexts/VideoDTOContext: small React context that
  scopes the active video DTO to the menu items (parallels
  ImageDTOContext).
- features/gallery/components/ContextMenu/MenuItems/
    ContextMenuItemDeleteVideo: window.confirm + deleteVideo mutation.
      Videos can't be referenced from canvas/nodes/refs, so the image
      modal's usage analysis is unnecessary; a one-step confirm matches
      the "minimal" scope.
    ContextMenuItemDownloadVideo: reuses the existing useDownloadItem
      hook against videoDTO.video_url / video_name.
    ContextMenuItemChangeBoardVideo: dispatches videosToChangeSelected
      and opens the (now polymorphic) ChangeBoardModal.
- features/gallery/components/ContextMenu/VideoContextMenu: singleton
  pattern lifted from ImageContextMenu — registers gallery video
  elements via a Map; right-click looks up the target node and opens
  the menu at the cursor.

Extended files:
- features/changeBoardModal/store/slice: added video_names alongside
  image_names plus a videosToChangeSelected action. The two arrays are
  mutually exclusive — setting one clears the other.
- features/changeBoardModal/components/ChangeBoardModal: now dispatches
  the matching video board mutations (add/removeVideoToBoard, plural
  endpoints don't exist yet so videos move one at a time — the menu
  acts on a single selection so this is a one-iteration loop).
- features/gallery/components/ImageGrid/GalleryVideoItem: registers
  itself with useVideoContextMenu.
- app/components/GlobalModalIsolator: mounts the singleton.

Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green;
pnpm test 1103/1103 pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(wan): Phase 6 - Wan 2.2 T2V/I2V workflow nodes
216322f 
Adds two new invocation nodes that produce MP4 videos from a Wan 2.2
A14B transformer + VAE, plus the supporting plumbing.

New invocations:
- WanVideoDenoise (wan_video_denoise) — multi-frame counterpart to
  WanDenoise. Same per-step logic (CFG, MoE expert swap at the
  boundary timestep, LoRA patching, scheduler dispatch) — reuses
  _ExpertSwapper, _resolve_variant, and the scheduler/LoRA helpers
  from wan_denoise. Difference: the noise tensor has a real temporal
  dim built from num_frames, and the I2V condition is built across
  all latent frames (frame 0 conditioned, rest zero). Defaults match
  the Wan 2.2 reference: 832x480 / 81 frames / 40 steps / CFG 5.0
  (high) + 4.0 (low). Inpaint / img2img are out of scope for this
  first cut. TI2V-5B is rejected; T2V/I2V A14B only.
- WanLatentsToVideo (wan_l2v) — VAE-decodes 5D latents to RGB frames
  via AutoencoderKLWan (T_pixel = (T_lat - 1) * 4 + 1), then encodes
  an MP4 with imageio[ffmpeg] (libx264, yuv420p for browser
  compatibility). The temp file is moved into outputs/videos/ via
  context.videos.save().

Backend shared pieces:
- make_noise gains num_latent_frames (default 1, backward compatible).
- Added num_latent_frames_for(num_frames, scale=4) helper.
- New encode_reference_image_to_video_condition mirrors diffusers'
  WanImageToVideoPipeline.prepare_latents with last_image=None and
  expand_timesteps=False: pads the reference image with zero
  pixel-frames, VAE-encodes the full pseudo-video, normalises, and
  builds the 4-channel temporal-rearranged first-frame mask. Verified
  numerically: 21 latent frames for num_frames=81, first latent
  frame's 4 mask channels = 1, rest = 0.
- The existing single-frame encoder is left untouched.

Schema / context:
- New VideoField primitive (parallel to ImageField) and VideoOutput
  invocation output (width/height/num_frames/fps/duration/video).
- New VideosInterface on InvocationContext with .save(source_path,
  width, height, duration, fps, ...) returning VideoDTO. Mirrors
  ImagesInterface — falls back to WithBoard / WithMetadata mixins
  and embeds the queue item's workflow/graph as a JSON sidecar.
- WanRefImageConditioningField now carries num_frames so the denoise
  nodes can sanity-check the I2V condition. WanRefImageEncoder bumps
  to v1.1.0 and gains num_frames=1 input (use 81+ for video I2V; the
  encoder dispatches between the single- and multi-frame helpers).
- Image WanDenoise now rejects multi-frame conditions with a clear
  message pointing at WanVideoDenoise.

Verified: pnpm lint (5/5) green; pnpm tests (multiuser auth 122/122
+ broader suite via prior runs); numerical shape checks for noise
and ref-image condition; end-to-end smoke via VideoService.create.

A restart of the InvokeAI server is required to pick up the new
invocations in the workflow editor.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(wan): add Wan 2.2 T2V and I2V starter video workflows
87982e9 
Two new default workflows for the workflow editor 'Browse' modal:

- 'Text to Video - Wan 2.2' — model loader -> two text encoders ->
  wan_video_denoise -> wan_l2v. Exposes prompt, model picks, CFG
  (high + low), dimensions, frames, fps, and steps.
- 'Image to Video - Wan 2.2' — same shape plus a wan_ref_image_encoder
  feeding the denoise node's ref_image input. Exposes the reference
  image and the frames field on the ref-image node (must match the
  denoise node's frames — there is a clear validation error if they
  diverge, but the starter has them in sync at 81).

Both default to the Wan 2.2 reference settings: 832x480, 81 frames @
16 FPS (~5 s), 40 steps, CFG 5.0 (high expert) + 4.0 (low expert),
seeded by a rand_int. Pass the existing _sync_default_workflows
validator (id starts with default_, meta.category=default).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
fix(wan): startup crash from stringified VideoOutput annotation
cac3662 
run_app.py validates every invocation's return-type annotation against
the output-class registry. wan_latents_to_video.py had a stray
'from __future__ import annotations' which made the `invoke()` return
annotation a string ('VideoOutput') at runtime. The registry mismatch
triggered the unregistered-output warning path, which itself crashed
on output_annotation.__name__ because the annotation was a str:

  AttributeError: 'str' object has no attribute '__name__'

The other Wan invocations don't use future annotations — drop the
import to match. Verified post-fix: api_app import populates 95
output classes, wan_l2v annotation resolves to the real VideoOutput
class and is in the registry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(wan): add Wan 2.2 Lightning T2V starter workflow
7f89d69 
Same graph as 'Text to Video - Wan 2.2' but with two Apply LoRA - Wan
2.2 nodes chained between the model loader and the denoise node, and
defaults retuned for the Lightning distillation: 4 steps and CFG 1.0
on both experts (CFG=1 skips the negative-conditioning forward pass
entirely, ~20x faster than the 40-step / CFG-5.0 baseline at similar
quality).

Adapted from a user-saved workflow; cleaned for distribution by
stripping the install-specific model/LoRA key bindings (defaults
should not bake in local UUIDs), bumping to a fresh default_-prefixed
id with meta.category=default, exposing the two LoRA fields (lora +
weight) so users can swap LoRAs without diving into the canvas, and
flagging the negative-prompt node as unused at CFG=1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(wan): add Wan 2.2 Lightning T2V and I2V starter workflows
325fbbd 
Two new default workflows that wire the Lightning LoRA pair into the
T2V and I2V video pipelines for a ~20x speedup:

- 'Text to Video - Wan 2.2 Lightning' — model loader -> apply LoRA
  (high) -> apply LoRA (low) -> text encoders -> wan_video_denoise
  -> wan_l2v. Defaults to 4 steps and CFG 1.0 (no negative branch).
  Cleaned-up version of Lincoln's saved Lightning workflow: stripped
  per-install model/LoRA keys, switched meta.category to 'default'
  with a default_ id, and exposed both LoRA loaders' lora/weight/
  target fields so users can swap LoRAs without diving into the
  canvas.
- 'Image to Video - Wan 2.2 Lightning' — same chain plus a
  wan_ref_image_encoder (v1.1.0 with num_frames) feeding the denoise
  ref_image input. Defaults match the non-Lightning I2V starter
  (832x480, 81 frames @ 16 FPS) but with 4 steps / CFG 1.0.

LoRA target defaults to 'auto' so properly-tagged Lightning LoRAs
route themselves; both workflow descriptions tell users to set
explicit 'high'/'low' targets if their LoRAs are untagged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
fix(wan): use FFMPEG plugin (not pyav) for MP4 encode
715df66 
wan_latents_to_video was passing plugin='pyav' to iio.imwrite, but the
runtime only has imageio-ffmpeg installed (no PyAV). The encode step
at the very end of generation crashed with:

  ImportError: The `pyav` plugin is not installed.
  Use `pip install imageio[pyav]` to install it

Switch to plugin='FFMPEG' — backed by the bundled imageio-ffmpeg
binary that pyproject already requires via imageio[ffmpeg]. libx264
yuv420p is the FFMPEG plugin's default for .mp4, so the explicit
pixel_format is dropped (specifying it just produced a "Multiple
-pix_fmt options" warning).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(wan): log VAE decode and MP4 encode milestones in wan_l2v
9117315 
The video VAE decode + MP4 encode tail can take 30-90s on top of the
denoise loop, and the toast-style signal_progress() messages don't
land in the server log. Add context.logger.info() at:

- VAE decode start: latent frame count -> pixel frame count + resolution
- MP4 encode start: frames, fps, duration, dimensions
- MP4 encode complete: encoded file size
- Video saved: final video_name

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
fix(wan): switch video thumbnail/probe to imageio[ffmpeg] backend
7b4eee1 
After wan_l2v wrote a successful libx264 MP4 to disk, the invocation
would hang in DiskVideoFileStorage.save() during the cv2.VideoCapture
thumbnail-extraction step. cv2 wheels on this build can't reliably
decode our libx264/yuv420p output (most often the wheel was compiled
without an h264 decoder, but the failure mode is silent hang rather
than a clear error). The net effect: the MP4 ends up in
outputs/videos but the queue item never completes, so the frontend
spinner spins forever and the gallery doesn't pick up the new entry.

Fix: rewrite extract_video_frame and probe_video to try imageio's
FFMPEG plugin first (same backend that did the encoding — so reading
our own output is guaranteed to work), with cv2 retained only as a
fallback for uploaded videos in formats imageio can't decode.

Also add fine-grained log lines + exception guards inside
DiskVideoFileStorage.save() so a future thumbnail failure can no
longer hang the whole save — it now logs a warning and continues,
leaving the video record in place even if the thumbnail step
errored. With logging at each step (video written, thumbnail
written, sidecar written) any future hang will be obvious from the
last log line.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
fix(videos): handle VideoField outputs in invocation_complete
a7dfb3e 
After wan_l2v wrote its MP4 successfully, the gallery and viewer were
never updated: the new video didn't appear and the viewer stayed stuck
on the previous "Saving video" progress spinner indefinitely.

Root cause: onInvocationComplete.tsx only inspected results for
isImageField / isImageFieldCollection. VideoField outputs were silently
dropped, so the polymorphic gallery list never invalidated and no
auto-switch happened. The viewer therefore kept rendering
CurrentImagePreview, whose ImageViewerContext-local $progressEvent /
$progressImage atoms intentionally aren't cleared on queue completion
when autoSwitch is on — they rely on the new image's DndImage onLoad
to clear them, which never fires for a video.

Fix: add isVideoField (mirrors isImageField against {video_name}) and
plumb video outputs through onInvocationComplete:
- getResultVideoDTOs pulls VideoDTOs via getVideoDTOSafe
- addVideosToGallery invalidates GalleryItemNameList / GalleryItemList
  so the polymorphic gallery refetches and the new video shows up
- auto-switch dispatches the video name into selection (selection is a
  polymorphic string[]; useGalleryItemDTO already discriminates by
  filename extension)

The selection change swaps CurrentImagePreview for CurrentVideoPreview,
which unmounts the stale progress overlay along with it — so the stuck
spinner clears as a side-effect of the auto-switch.

Also drops the now-stale @knipignore on getVideoDTOSafe, which has a
real consumer now.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(video): add 'Frame from Video' invocation
9f212d7 
Extracts a single frame from a VideoField input and saves it as a
regular ImageDTO via context.images.save, so it appears in the gallery
like any other generated image.

Primary use case is I2V "shot extension": take the last frame of a
Wan-generated clip (default frame_index=-1) and feed it back as the
reference image for the next clip, then stitch the MP4s to get videos
longer than the model's single-shot frame budget at a given VRAM.

Negative frame_index is resolved against the actual decoded frame count
via probe_video() rather than passed through to imageio — not all
imageio plugins handle index=-1 uniformly, and being explicit lets us
emit a precise out-of-range error.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
feat(video): add 'Concatenate Videos' invocation
e29e738 
Joins two or more videos into a single MP4 with one of three transition
modes between consecutive clips:

- cut: hard splice, no blending. Total length = sum of inputs.
- crossfade: linear A→B dissolve over transition_frames. Each boundary
  consumes N frames from both surrounding clips, shrinking total length
  by N per boundary.
- fade_through_black: A fades to black, then B fades in. Each boundary
  consumes N/2 from each side and emits N output frames — total length
  is preserved.

Implementation decodes via imageio's FFMPEG plugin (matching wan_l2v on
the encode side) and runs the blends in numpy. All decoded frames are
kept in memory at once; fine for the few-hundred-frame I2V chains that
motivated this, would want streaming if anyone ever feeds in hour-long
uploads.

Up-front validation enforces matching dimensions across inputs and
checks that each clip has enough frames to spare from its head and tail
for the requested transitions — saves a wasted decode pass when the
transition window is too wide for one of the clips.

Pairs with 'Frame from Video' for I2V shot extension: generate N clips
chained via last-frame-as-ref-image, then glue them with a crossfade.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lstein @claude
fix(videos): show full-resolution first frame in viewer
e5d1fd8 
The viewer used a chakra <Image src={thumbnail_url}> in the idle (not-
playing) state, so once a clip auto-selected after generation the
preview snapped from the full-resolution denoise progress image to the
small WebP gallery thumbnail upscaled to fit — visibly soft compared to
what the user was watching seconds earlier.

Switch to a single <video> element that spans both states:

- idle: muted, no controls, preload="metadata". With no `poster` attr
  the browser decodes and shows the video's actual first frame at full
  resolution (this is the documented HTMLVideoElement default).
- playing: same DOM node with controls+audio toggled on, kicked off via
  ref.play(). No reload between states — the decoded buffer carries
  over.

`key={videoName}` swaps the element cleanly when the user moves to a
different clip, dropping any in-progress playback state.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lstein and others added 2 commits July 27, 2026 19:52
@lstein
Merge remote-tracking branch 'origin/main' into lstein/feature/wan-vi…
553b0da 
…deo-support

# Conflicts:
#	invokeai/frontend/web/src/features/nodes/util/schema/buildFieldInputTemplate.ts
@JPPhoto
Merge branch 'main' into lstein/feature/wan-video-support
f021091
@lstein

lstein commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator Author

@lstein I am going to resolve conflicts and it can be merged as soon as everything is green - can you open that follow-up PR or issue?

I'll open up the follow-up PR right after the merge. I'm waiting for the CI tests to go green.

@lstein
lstein merged commit eb9a951 into invoke-ai:main Jul 28, 2026
17 checks passed
@lstein
lstein deleted the lstein/feature/wan-video-support branch July 28, 2026 00:42
This was referenced Jul 28, 2026
Merged
Merged
lstein added a commit to lstein/InvokeAI that referenced this pull request Jul 28, 2026
@lstein @claude
Merge branch 'main' into lstein/feat/multi-gpu
4ce0a78 
Conflict in image_files_disk.py: main's staged-delete protocol (invoke-ai#9163)
supersedes the old delete() body; its cache eviction in stage_delete()
now takes this branch's __cache_lock, preserving the multi-worker
thread-safety the locked removal in the old delete() provided.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 28, 2026
Merged
Merged
Merged
Merged
Pfannkuchensack added a commit to Pfannkuchensack/InvokeAI that referenced this pull request Jul 28, 2026
@Pfannkuchensack
Merge remote-tracking branch 'upstream/main' into feat/krea2-turbo-su…
da459b5 
…pport

Resolves conflicts with upstream video generation (invoke-ai#9163), Ideogram 4
(invoke-ai#9303), T5 GGUF encoder (invoke-ai#9324) and the Qwen VAE device fix (invoke-ai#9373).

Notable resolutions:
- qwen_image_latents_to_image: keep the as_qwen_image_vae() reinterpretation
  but adopt upstream's vae_info.compute_device fix (invoke-ai#9373)
- graphBuilderUtils: keep the allow-list isMainModelWithoutUnet predicate,
  which covers wan_model_loader automatically
- generationSettingsVisibility: add 'wan' and 'ideogram-4' to
Pfannkuchensack added a commit to Pfannkuchensack/InvokeAI that referenced this pull request Jul 28, 2026
@Pfannkuchensack
Merge branch 'feat/krea2-turbo-support' from origin
e020019 
Brings in the newer Krea-2 fixes (seed variance calibration, metadata
recall ranges, diffusion_model LoRA layout) on top of the local merge of
upstream video generation (invoke-ai#9163).

Conflict resolutions:
- Took origin's ordering/formatting for main.py, factory.py and
  qwen_image_latents_to_image.py (content-identical)
- Kept the local side wherever Wan/video code is involved: starter model
  bundles, BASES_WITHOUT_STANDARD_SCHEDULER, and the isWan guards plus
  ParamWanModelSelects block in AdvancedSettingsAccordion
Pfannkuchensack added a commit to Pfannkuchensack/InvokeAI that referenced this pull request Jul 28, 2026
@Pfannkuchensack
Merge remote-tracking branch 'upstream/main' into feature/flux2-dev-s…
b45f4d3 
…upport

Resolves conflicts with upstream video generation (invoke-ai#9163) and Ideogram 4
(invoke-ai#9303).

Conflict resolutions:
- Kept both new encoder types side by side (mistral_encoder for FLUX.2 [dev],
  wan_t5_encoder for Wan 2.2) across taxonomy, invocation fields, model
  manager and node types
- isNonCommercialMainModelConfig now covers FLUX dev, FLUX.2 Klein 9B,
  FLUX.2 dev and Ideogram 4
- Regenerated uv.lock from the merged pyproject (adds imageio/imageio-ffmpeg)
Pfannkuchensack added a commit to Pfannkuchensack/InvokeAI that referenced this pull request Jul 28, 2026
@Pfannkuchensack
Merge remote-tracking branch 'origin/feature/flux2-dev-support'
1f34e62 
Brings in the round-2 review fixes and the tightened mistral-common pin on
top of the local merge of upstream video generation (invoke-ai#9163).

Conflict resolutions:
- modelSelected: kept the Wan 2.2 auto-default block alongside origin's
  reworded FLUX.2 variant-switch comment covering both encoder slots
@lstein lstein mentioned this pull request Jul 31, 2026
Merged
3 tasks
@joshistoast joshistoast mentioned this pull request Jul 31, 2026
Merged
5 tasks
lstein added a commit to lstein/InvokeAI that referenced this pull request Aug 2, 2026
@lstein @claude
fix(ui): seed Wan params fields in the v3->v4 persist migration
37f5776 
The Wan video PR (invoke-ai#9163) added five keys to zParamsState
(wanTransformerLowNoise, wanComponentSource, wanVaeModel,
wanT5EncoderModel, wanGuidanceScaleLowNoise) while the persisted params
schema was still at _version 3, without a version bump or migration
seed. The keys are .nullable() with no .default(), which zod treats as
required, and migrate() ends with zParamsState.parse() whose failure
makes the store silently replace the slice with its initial state.

Released v6.13.x builds write v3 blobs without these keys, so any user
upgrading from a release to a build containing Wan loses their entire
params slice (prompts, prompt history, model selection, dimensions,
generation settings) on first launch. Dev machines don't reproduce it
because v3 blobs written after the Wan merge already carry the keys,
and the migration test fixtures spread getInitialParamsState() which
carries them too.

Seed the five keys conditionally (?? null) in the v3->v4 step so
released-build blobs migrate cleanly while dev-build blobs keep any
values they already hold. Add a field-accurate released-build v3
fixture that fails without the seed, plus a test that existing Wan
values survive.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 6, 2026
Open
Merged
lstein added a commit to lstein/InvokeAI that referenced this pull request Aug 8, 2026
@lstein @claude
fix(ui): pin the conditional half of the Wan v3->v4 seeds
257e28a 
The Wan video PR (invoke-ai#9163) added five keys to zParamsState
(wanTransformerLowNoise, wanComponentSource, wanVaeModel,
wanT5EncoderModel, wanGuidanceScaleLowNoise) while the persisted params
schema was still at _version 3, without a version bump or migration
seed. The keys are .nullable() with no .default(), which zod treats as
required, and migrate() ends with zParamsState.parse() whose failure
makes the store silently replace the slice with its initial state.

Released v6.13.x builds write v3 blobs without these keys, so any user
upgrading from a release to a build containing Wan loses their entire
params slice (prompts, prompt history, model selection, dimensions,
generation settings) on first launch. Dev machines don't reproduce it
because v3 blobs written after the Wan merge already carry the keys.

The seeds themselves are no longer this commit's job: they reached main
with the FLUX.2 [dev] merge (f10d2a4), together with a field-accurate
released-build v3 fixture that fails without them. What main does not
cover is the other half of the contract — that the seeds are written
with `??` rather than assigned, so a v3 blob from a dev build after the
Wan merge keeps the values it already holds instead of having them reset
to null. Add that test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lstein added a commit that referenced this pull request Aug 8, 2026
@lstein @claude
fix(ui): seed Wan params fields in the v3->v4 persist migration (rele…
9f6610b 
…ase-upgrade params wipe) (#9408)

* fix(ui): pin the conditional half of the Wan v3->v4 seeds

The Wan video PR (#9163) added five keys to zParamsState
(wanTransformerLowNoise, wanComponentSource, wanVaeModel,
wanT5EncoderModel, wanGuidanceScaleLowNoise) while the persisted params
schema was still at _version 3, without a version bump or migration
seed. The keys are .nullable() with no .default(), which zod treats as
required, and migrate() ends with zParamsState.parse() whose failure
makes the store silently replace the slice with its initial state.

Released v6.13.x builds write v3 blobs without these keys, so any user
upgrading from a release to a build containing Wan loses their entire
params slice (prompts, prompt history, model selection, dimensions,
generation settings) on first launch. Dev machines don't reproduce it
because v3 blobs written after the Wan merge already carry the keys.

The seeds themselves are no longer this commit's job: they reached main
with the FLUX.2 [dev] merge (f10d2a4), together with a field-accurate
released-build v3 fixture that fails without them. What main does not
cover is the other half of the contract — that the seeds are written
with `??` rather than assigned, so a v3 blob from a dev build after the
Wan merge keeps the values it already holds instead of having them reset
to null. Add that test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ui): seed post-v3 params fields and add a migration safety net

Addresses review feedback on #9408.

Finding 1 (incomplete fix): the v3->v4 Wan seeds fixed upgrades from
v6.13.x, but releases v6.10.0 - v6.12.0 persist _version 2 blobs, and 15
keys added to zParamsState after v3 was cut are required (no .default(),
.optional() or .catch()) and seeded nowhere: fluxDype{Preset,Scale,
Exponent}, zImageShift, zImageSeedVariance{Enabled,Strength,
RandomizePercent}, anima{VaeModel,Qwen3EncoderModel,Scheduler},
klein{VaeModel,Qwen3EncoderModel} and qwenImage{ComponentSource,
Quantization,Shift}. Seed them conditionally in the v2->v3 step, so
released v2 blobs migrate cleanly and dev-build v2 blobs keep the values
they already hold. Verified by running the real migrate() over a blob
built from the v6.10.0 release key set: it threw on exactly those 15
paths before this change.

Finding 2 (tests can't catch the next occurrence): the fixtures spread
getInitialParamsState(), so they carry every current key and are inert
against the general defect. Replace them with the top-level zParamsState
key sets as actually shipped, read out of the release tags and checked
in, one per persisted version still in the wild (v6.10.0 for v2 and
v6.13.7 for v3 - each the narrowest key set among the releases writing
that version, so a subset of every real blob). Add a schema-completeness
test that runs the version steps over each release blob and asserts no
key of the current schema is left unhandled, naming the offending keys
and the step to fix. It fails on any future required-no-default key
added without a seed.

Finding 3 (fail-open-and-destroy): a single missing key made
zParamsState.parse() throw, and the caller in store.ts falls back to the
initial state, wiping prompts, model selection and dimensions with only
a log.warn. Add backfillMissingParamsKeys(): after the version steps,
fill any key that is absent and that the schema cannot fill itself, and
warn with the key names. Narrow by design - a key that is present but
invalid still throws, and anything with a .default()/.catch()/.optional()
is left to zod. So a forgotten seed now costs one field at its default
instead of the user's whole params slice. The completeness test above
deliberately bypasses the net so it still fails CI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ui): cover the oldest v2 releases and harden the migration edges

Follow-up from an adversarial review of the previous commit.

The v2 fixture was not the narrowest v2 release. The earlier survey
globbed tags as v6.1*, which silently excluded v6.7.0 - v6.9.0 — four
stable releases that also persist _version 2, with only 46 keys against
v6.10.0's 52. Six further keys are required-with-no-default and seeded
nowhere: fluxScheduler, zImageScheduler, colorCompensation,
zImageVaeModel, zImageQwen3EncoderModel and zImageQwen3SourceModel.
Verified by running the version steps over a v6.7.0-shaped blob: parse()
throws on exactly those six. Seed them in the v2 -> v3 step and replace
the fixture with the true narrowest set (v6.7.0, confirmed a strict
subset of v6.10.0/v6.11.x/v6.12.0). Add a v6.6.0 (_version 1) fixture
too; it is the v6.7.0 set minus positivePromptHistory, which the v1 step
already seeds.

Also close three edges the safety net did not cover:

- The v0 step dereferenced state.dimensions.rect unguarded, so a blob
  lacking dimensions threw a TypeError straight out of migrate() — the
  one remaining path that could still wipe the slice. Guard it and let
  the backfill repair dimensions instead.
- The v0 branch tested key presence (!('_version' in state)) while the
  backfill tests value (!== undefined). A blob with an explicit
  undefined _version matched no branch, reached the parse and took the
  slice down. Detect v0 by value so the two agree.
- Exclude _version from the backfill loop, so a future change cannot
  turn it into a version-detection bypass that stamps a blob current
  without running a single step.

Each fix is mutation-checked: reverting any one of them fails at least
one test, and the six seeds fail the schema-completeness test.

Not covered: v6.2.0a1 - v6.5.1 persist a blob with no _version at all
and predate the current dimensions shape, so a faithful fixture can't be
built by filtering getInitialParamsState(). Noted in the test file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ui): close the v4 and v0 gaps in the params migration invariant

Addresses the four follow-ups from Pfannkuchensack's second review.

The v4 tier was unguarded: the PiD fields landed a day after the
_version 3 -> 4 bump, so dev builds from that window persist v4 blobs
without them, and a v4 blob matches no branch in the migration chain.
Give the four fields zod defaults, matching the ernieImage* precedent
set by the other two post-bump additions, and pin a RELEASE_PARAMS_KEYS
entry to the bump commit's key set so the invariant covers the tier no
version step can reach.

Add v0 fixtures. The claimed v0 range was wrong: it spans v6.0.0a1 -
v6.6.0rc2, and the oldest builds have no `dimensions` key at all, which
no step seeded — the invariant only held there because the safety net
caught it. Seed `dimensions` in the v0 step and cover both v0 shapes.

Widen the safety net from omissions to any key whose persisted value
fails its own field schema, so a `dimensions` the v0 guard left
incomplete, or a `model` whose base has since left zBaseModelType,
costs that one field instead of the whole slice. This makes true what
the guard's comment already claimed.

Fix the comments that still described the presence check this branch
replaced with a value check.

Also close three holes in the tests themselves, all found by mutating
the production code and watching nothing fail: the _version guard test
never reached the guard (the version steps normalise _version first),
the PiD test could not distinguish the new defaults from the safety net
backfilling the same values, and the fixtures carried initial values
throughout — including `model: null`, so nothing noticed a model being
silently cleared.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(ui): give the HiDiffusion params fields zod defaults

HiDiffusion (#8787) landed in main while this PR was in review, adding
five keys to zParamsState after the _version 3 -> 4 bump. Keys added
after a bump land in a tier the migration chain cannot reach — a v4
blob matches no branch — so they were seeded by an ad-hoc block inside
migrate() instead.

That works, but it sits outside applyParamsVersionMigrations(), so the
completeness invariant added by this PR cannot see it and reports the
five keys as unseeded. Give them zod defaults, the same route the
ERNIE-Image and PiD fields take, and drop the now-redundant block: the
defaults carry the identical values, and the repair pass covers the
blob before the parse either way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(ui): close three inert spots in the params migration suite

Found by mutation-testing this branch's own suite: 116 single-line
mutations of the production code, each run against the tests. Three
classes of mutation left the suite green.

Conditional seeds. Turning `state.X = state.X ?? V` into `state.X = V`
went undetected for 21 of the 31 conditional seeds, because only 10 were
probed by the two hand-written "preserves ... dev-build" tests. The `??`
is the entire point of those lines: the field landed in the schema
before the bump that seeds it, so a blob from that window already holds
a real user value, and an unconditional assignment resets it. Replace
the two tests with a table covering every conditional seed, one probe
per key. The table self-validates — each probe must satisfy the field's
own schema and must differ from what the step would seed — so a probe
that stops discriminating fails rather than going quiet. Verified: seven
representative mutations now fail, each naming its own key.

Fixture erosion. `buildReleaseBlob` sources values from
getInitialParamsState(), so a fixture key that leaves zParamsState is
dropped from the blob silently while still sitting in the table, and the
fixture stops reproducing the shape it names. That already happened:
kleinVaeModel left the schema when the v4 -> v5 step folded it into
flux2VaeModel, and it is the input to that fold. Add
REMOVED_SCHEMA_KEY_VALUES so removed keys are still reproduced, plus a
guard test that fails on any fixture key in neither the schema nor that
table.

Post-bump defaults. Three mechanisms now write the PiD fields with
identical values — the v3 -> v4 seed, the v4 -> v5 seed and the zod
default — so any one could be reverted with the other two covering for
it, and the test that claims to prove the defaults could not see it.
Pin the property directly: every key added after the last bump must
satisfy `shape[key].safeParse(undefined)`, which is what lets it survive
on a tier that has no migration step.

Also assert the version steps leave no key holding a schema-rejecting
value, and note in the completeness test what it does not cover: fixture
values are initial values, so a tightened *nested* schema still costs a
top-level key via `reset` without any fixture noticing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
lstein added a commit that referenced this pull request Aug 9, 2026
@lstein @claude @JPPhoto
fix(api): run single-video delete/update off the event loop (#9392)
c7de47b 
`delete_video` and `update_video` were declared `async def` while doing
blocking SQLite and filesystem work. FastAPI runs an `async def` handler
directly on the event loop, so each call stalled every other request and
socket event for its duration. Their batch siblings
(`delete_videos_from_list`, `star_videos_in_list`, `unstar_videos_in_list`,
`delete_uncategorized_videos`) were already converted to sync `def` — which
FastAPI offloads to the threadpool — during the #9163 review; these two
single-item routes were deferred to this follow-on.

Deferred non-blocker from PR #9163.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com>
lstein added a commit that referenced this pull request Aug 9, 2026
@lstein @claude @JPPhoto
fix(video): bound iter_video_frames by one deadline, not two (#9393)
7b82f29 
`iter_video_frames` waited up to `timeout` for a decoder/stream slot and
then handed `_iter_video_frames_unbounded` a fresh full `timeout` for the
first frame, so a call could take nearly 2x the bound its caller thought it
was enforcing (upload probing and the video nodes both size their budget on
that value).

The capacity wait is now charged against the same deadline as the first
frame — the remaining-time-after-acquire pattern `_run_worker` already uses.
Frames after the first still get a full `timeout` each: past the first frame
the budget is a decoder-inactivity bound, not a queueing one, and shrinking
it would kill legitimate long decodes that had queued for a slot.

Deferred non-blocker from PR #9163.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com>
lstein added a commit that referenced this pull request Aug 9, 2026
@lstein @claude @JPPhoto
perf(boards): batch the owner lookup in admin board listings (#9395)
45ad17b 
`BoardService.get_many` / `get_all` ran one `users.get` per board to resolve
the owner display name shown to admins — 50 boards meant 50 extra queries for
what is usually a handful of distinct owners. Adds `UserService.get_many`
(one `IN (...)` query, deduped and chunked under SQLite's bound-parameter
limit) and folds the two duplicated DTO-building loops into a single
`_to_dtos` helper that fetches media summaries and owner names once per page.

Non-admin listings never show owner names, so they now skip the lookup
entirely instead of relying on `is_admin` inside the loop.

The other 4-queries-per-board half of this finding was already fixed in #9163
itself (`gallery.get_board_media_summaries` batches covers and counts into one
windowed query); this is the residual.

Deferred non-blocker from PR #9163.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com>
@DustyShoe DustyShoe mentioned this pull request Aug 20, 2026
Merged
5 tasks
@lstein lstein mentioned this pull request Aug 24, 2026
Merged
lstein added a commit that referenced this pull request Aug 25, 2026
@lstein @claude @JPPhoto
fix(api): report partial failures and bound batch bodies on image rou…
1ea7771 
…tes (#9394)

* fix(api): report partial failures and bound batch bodies on image routes

Three related gaps on the image endpoints, all of which the video endpoints
already handle (they were fixed there during the #9163 review):

1. `star_images_in_list` / `unstar_images_in_list` re-raised the first
   HTTPException mid-batch, so one foreign name discarded the response payload
   for images that HAD been starred — the client never invalidated their caches
   and the UI showed them unstarred until the next full refresh. They now skip
   foreign/missing names like `delete_images_from_list` does, and dedup repeated
   names so one name can't land in two result buckets.

2. Those same handlers swallowed genuine storage failures with `except
   Exception: pass`, reporting a success-shaped response for images that were
   never updated. `StarredImagesResult` / `UnstarredImagesResult` gain
   `failed_images` (mirroring `DeleteImagesResult` and the video models), and the
   frontend toasts a partial-failure warning like the video star/unstar
   mutations do.

3. The `image_names` batch bodies (delete/star/unstar/images_by_names) were
   unbounded, and `list_image_dtos` had no pagination bounds — a negative LIMIT
   means *unlimited* in SQLite. Adds MAX_IMAGE_BATCH_SIZE (mirroring
   MAX_VIDEO_BATCH_SIZE), a 255-char per-name cap, and ge=0/le=MAX_PAGE_SIZE on
   the list route. The lower bound on `limit` is 0, not 1: the frontend issues
   count-only queries with limit=0.

Deferred non-blocker from PR #9163.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: regenerate openapi.json for the new failed_images fields and batch bounds

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(api): bound the /images/download name list too

`/download` was the one explicit-name batch route the bounds pass missed. It
accepts `image_names`, authorizes every name individually, then schedules the
bulk-download background task, so an authenticated client could still submit an
oversized body and buy a per-name DB lookup each.

Applies the same `ImageName` / `MAX_IMAGE_BATCH_SIZE` constraints the other four
routes already use. Rejection is FastAPI request validation, so it lands before
any authorization lookup or background task.

Adds `/download` and `/images_by_names` to the existing bounds test, and a drift
guard that walks the published OpenAPI schema and fails if any `/v1/images`
request body takes an `image_names` array without both a list bound and a
per-name length bound — the limits were applied route-by-route, which is how
`/download` was missed in the first place.

Scoped to the images router deliberately: `/v1/board_images/batch` and
`/batch/delete` are unbounded too, but bounding them would reject a
change-board request the UI can produce today, so they need to be paired with
client-side chunking in a follow-up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(api,ui): bound every image_names batch body, chunk oversized ones client-side

The bounds pass applied its limit route-by-route, which left two gaps.

Backend: `/images/download` accepted `image_names` unbounded, authorized every
name individually, then scheduled the bulk-download task. `/board_images/batch`
and `/batch/delete` were unbounded too, and loop per name for permission checks.
All three now carry the same `MAX_IMAGE_BATCH_SIZE` bound as the rest.

That constant's comment understated the cost it guards. The authorization helpers
short-circuit on the first hit, so an admin or a direct owner costs 0-1 queries per
name -- but a user reading someone else's Shared/Public board falls through to
`boards.get_dto()`, which is six queries including three COUNT aggregates over the
board's contents. Since the routes are `async def` and the loop is synchronous,
that work blocks the event loop. Hence one uniform bound, with no route granted a
laxer one.

Frontend: nothing capped a gallery *selection*. Select-all reads the whole board's
name list, so one keystroke on a large board produced a selection an order of
magnitude past the bound, and no batch call chunked. Delete was the worst case --
`handleDeletions` swallows the rejection, so an oversized delete silently did
nothing. All seven batch calls now split oversized bodies into conforming requests.
The five mutating ones merge the per-chunk results so callers and `invalidatesTags`
still see one aggregate result; `images_by_names` concatenates (a plain ordered
list, and its caller only upserts by name); `/download` cannot merge, so an
oversized selection becomes several zips -- the socket handler already fetches per
`bulk_download_complete` event, keyed on the event's item name.

Chunks run sequentially: each is already up to 1000 names of DB work, and firing
them concurrently would hand back exactly what the bound took away.

A mid-run failure resolves to a partial success, not an error. The earlier chunks
are already committed, and a bare error would discard their payload -- the very bug
the partial-failure reporting on these routes exists to fix. It is not only the RTK
cache at stake: `handleDeletions` drives the gallery selection and strips deleted
images out of nodes, canvas layers and reference images off `deleted_images`, and
none of that runs on a rejection. So the merged result is returned, the unreached
names are toasted as failures, and only a run where nothing landed is an error.

Tests: the bounds test covers all seven batch routes; a drift guard walks the
published OpenAPI schema and fails if any `image_names` body ships without both a
list bound and a per-name length bound, pinning the exact route set so a route the
walk *skips* cannot pass unnoticed. Frontend unit tests cover chunk splitting,
result merging, and both failure paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(api): board batch moves skip foreign names instead of aborting

Extends §1 of this PR to the two routes it missed. `add_images_to_board` and
`remove_images_from_board` did `except HTTPException: raise` inside the per-name
loop, so one name the caller doesn't own — or one deleted by a concurrent session
— discarded the response payload for every image that had *already* been moved in
the same request. Those moves are committed; only the report is lost, so the
client never invalidated their caches and the UI kept showing them on their old
board until a full refresh.

Both now skip such names and dedup repeated ones, matching star/unstar/delete.
`AddImagesToBoardResult` / `RemoveImagesFromBoardResult` gain the `failed_images`
list the other batch results already carry, populated for genuine storage failures
only — an auth skip is not a failure and must not be toasted as one. The field is
required rather than defaulted because the client toasts off it, and an optional
one reaches TypeScript as `undefined`; the single-image routes pass an empty list,
where a failure is a 500 and never a partial success. `DeleteImagesResult` is
tightened the same way, and delete finally toasts its partial failures — it never
did, and `handleDeletions` swallows every outcome, so a delete that only partly
landed said nothing at all.

Skipping removes the early abort that used to cap an unauthorized batch at one
check, so `remove_images_from_board` memoizes board write-access per board id.
`_assert_board_write_access` goes through `boards.get_dto()` — six queries,
three of them COUNT aggregates over the board's contents — and both routes are
`async def` with synchronous DB calls, so unmemoized a 1000-name batch on one
board is ~6000 blocking queries on the event loop: exactly what the bound in the
previous commit exists to prevent. The check sits outside the per-name try so
there is precisely one way to skip a name for authorization; folding it in left
two paths to the same outcome and neither was individually load-bearing.

`remove_images_from_board` resolves the DTO in its own block, narrowed to
`ImageRecordNotFoundException`. It is the one route that reads the DTO *before*
any authorization check, so an image deleted between the client building its
selection and this request would otherwise be indistinguishable from a storage
failure and toasted as one — while a real storage error must still reach
`failed_images`. The maintenance pre-check loop skips the same exception: raised
from inside an `except HTTPException:` handler it would replace the 409 with a
500.

The authorization guarantees are unchanged — only the reporting is.
`test_non_owner_cannot_batch_add_other_users_images_to_own_board` is updated for
the new shape and asserts the move was never *attempted*, since `board_images` is
a MagicMock in that fixture and asserting on `board_image_records` would pass no
matter what the route did. Three new tests cover the remove side, which had no
authorization test at all — and could not have had one: the fixture left `urls`
as None, so `ImageService.get_dto` raised `AttributeError` for every image and
every name was skipped before the ownership check ran, passing regardless of what
the route did.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(api): treat a mid-batch not-found as a skip, not a storage failure

`/images/delete`, `/star` and `/unstar` reported an `ImageRecordNotFoundException`
raised after the ownership check in `failed_images`, so an image deleted by a
concurrent session between the client building its selection and the request
landing toasted "1 image could not be updated" for an outcome the user actually
got. `remove_images_from_board` already resolves the same race as a skip; these
three now match it, and the name is absent from both result lists.

Star/unstar reach the race through the `get_dto` read-back inside
`ImageService.update`: the UPDATE matches no row and raises nothing, so a name
that vanished mid-batch surfaces only on the read that follows.

The skip is only sound if the exception means what its name says, and it did
not: `image_records.get()` re-raised every `sqlite3.Error` as
`ImageRecordNotFoundException`, so a locked, corrupt or unreadable database was
indistinguishable from a concurrent delete. Under the new skip that would have
turned a wholly failed batch into 200 with two empty lists and no toast at all —
strictly worse than the spurious warning the skip removes. The translation is
dropped in `get()` and `get_metadata()`; storage errors now propagate as
themselves, which also un-swallows them for the two existing skips in
`board_images.py`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ui): don't report a partially-scheduled bulk download as failed

The chunked `/images/download` loop returned a bare error on any chunk failure.
The route answers 202 the moment it has scheduled the background task, so every
chunk before the failing one is already producing a zip: the user saw "Problem
preparing download" from the `matchRejected` listener while those zips landed in
their downloads anyway.

It now follows `buildChunkedImageBatchQueryFn` — only a run where nothing was
scheduled surfaces as an error; a partial run resolves with the first chunk's
payload and warns with the count of names that made it into no zip. The warning
has its own toast id, since the toast system updates in place and sharing
`IMAGES_FAILED_TO_UPDATE` would let one count replace the other.

"Something was scheduled" is tracked in its own flag rather than inferred from
the payload: `fetchBaseQuery` resolves an empty response entity as `data: null`,
so a 202 whose body did not survive the trip back leaves nothing to return even
though the task was scheduled.

The `queryFn` is extracted as `bulkDownloadQueryFn` so it can be tested.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(api): decide board write access per name, and report what it cannot decide

The batch remove route memoized the write-access decision for the whole request. A board
flipped from Public to Private mid-batch therefore kept accepting removals from a contributor
whose permission had just been revoked, for the rest of the names. The add route had the same
window in a different shape: one check on the target board before a 1000-name loop.

Both now decide for every name. That is only affordable because the decision reads the board
record rather than its DTO -- ownership and visibility are two columns, while boards.get_dto()
also resolves a cover image and runs three COUNT aggregates over the board's contents.

Doing so exposed two more problems in the same path:

- SqliteBoardRecordStorage.get translated every sqlite3.Error into BoardRecordNotFoundException,
  the same exception a board that does not exist raises. A decision taken once per request could
  only turn that into a visible 404; taken per name it silently dropped names out of the
  response -- absent from added/removed, absent from failed_images, no toast -- and the client
  went on showing them as moved. The translation is gone, as it already is for image records,
  and a name whose decision could not be taken is now reported rather than skipped.

- remove_image_from_board deleted by image_name alone, so the decision followed the image rather
  than the board it was taken about: authorize against a public board, have the image moved to a
  private one in between, and the delete lands on the private board. The predicate is now on the
  write.

Also drops an unreachable ImageRecordNotFoundException clause on the add path -- nothing in that
block raises it, and the deleted-mid-batch case it was meant for arrives as a foreign-key error,
which the record probe below already classifies.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ui): keep a chunked batch inside the session that started it

Requests read the bearer token out of localStorage when they are sent, so a selection split
across several chunks carries no identity of its own. Log out and back in as someone else while
one is running and the remaining chunks are applied as that user -- on a public board those
writes land, committing half of one user's delete or board move under another's name, with
nothing to roll it back. RTK Query's resetApiState does not help: it clears the store, not a
queryFn that is already running.

The session is captured before the first chunk and rechecked before each one. It is compared by
the identity the token carries, not by the token and not by the auth generation counter. Not the
bytes, because the sliding-window refresh mints a new token for the same login mid-batch. Not
the counter, because beginAuthTransition bumps it when a login or logout request is *sent* --
before anything has changed, and whether or not it succeeds -- so a second tab visiting the
login page would abort an unrelated batch in this one.

The read path also rechecks after the response, before publishing: those DTOs were fetched as
whoever was logged in when the chunk went out, and the store they would be written into may
since have been reset for someone else.

Two fixes to the download path while here:

- A body carrying both board_id and image_names is a board download, not a selection to split.
  The server prefers board_id, so chunking the names scheduled the same full-board zip once per
  chunk. Normalized to the single request the server will honour.

- A 202 whose body did not survive the trip back leaves no item name. The fulfilled listener
  dereferenced it, and reading it through optionals alone would be no better: the toast is
  persistent and is dismissed by name when the zip lands, so a keyless one gets a random id that
  the socket handler can never match, leaving a "preparing" banner up forever for a download that
  already arrived. With no name there is now no toast. This is also the case the mid-run return
  had been failing tsc over, which is why frontend-checks has been red.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ui): wait for the board move before clearing the change-board selection

The image batch mutation was fired and forgotten. ConfirmationAlertDialog calls acceptCallback
and then onClose without awaiting, so changeBoardReset ran while the request was still in
flight and took its failed_images with it: the names that did not move were cleared along with
the ones that did, leaving nothing to retry from.

It is now awaited alongside the video promises, and the names the server could not move stay
selected. A whole-request rejection retains the full selection, since nothing moved -- note
that nothing toasts those rejections today, neither the endpoint's handler nor a matchRejected
listener, which is unchanged here.

Every one of those writes lands after an unbounded await, though, so each goes through
canRetainFailedSelection first. By then the user may have reopened the dialog on a different
selection -- overwriting it would move a set they never chose to the board they picked for
something else -- or the session may have ended, and the logout listener clears this slice
deliberately. The success-path reset is guarded too; it had the same exposure already.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ui): guard async change-board results

* fix(ui): report a failed video move regardless of who owns the modal

The ownership guard added in 46c47b2 sits ahead of the VIDEOS_FAILED_TO_MOVE
toast, which is the only failure report the video board routes have: they have
no `onQueryStarted` handler and no `matchRejected` listener, unlike the image
batch routes, which now toast from the endpoint and so are unaffected by
anything the modal decides. Open and cancel any second dialog while a video
move is in flight and the guard refuses, so a move that failed says nothing at
all. Reported ahead of the guard now, gated on the session alone -- the guard
protects a shared slice from a stale write, it does not decide who is told
about a request they started themselves.

The two halves of that commit were also invisible to the tests: deleting the
rejection branch from all five `onQueryStarted` handlers, or reading
`operationId` after the await (which makes the guard compare the current value
against itself), both left the suite green. The five identical handlers are now
one exported `reportImageBatchOutcome`, tested on both branches, with a source
guard counting its wiring against the chunked endpoints so a sixth one cannot
forget it; the capture ordering and the toast ordering get source guards in the
manner of the ones already in this file.

Also corrects the new `canRetainFailedSelection` docstring: the stale retain
cannot reach a wrong-board move, because all four openers re-seed the selection
before showing the dialog.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ui): close the holes an adversarial pass found in the new guards

An adversarial review of 55b370e built working implementations that defeat all
three source-level guards it added, and falsified two claims in its comments.

- The wiring count matched `queryFn: buildChunkedImageBatchQueryFn(`, so
  hoisting the call to a const was enough to add a sixth chunked mutation that
  swallows every failure and still pass. Matched wherever the call appears now.
- The capture-ordering guard only pinned the line order, so re-reading the slice
  and passing today's value as both operands left it green while admitting every
  stale operation. The call now has to hand the guard the captured constant.
- The video-toast guard only pinned position, so re-checking ownership inside
  the toast's own condition put it back behind the guard by another route. The
  toast's gate is now asserted not to mention the operation at all.

Each is verified against the implementation that defeated its predecessor.

Two comments were also wrong. `settleVideoBoardMutations` (the drag-and-drop
path) emits the same VIDEOS_FAILED_TO_MOVE id for the same two routes, so this
toast is not "the only failure report the video board routes have" — it is the
only one for a move made from this dialog, since that helper settles only the
mutations it fired itself. And a rejection reaching `reportImageBatchOutcome`
is not unconditionally a nothing-committed run: the mid-run path can throw out
of `getTags(merged)` or `merged.failed_images.concat(...)`, both of which read
keys off a server payload, which over-counts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(images): prevent stale batch results across sessions

* test(images): ignore unordered delete response

* test(ui): bound the reopen guards, and correct two stale comments

The two guards added for the retry reopen match through an unbounded
`[\s\S]*`, and this file holds two reopens — so the image assertion is
satisfied by the *video* reopen further down. Deleting the image reopen, the
half that matters most here since image batches are what these routes are
about, leaves the suite green. Both are matched adjacently now, and each was
checked by deleting its own reopen and confirming only its own test fails.

`reportImageBatchOutcome`'s docstring still said a rejection can only mean
nothing committed. The auth-changed abort falsifies that: it now returns an
error however many chunks landed, deliberately, so the new session cannot
consume the old one's aggregate. That is safe because of the session check in
the handler itself, which is what the docstring should say.

The delete-race comment lost its tie to remove_images_from_board when it
stopped skipping, leaving a divergence a reader would be tempted to "fix".
Recorded why that route cannot follow: its result list feeds
getTagsToInvalidateForImageMutation, so a vanished name would invalidate
getImageDTO for a record that is gone and drive a 404 refetch, and it reads the
DTO before any authorization check, so answering success there would cover
names the caller was never entitled to touch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(api): keep an expired session's 401 a failure, not an abort

The post-response session check in 806f985 rewrites any response that comes
back into a changed session. `dynamicBaseQuery` dispatches sessionExpiredLogout
on a 401 *before* it returns and that reducer clears auth_token synchronously,
so the session has always "changed" by the time the check runs -- every
expired-session 401 came back as an abort instead.

That is not a distinction without a difference. An abort is fatal to the whole
run by design, so a 401 on chunk 2 of a delete skipped the partial-success path
that reports what chunk 1 committed. handleDeletions never saw deleted_images,
and nothing else prunes: gallerySlice handles `logout` only, and canvasSlice,
nodesSlice and refImagesSlice have no logout handling at all, while all of them
are persisted. The canvas, nodes and reference images kept pointing at deleted
images across the next login. On the download path it was worse -- the 401
became `{ data: undefined }`, a fulfilled result, so matchRejected never fired
and the failure toast was lost outright.

Only a successful response is laundered now. An error has no payload to leak
into the next session, and rewriting it destroys what it was.

Four more from the same pass:

- delete_images_from_list reported a name that never existed. assert_image_owner
  returns immediately for an admin -- the default single-user identity -- without
  touching storage, so get_dto's own not-found reached the new deleted_images
  branch. Gated on having actually read the record; a not-found from the read is
  a skip again, and only the delete losing the race is a deletion.
- That branch also dropped affected_boards even though board_id was already
  bound. getDeleteImagesTags derives every board-scoped tag from it and ignores
  deleted_images by design, so the board's counts stayed stale for a name
  reported gone.
- imageDTOsByNamesQueryFn's own check was not redundant: fetchChunk is async, so
  resuming from it is a microtask hop, and a logout landing in that hop passes
  its check and still clears the cache before the upsert. Restored.
- The auto-reopen could show the "select a board" placeholder while still armed
  for the previous target, since `options` drops the board being viewed.

Tests for all of it, plus the gaps the pass proved: dropping the post-response
check entirely left 27/28 green, the bulk-download branch could not tell
"return nothing" from "leak the old item name", the guard in
reportImageBatchOutcome was tested only on the branch an auth change never
takes, and the reopen guards accepted a reopen moved above the session check.
Each fix was reverted individually and its test confirmed to fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(api): triage a failed session check into expiry vs takeover

dbb5297 stopped a chunk's own 401 from being rewritten into an abort, but that
closed one door of three. The token does not only vanish via this chunk's own
response: a 401 on any concurrent request (a gallery poll, a board refetch)
dispatches sessionExpiredLogout, whose reducer clears auth_token synchronously,
at whatever await the batch happens to be parked on. Land it between chunks and
the next pre-request check hard-aborts; land it while a successful chunk is in
flight and the post-response check rewrites that success into an abort. Either
way the committed chunks go unreported and the persisted canvas/nodes/
reference-image slices keep their references across the next login -- the same
bug, through the doors the last fix did not cover. Reproduced: chunks 1-2
committed, token cleared during chunk 2's flight, run returned a bare abort
with no invalidation and no partial payload.

The underlying conflation is that isSameAuthContext goes false for two things
that need opposite treatment. A takeover (someone else's token now in
localStorage) must abort hard: nothing from the old run may be consumed. An
expiry (token dropped, no successor) must degrade into an ordinary chunk
failure: there is no new session to protect, and the partial-success path is
what reports the committed work. So a failed check is now triaged --
SESSION_ENDED_ERROR is not matched by isAuthChangedError and takes the partial
path -- and a successful response is consumed unless another user has actually
taken over, since a same-user expiry dropping its own committed payload would
just un-report work that happened.

The expiry test now asserts the partial reporting it exists to protect (it
only counted calls before, which both behaviors satisfy), the takeover tests
pin the "session changed" wording so the two aborts cannot stand in for each
other, and all four collapse directions were reverted individually and fail
their tests: expiry-as-hard-abort, takeover-as-soft-stop, post-response
aborting on mere expiry, and no post-response check at all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ui): silence downloads on expiry too, and pin the untested triage arm

An adversarial pass on the triage commit proved its takeover arm untestable
from the loops and untested outright: collapsing sessionMismatchError to
always-soft left all 32 tests green, despite the previous commit message
claiming otherwise. The claim was true only of collapsing the triage and the
post-response check together. No loop-level test can do better -- same-tab,
everything from one chunk's post-response check to the next's pre-request
check is a single synchronous drain, so a takeover staged inside a mocked
baseQuery is always caught by the post-response check first. The arm is live
only cross-tab, where another tab's login writes localStorage between chunks,
and it is the guard that stops the next chunk going out as the new user. So
the triage is exported and unit-tested directly on both arms, with the
reachability argument recorded on the function.

The same pass caught bulk downloads applying the opposite policy to expiry:
SESSION_ENDED fell onto the ordinary reporting branches, toasting the failure
count at the login screen and -- via the fulfilled item name -- raising the
duration:null "preparing" toast there, dismissable only by a socket event the
dying session never receives. Downloads now report nothing under either
mismatch flavor. The asymmetry with the mutating loops is deliberate and
documented: they let expiry through to the partial path because handleDeletions
has pruning to do off the payload; a download has no state work, and both of
its outputs are wrong for a session that is ending.

Also trims the reportImageBatchOutcome docstring claim that invalidation "does
the state work" under expiry -- resetApiState has already emptied the store by
then, so it is a no-op and the pruning is the part that matters.

All three fixes reverted individually and their tests confirmed to fail, the
triage in both collapse directions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ui): clear the workspace slices and their undo stacks on account change

Round-6 review blocker: a cross-user takeover mid-batch hard-aborts the run --
correctly, the new session may consume nothing of the old one's -- but that
means handleDeletions never prunes what the committed chunks deleted, and the
persisted canvas, nodes and reference-image slices had no account-change
handling at all. The next account inherited the previous user's workspace
wholesale, stale references to irreversibly deleted images included.

The fix is at the account boundary, not in the abort: canvasSlice, nodesSlice
and refImagesSlice now reset on `logout`, exactly as gallerySlice and
paramsSlice already do. That covers both account-change paths in one place,
because accountAwareRootReducer already funnels a cross-tab foreign-token
adoption through a synthetic logout() reducer pass. sessionExpiredLogout is
deliberately not handled -- a timeout must not destroy unsaved work, and under
expiry the batch loops resolve with partial data precisely so handleDeletions
can prune the same user's deleted references.

The reset alone is not enough for the two undoable slices: their filters keep
cross-slice actions out of history without emptying it, so the previous
account's states stay one ctrl+Z away. accountAwareRootReducer therefore chains
the history clears onto any logout pass -- canvasClearHistory for canvas's
overridden clearHistoryType, redux-undo's default clear for nodes and any
undoable slice added later. Chained in the reducer rather than the logout
listener because the synthetic adoption pass never reaches listeners.

Tests assert present-state resets, empty undo stacks (directly -- with few
seeded actions an undo's target can coincide with the initial state, so the
behavioral check alone cannot see a missing clear), survival across mere
expiry and same-user token refresh, and both change paths. Also adds the
reviewer-suggested drift check pinning IMAGE_BATCH_CHUNK_SIZE to the maxItems
bound the server publishes in openapi.json. Every piece was knocked out
individually and its test confirmed to fail, including wiping on expiry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ui): purge upscale too, spare the mode switch, and read anyOf bounds

An adversarial pass on the account-change purge found the two holes that
matter and four test gaps.

The purge covered three of the four slices the codebase itself tracks image
references in: getImageUsage enumerates nodes, canvas, refImages AND
upscale.upscaleInitialImage, which is persisted and had no logout handling --
and which handleDeletions does not prune either, so the takeover scenario had
no cleanup path for it at all. upscaleSlice now resets on logout with the
other three.

Worse, the wipe escalated a dispatch that is not an account change.
ProtectedRoute dispatches logout() when the app boots with a leftover
multiuser token while the server reports single-user mode -- a mode switch
that keeps the same human at the machine. The new logout cases turned that
into a workspace wipe, and single-user mode accepts the unauthenticated
persist, so 300ms later redux-remember overwrote the stored canvas, workflow
and reference images for good. The mode switch now dispatches a dedicated
staleCredentialsDiscarded action: same credential clearing, same api-cache
reset (what is cached was fetched under multiuser visibility scoping), no
workspace wipe, no history clears. logout() belongs to UserMenu alone, and a
source guard pins ProtectedRoute to the new action -- reverting it is
invisible to every store-level test.

Test gaps, each proven by a mutant the old suite accepted:

- The openapi drift check read only properties.image_names.maxItems, but the
  download body is nullable and carries its bound inside anyOf -- mutating
  that one cap in openapi.json passed. The extractor now recurses into anyOf
  and the schema-count floor is 7, so the anyOf handling regressing back to
  the flat six also fails.
- Expiry survival was pinned only for nodes; wiping the canvas on
  sessionExpiredLogout passed all tests. The keep-the-workspace test now
  covers all four slices and runs for expiry, same-user token refresh, and
  the mode-switch discard.
- Nothing pinned the clears to run AFTER the reset pass: clears moved before
  it passed the suite, yet leave the filtered reset as _latestUnfiltered, so
  the next account's first action pushes the previous account's state into
  past and one ctrl+Z resurrects it. The account-change test now dispatches
  as the new account, undoes, and asserts the reset comes back.

Every fix reverted individually and its test confirmed to fail.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(api,ui): classify the zero-row remove, fail a revoked destination, say when downloads are lost

Round-8 review, three findings, none blockers but all real.

The scoped DELETE in the batch remove reported a removal whatever it matched.
The scoping (round 3) was what kept an authorization decision from following
the image onto a board it was never taken about -- but a zero-row miss then
means the image left the board between the read and the write, and reporting
it removed invalidates the wrong boards while the client counts the name as
done. The service layers now return the row count (the sqlite layer has its
own test: a None return silently restores the old behavior at the route,
since `None == 0` is False), and a miss is classified by where the image went:
moved to another board = failed (the ask is not satisfied, and a retry
re-authorizes against the board it actually sits on); concurrently
uncategorized = removed (the postcondition holds and the invalidation lets
this client's stale view catch up -- safe in removed_images, unlike a deleted
name, because the DTO exists and tag refetches succeed); concurrently deleted
= skip, matching the route's existing treatment of vanished names, since
removed_images would drive a getImageDTO refetch into a 404.

The batch add's per-name destination re-check shared an except arm with the
per-image skips, so a destination board revoked or deleted mid-batch emptied
the rest of the request into a silent 201 -- which the client reads as success
and clears the user's selection over. The destination decision now has its own
arm: refusals are reported as failed, per name, with the loop continuing so
access restored mid-batch lets later names land. A foreign or vanished image
stays a skip; the two refusals mean opposite things.

Expiry mid-download now says what was lost. Scheduled zips keep building
server-side but their completion events fire into a socket the expired session
is tearing down, so nothing will ever offer them; withholding the payload
(round 6) was right, but the silence read as a download that never came. A
finite, dismissible toast now tells the user to re-run after signing in --
scheduled-work-lost expiry only: a takeover stays fully silent for the new
user's sake, a first-chunk 401 stays an ordinary rejection for matchRejected,
and nothing-scheduled expiry has lost nothing. Queue-and-replay after
re-authentication remains the real fix, as a follow-up.

Every fix was reverted individually and its test confirmed to fail, including
the sqlite row-count return and the destination arm reverted to a skip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(api,ui): repair the interruption toast's key, cover the everyday 401, probe positively

An adversarial pass on the round-8 fixes caught the delivered toast not
working at all, its everyday trigger missing, a success manufactured from a
storage error, and an untested service seam.

The interruption toast referenced gallery.downloadsInterrupted while the key
sits under toast. -- with returnNull false, i18next renders the raw key
string. The test could not see it: the i18n mock echoes keys and the
assertion checked only id and status. Reference fixed and the title pinned,
which is what makes a wrong section unrepresentable.

The toast also only fired when the pre-request check between chunks saw the
token gone -- but the everyday expiry arrives as the download chunk's own
401, token already cleared by dynamicBaseQuery, which sailed past the
mismatch check into the partial path: a failure-count toast at the login
screen plus, via the returned item name, the permanent "preparing" toast.
The error branch now also treats any error that came back into a changed
session as a session outcome. The mutating loops deliberately do not do
this -- their own-401 belongs on the partial path, where the payload feeds
handleDeletions -- and the comment says so, so a consistency cleanup cannot
quietly reintroduce it.

In the zero-row remove classification, the existence probe used
_image_record_exists, which answers True on a storage error -- conservative
where True means failed (the add loop), but here True meant removed: a
transient error manufactured a success whose tag-driven getImageDTO refetch
then 404s. The arm now probes image_records.get directly: not-found is the
skip, a storage error propagates to the failed arm, and only a record
positively known to exist is reported removed.

Also: the facade passthrough gets its own test -- CI has no type checker, so
a dropped `return` silently restores None == 0 = False at the route while
the route tests mock the facade and the sqlite test pins the layer below --
and a name already uncategorized short-circuits before the scoped DELETE,
which can never match it, saving the classification's two reads per name.
The revoked-destination expectation in test_multiuser_authorization moves to
the new failed-not-skipped semantics, exercising them through the real
access stack.

Every fix reverted individually and its test confirmed to fail.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(api,ui): classify the single remove's zero-row race, reconcile lost chunks, close the final-chunk expiry window

Three review findings, one per layer of the batch hardening:

- board_images.py: the single-image remove ran the same read-then-scoped-DELETE
  sequence as the batch loop but ignored the row count, so an image that left
  the board between the read and the write was reported removed anyway. The
  batch loop's zero-row classification is extracted into
  _remove_from_board_and_classify and both routes now share it.

- images.ts: a chunk failing with a transport-shaped error (FETCH_ERROR,
  TIMEOUT_ERROR, PARSING_ERROR, 5xx) was treated as proof the chunk applied
  nothing, leaving caches unreconciled when the server had committed and only
  the response was lost. Such chunks are still reported failed - retrying a
  satisfied name is safe - but their tags are now invalidated as if the chunk
  had landed, via a per-endpoint assumeCommitted result so the tags come from
  the same getTags the endpoint publishes.

- bulkDownloadQueryFn: an expiry during the final chunk's await was caught by
  no check (fetchChunk deliberately passes mere expiry through, and there is
  no next iteration's pre-request check), so matchFulfilled raised the
  undismissible 'preparing' toast while the scheduled zips were lost silently.
  A post-loop session check now applies the same triage as the in-loop arm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UE38kzGWoj4w6dYiPqa3CY

* fix(ui): close the self-review's two residual gaps in the lost-chunk reconciliation

- The failed_images DTO invalidation landed only on the single-image
  removeImageFromBoard endpoint, which nothing in the app calls; the live
  batch path classifies the same zero-row MOVED race and left the stale DTO
  in place. getRemoveImagesFromBoardTags now invalidates failed names too.

- assumeCommitted's empty affected_boards reached the global gallery-list
  tags but none of the board-keyed ones, so a lost single-chunk delete left
  every board count stale until something unrelated bumped it. The
  indeterminate-path dispatch now appends the board-keyed tag types
  type-wide, since the lost chunk's boards are unknowable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UE38kzGWoj4w6dYiPqa3CY

* fix(api,ui): abort errored chunks under takeover, and stop laundering storage errors into auth skips

Two round-10 review findings:

- fetchChunk triaged only *successful* responses for takeover, on the theory
  that an error carries nothing the next session could consume. Since the
  indeterminate-error reconciliation, the error path consumes plenty: it
  dispatches as-if-committed invalidations and returns partial aggregates the
  UI applies. An error returning into a taken-over session is now rewritten to
  the auth-changed hard abort, so the loops consume nothing. Mere expiry still
  passes through untriaged - the everyday 401 must keep reaching the partial
  path, where committed work is reported and pruned.

- assert_image_owner wrapped the board-ownership fallback in a bare
  except-pass, so a database error during the board lookup became a 403 - and
  the batch star/unstar loops treat a 403 as a silent auth skip: not applied,
  not reported, nothing toasted. The lookup now reads the board record (owner
  and visibility are all the decision needs), catches only
  BoardRecordNotFoundException, and lets storage errors propagate into each
  loop's failed_images arm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UE38kzGWoj4w6dYiPqa3CY

* test(api): pin the narrowed catch from the not-found side too

An implementation that let BoardRecordNotFoundException propagate alongside the
storage errors would toast a failure for a name whose only problem is that its
board vanished mid-request. The gone-board arm stays a silent auth skip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UE38kzGWoj4w6dYiPqa3CY

* fix(ui): stop a stale 401 from ending the session that replaced it

`dynamicBaseQuery` ended the session on any 401 that carried a token, using
the token captured when the request went out. A request that is still in
flight when someone else takes over the tab — a login here, or one in another
tab, since localStorage is shared — then logs out the user who never issued
it. The 401 is only evidence about the credential that was sent, so the
session ends only while that credential is still the live one.

Byte equality, deliberately the opposite of `isSameAuthContext`: a
sliding-window refresh must not qualify either, because a 401 for the token it
replaced says nothing about the replacement. Nothing is lost by waiting — the
next request carries the live token and its 401 ends the session here.

Also invalidate `failed_images` on star/unstar. `ImageService.update` writes
the record and then reads the DTO back; a failure in that read reports the
name as failed with the row already starred, and invalidating only the
successes leaves the client showing the pre-star value until a full reload.
The remove-from-board helper already does this for the same reason.

* fix(ui): close the paths the self-review found around the new 401 guard

Three things the adversarial pass turned up.

The blocker was only half fixed. `ProtectedRoute` ends the session on a 401
from `getCurrentUser` with no freshness check at all, and `sessionExpiredLogout`
removes `auth_token` from localStorage — which is shared across tabs. So the
same stale 401 still deleted the replacement session's credential: the query
goes out during page load carrying an expired token, another tab logs in, and
the 401 lands before the adoption poll runs. Both sites now ask the same
predicate.

Invalidating `failed_images` on star/unstar hands three components a new way to
discard the user's input. A node image field and both reference-image
components clear their value on ANY query error, so the refetch that the new
invalidation triggers can silently drop a workflow input — and the refetch is
likelier than usual to fail, because a name is in `failed_images` precisely
when a storage failure interrupted its write. Only a 404 proves the image is
gone; the video field was narrowed this way already, and the image side now
has the same predicate.

The new guard also let an empty-string token qualify, which sets no
Authorization header and so proves nothing about any session.

* fix(api,ui): let a client trust the 403 it drops an image reference on

Narrowing the components to a 404 was wrong on its own: a deleted image only
answers 404 for an admin. `assert_image_read_access` decides on
`images.user_id`, which is gone with the row, so in a multiuser deployment a
deleted image is indistinguishable from someone else's and both are refused
403. Requiring 404 would have stranded every deleted image in the workflows
referencing it, which is worse than the over-broad clear it replaced.

So the clients act on 403 as well — and that puts an obligation on the answer.
The read helper laundered every storage error from its board lookup into that
same 403, so an unreadable database presented as a permission decision and
would now take the user's references down with it. It reads the board record
and catches only a positive not-found, matching what `assert_image_owner`
already does on the mutation side; anything undecidable propagates.

* fix(api,ui): key the identity query by its token, and give videos the same answer

The `ProtectedRoute` guard only postponed the logout it was meant to prevent.
The store's token catches up when the poll adopts the new one, the effect
re-runs, and the superseded 401 is still sitting in the cache: `getCurrentUser`
is shared across logins, its argument never changes, it carries no tags, and
the API-state reset a login normally brings is deliberately skipped when the
new token belongs to the same user. On the ordering where the storage event
beats the response — the common one, since it is delivered without a network
round trip — the guard never even delayed it.

Keyed by the token instead, so the adopted session reads its own entry and the
superseded 401 is no longer in hand to act on. The argument is never sent — the
token goes in a header — but it is what the answer is about, and the call site
now cannot forget to pass it: the endpoint's argument type is what enforces it.

Videos get the read-side pair the images just got. Trusting a 403 as "gone" is
what lets a deleted item clear itself, and `isVideoMissingError` accepted only
404 — which no non-admin ever sees for a deleted video — so a deleted video
would have stayed pinned in a workflow field forever. The video read helper
stops laundering storage errors into that same 403.

* fix(api,ui): answer gone and denied differently, instead of guessing at the client

Clearing a reference on a 403 was wrong: revoking access to a shared board
refuses every image on it and every one of them still exists, so a board
flipped to Private would clear the workflow fields pointing at its images, and
flipping it back would not bring them back. Requiring a 404 was also wrong, for
the reason that produced the 403 rule — the ownership decision rests on a
`user_id` that is gone with the row, so a deleted image reached the same
refusal as a foreign one and no non-admin ever saw a 404 for one.

Neither answer was the client's to guess, so the server draws the line. On the
refusal path only, both read helpers ask whether the record is actually there:
absent answers 404, present answers 403. The clients go back to treating 404
alone as gone. The cost is that a caller can now tell an absent image from one
they may not read, which admins could always do, and image names are generated
UUIDs.

That makes `video_records.get` load-bearing, so it stops translating storage
errors into not-found — otherwise a locked database would present as a deleted
video and clear the user's fields. It also un-breaks the staged-delete
recovery, which read that same exception as "the delete committed" and purged
the staged files on a database it merely could not read.

An uncertain delete now refetches the names it could not confirm. Their
references are left in place, since pruning on a guess would discard work over
a request that merely failed, and asking is what settles it: gone answers 404
and the components holding it let go, survived answers with its DTO. Partial by
construction — canvas layers hold names with no DTO query behind them, and
only handleDeletions prunes those (#9533).

* fix(api): stop any failure from wearing the deleted-image answer

Both DTO routes ended `except Exception: raise HTTPException(404)`, so a board
lookup against an unreadable database, or a URL service failure, answered the
same 404 as a missing row. That was survivable while a 404 only meant a stale
cache entry. It is not survivable now that the clients drop the user's
reference on one: a locked database would clear live images out of the
workflows using them.

Only a genuinely missing record answers 404. Narrowed on these two routes
alone, because these are the 404s that are acted on destructively — the media,
metadata and workflow routes keep theirs.

* fix(api,ui): close what the self-review found around the gone/denied split

Four things.

The existence probe read the whole record, so a row this version cannot
deserialize — an enum value written by a newer one — failed exactly as absence
does, and would have reported a live image gone. It is a bare row probe now,
which also drops it to one point SELECT per refused name.

`video_records.get` not translating storage errors had no test at all, and
after the DTO routes were narrowed it is the only thing standing between an
unreadable database and a 404 that clears the user's fields. Pinned from both
ends: the store propagates rather than reporting the row missing, and the
staged-delete recovery keeps the staged files instead of purging them when it
cannot read the record.

Reconciling an uncertain delete was inert for the deletes people actually
perform. Anything up to the batch cap is a single chunk, and a single chunk
that fails reports nothing back, so the endpoint's invalidation never runs on a
result — the only invalidation is the one the queryFn dispatches for the lost
chunk, and it described those names as committed, which is exactly the case the
delete tag set skips DTOs for. It now describes them as unconfirmed too, which
is what they are, so the refetch that settles them actually happens.

Both arms of the reference-image reset now wait for the connection; the
original's used to clear regardless of it.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com>
Co-authored-by: JPPhoto <jpollack@jpollackphoto.com>
lstein added a commit that referenced this pull request Aug 31, 2026
@lstein @claude @JPPhoto
perf(video): stream uploads into a single on-disk copy (#9396)
e012ad3 
* perf(video): stream uploads into a single on-disk copy

Declaring `file: UploadFile` makes Starlette parse the multipart body into its
own spooled temp file before the route runs; the route then copied that into a
named temp file it could hand to ffmpeg and `videos.create`. Every in-flight
upload therefore held TWO full-size copies in temp storage (up to
2 x MAX_UPLOAD_SIZE x MAX_CONCURRENT_VIDEO_UPLOADS = 4 GB) for the whole
probe/thumbnail/create phase. #9163 only shrank the overlap window by closing
the spool right after the copy loop; the spool's own path can't be reused,
because once rolled over it is an unlinked anonymous file with no path.

The route now parses the body itself with python_multipart's streaming parser
(already a FastAPI dependency), writing the `file` part directly into the one
temp file and buffering only the small `metadata` field. Peak temp usage per
upload is halved.

Two behavioral improvements fall out of streaming the body:

- The filename/MIME gate fires from the part headers, so an unsupported file is
  rejected before any of its bytes reach the disk instead of after the whole
  body has been spooled.
- MAX_UPLOAD_SIZE is enforced as the bytes arrive rather than after.

Parsing runs in the thread pool (it writes to disk), and the request schema is
pinned with `openapi_extra` so the documented multipart contract is byte-for-
byte what the `file` + `metadata` parameters generated — the only OpenAPI
change is that the body schema is now inline rather than a `Body_upload_video`
component, which nothing references.

Deferred non-blocker from PR #9163.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: regenerate openapi.json for the inlined upload_video request body

Same fields, types and requiredness — the body schema is now inline rather than
a Body_upload_video component (nothing references it).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(video): close the connection when an upload is answered early

Addresses JPPhoto's review of #9396.

Blocker 1 — a rejection that returns before the body is read released the
upload middleware's concurrency leases while the client was still sending, so
the 429 bound, idle timeout and duration cap stopped applying to it. Draining
the body in the route does not work: FastAPI answers an invalid query string
before the route body runs at all, and holding the drain would pin one of only
MAX_CONCURRENT_VIDEO_UPLOADS slots for the full 30-minute duration cap per
rejection — a cheaper denial of service than the hole it closed (measured: two
slow `Content-Type: text/plain` requests wedged all uploads into 429s).

VideoUploadLimitASGIMiddleware now asks the server to close the connection on
any response sent before the body was read to completion, which ends the
in-flight upload along with the response. This covers its own 401/413/429s and
every early answer beneath it, including FastAPI's query validation. It is
deliberately not seeded from Content-Length: h11 accepts `Content-Length: 0`
alongside `Transfer-Encoding: chunked`, which would let a client suppress the
close and then stream with no lease held and none of the caps applying. The
byte-cap, idle-timeout and duration-cap aborts likewise leave the flag alone,
since they fire precisely because the client is still uploading. Uploads that
finish sending keep their connection (verified against a live uvicorn: two
uploads over one socket).

Blocker 2 — `MultipartParser.finalize()` is a documented no-op that does not
check the parser reached its end state, so a body ending after the file bytes
without the closing boundary looked complete and went on to be probed and
persisted. The parser's `on_end` callback now proves completeness; a truncated
body is a 422. This also catches the parser's silent `max_size` truncation.

Also: a malformed body is a 422 rather than an unhandled 500; a client-side
disconnect is a 400 rather than a 500 raised above the middleware; the
Content-Type media type is compared case-insensitively (RFC 7231); and
`MAX_CONCURRENT_VIDEO_UPLOADS` no longer claims two temp copies per upload.

Per JPPhoto's second suggestion, a sink-backed Starlette parser is not
available (MultiPartParser hardcodes SpooledTemporaryFile), so instead
test_python_multipart_contract_the_upload_route_depends_on pins the three
python_multipart behaviours the hand-rolled parsing rests on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FVXZ215segxmicLV7tNy88

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JPPhoto JPPhoto JPPhoto approved these changes
@blessedcoolant blessedcoolant Awaiting requested review from blessedcoolant blessedcoolant is a code owner
@dunkeroni dunkeroni Awaiting requested review from dunkeroni dunkeroni is a code owner
@Pfannkuchensack Pfannkuchensack Awaiting requested review from Pfannkuchensack Pfannkuchensack is a code owner

Assignees

@Pfannkuchensack Pfannkuchensack

@JPPhoto JPPhoto

Labels

6.14.0 api backend PRs that change backend files CI-CD Continuous integration / Continuous delivery docs PRs that change docs frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files python-deps PRs that change python dependencies python-tests PRs that change python tests Root services PRs that change app services

Projects

Invoke - Community Roadmap
Status: 6.14.x Theme: USER EXPERIENCE

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@lstein @Pfannkuchensack @JPPhoto