diff --git a/docs/src/content/docs/configuration/fp8-storage.mdx b/docs/src/content/docs/configuration/fp8-storage.mdx index e28b68e2400..2a28e070d67 100644 --- a/docs/src/content/docs/configuration/fp8-storage.mdx +++ b/docs/src/content/docs/configuration/fp8-storage.mdx @@ -11,7 +11,7 @@ FP8 Storage cuts a model's VRAM footprint roughly in half by keeping weights on It pairs well with [Low-VRAM mode](/configuration/low-vram-mode/): low-VRAM mode streams layers between RAM and VRAM, while FP8 Storage shrinks the layers themselves. :::caution[For full precision models only] -FP8 Storage only applies to **full precision** checkpoints (FP16 / BF16 / FP32). It is **silently a no-op** for already-quantized formats — **GGUF**, **NF4**, and **int8** checkpoints carry their own storage precision and the loader returns a different module type that the FP8 layer cast does not touch. If your model is already quantized, the toggle has no effect; use the full-precision variant of the model if you want to enable FP8 Storage. +FP8 Storage only applies to **full precision** checkpoints (FP16 / BF16 / FP32). It is **silently a no-op** for already-quantized formats — **GGUF**, **NF4**, **int8**, and [**SDNQ**](/configuration/sdnq-quantization/) checkpoints carry their own storage precision and the loader returns a different module type that the FP8 layer cast does not touch. If your model is already quantized, the toggle has no effect; use the full-precision variant of the model if you want to enable FP8 Storage. ::: ## Requirements diff --git a/docs/src/content/docs/configuration/sdnq-quantization.mdx b/docs/src/content/docs/configuration/sdnq-quantization.mdx new file mode 100644 index 00000000000..921a430fcce --- /dev/null +++ b/docs/src/content/docs/configuration/sdnq-quantization.mdx @@ -0,0 +1,130 @@ +--- +title: SDNQ Quantization +sidebar: + order: 4 +--- + +import { Steps } from '@astrojs/starlight/components'; + +SDNQ ([SD.Next Quantization Engine](https://github.com/Disty0/sdnq)) is a quantization scheme that stores model weights at 4–5 bits with an optional low-rank SVD correction. InvokeAI loads SDNQ-quantized models as full HuggingFace diffusers pipelines and dequantizes weights on the fly during inference, with no extra Python package required. + +## Supported models + +| Model family | Status | Format(s) supported | +| ------------------ | ------ | ---------------------------------------------------- | +| **FLUX.1 schnell / dev** | ✅ | Diffusers pipeline (uint4 + SVD), single-file | +| **FLUX.2 Klein 4B / 9B** | ✅ | Diffusers pipeline (uint4 / int5 mixed, ± SVD), single-file | +| **Z-Image Turbo** | ✅ | Diffusers pipeline (uint4 + SVD), single-file | +| **T5 Encoder** | ✅ | Folder + standalone | +| **Qwen3 Encoder** | ✅ | Folder + standalone | +| **VAE (AutoencoderKL)** | ✅ | Folder | +| **SDXL / SD1 / SD2** | ❌ | Not yet — UNet pipeline conversion outstanding | + +:::caution[SDNQ vs SVDQuant / Nunchaku] +SDNQ ("SD.Next Quantization") and SVDQuant ("[Nunchaku](https://github.com/nunchaku-ai/nunchaku)") both apply SVD low-rank correction to 4-bit weights, but they use **different on-disk formats** and **different inference engines**. A file like `svdq-int4-flux.1-schnell.safetensors` from `mit-han-lab` is a Nunchaku checkpoint and will *not* load through InvokeAI's SDNQ path — it has keys like `qweight`, `wscales`, `smooth`, `lora_up` rather than SDNQ's `weight`, `scale`, `zero_point`, `svd_up`. Look for the `Disty0/...-SDNQ-...` repo prefix on HuggingFace to be sure you're picking up the right format. +::: + +## Memory footprint + +Typical reductions vs. the bfloat16 baseline: + +| Model | bfloat16 | SDNQ uint4 + SVD | Approx. VRAM at inference | +| --------------------------- | -------- | ---------------- | ------------------------- | +| FLUX.1 schnell | ~33 GB | ~15 GB | ~12 GB | +| FLUX.2 Klein 4B (dynamic) | ~8 GB | ~5 GB | ~5 GB | +| FLUX.2 Klein 9B (dynamic + SVD) | ~18 GB | ~13 GB | ~11 GB | +| Z-Image Turbo | ~12 GB | ~5 GB | ~5 GB | + +The actual peak VRAM depends on resolution, batch size, attention backend, and whether [Low-VRAM mode](/configuration/low-vram-mode/) is enabled. + +## Installing SDNQ models + +The easiest way is via the **Starter Models** picker — search for "SDNQ": + + +1. Open the **Model Manager** → **Starter Models** tab. +2. Search for `SDNQ`. +3. Click **Install** on the variant you want (each entry shows the HuggingFace source). + + +To install a different SDNQ model from HuggingFace: + + +1. Open the **Model Manager** → **Add Model** → **HuggingFace** tab. +2. Enter the repo, e.g. `Disty0/FLUX.2-klein-9B-SDNQ-4bit-dynamic-svd-r32`. +3. Click **Install**. The whole pipeline folder downloads (transformer + text encoder + tokenizer + VAE). +:::tip +For very large SDNQ models, you can also pre-download with `huggingface-cli` and then point InvokeAI at the local folder via **Add Model** → **Folder**. +::: + + +InvokeAI auto-detects the SDNQ format from `transformer/quantization_config.json` (the `quant_method: "sdnq"` marker). The Model Manager shows the format as **sdnq** in the model badge once installed. + +## What gets quantized + +Inside a typical SDNQ pipeline folder: + +- **`transformer/`** — diffusion transformer weights (most of the savings come from here). Quantized to uint4 or, for dynamic-mixed exports, a per-layer mix of uint4 and int5. +- **`text_encoder/`** — for FLUX.1 this is T5 + CLIP (T5 is SDNQ'd, CLIP stays full precision); for FLUX.2 Klein and Z-Image, Qwen3 is SDNQ'd. +- **`vae/`** — left as bfloat16 in current Disty0 exports (the VAE is small enough that quantizing it isn't worth the quality risk). + +Layers in the producer's `modules_to_not_convert` list (typically embeddings, final projection, layer norms) stay full precision in all cases. + +## LoRA compatibility + +LoRAs apply to SDNQ-quantized models via the **sidecar patching path**: instead of merging the LoRA delta into the quantized weight, InvokeAI keeps the LoRA as a separate residual that runs alongside each forward pass. + +- ✅ Standard LoRA, LoKr, DoRA, FluxControl-LoRA, FullLayer patches all work. +- ⚠️ Inference is slightly slower per step than non-quantized LoRA application (the sidecar adds an extra matmul per patched layer), but the loss is small in practice. +- ❌ LoRA training against SDNQ-quantized weights is **not supported**. + +If you stack many LoRAs on a heavily quantized model and notice quality drift, try lowering individual LoRA weights — the 4-bit base already eats some headroom for cumulative perturbations. + +## Quality trade-offs + +The dynamic mixed-precision FLUX.2 Klein exports (`Disty0/FLUX.2-klein-{4B,9B}-SDNQ-4bit-dynamic-...`) let SDNQ promote individual layers from uint4 to int5 if the layer's quantization error exceeds a per-group budget. In practice this keeps the most sensitive attention projections at int5 while everything else stays uint4, with no user-visible quality regression vs. bfloat16 in most prompts. + +The static uint4 + SVD exports (`...-SDNQ-uint4-svd-r32`) are slightly more aggressive but use rank-32 SVD residuals to recover the lost precision. The SVD correction adds ~3 % of the original weight size back to the file but largely closes the quality gap. + +You will most likely see SDNQ-specific quality issues at: + +- **Very high CFG values** (> 8) on Klein 4B dynamic — the 4-bit attention saturates faster than bfloat16. +- **Long generations with heavy LoRA stacks** — cumulative quantization noise becomes visible after dozens of steps. + +If you need higher quality and have the VRAM, the static `Disty0/FLUX.2-dev-SDNQ-uint4-svd-r32` (FLUX.2 dev, 12 B params) is the most faithful SDNQ option. + +## Comparison with other quantization formats + +| Format | Size | VRAM at inference | LoRA support | Loading path | +| ----------------- | ----- | ----------------- | ------------ | ---------------------------------------- | +| **SDNQ uint4 + SVD** | ~50 % | ~50 % | ✅ sidecar | Full diffusers pipeline | +| **GGUF Q4_K_M** | ~30 % | ~30 % | ✅ sidecar | Single-file transformer + separate encoders/VAE | +| **BnB NF4** | ~50 % | ~50 % | ✅ sidecar | Single-file transformer + separate encoders/VAE | +| **FP8 storage** | ~50 % | ~50 % | ✅ direct | Any full-precision model (toggle in Model Manager) | + +The headline differences: + +- **SDNQ models are pipeline-shaped**: one install pulls everything you need (transformer + encoders + VAE). GGUF and BnB usually need you to also install a T5 / Qwen3 / VAE separately. +- **SDNQ has the cleanest dynamic-precision story**: GGUF picks one bit-width per file; SDNQ dynamic-mixed exports tune precision per layer. +- **GGUF is more memory-efficient** at the same nominal bit-width because it uses smaller groups. SDNQ trades that for the SVD correction option. + +For low VRAM (~6–8 GB), GGUF Q4 is still the best fit. For 12–16 GB cards that can host a FLUX-class model, SDNQ is the simplest "install one thing, get a working pipeline" option. + +## Troubleshooting + +### "Non-diffusers FLUX.2 Klein models require a standalone Qwen3 Encoder" (Invoke button greyed out) + +The Klein SDNQ pipeline carries its own Qwen3 encoder, so this readiness gate shouldn't fire — if it does, the install most likely happened before SDNQ-pipeline support was wired up and the model is cached with the wrong format in the database. Delete the model from the Model Manager and re-install it; the second install will pick up the correct `sdnq_quantized` format with the submodels populated. + +### Heavy high-frequency noise overlay on output + +If the structure of your image is recognizable (rough subject + composition) but a colored static is layered over it, you've hit a quantization-loader bug. Open an issue with: + +- The exact HuggingFace repo of the model. +- A side-by-side with a non-SDNQ variant of the same prompt (e.g. compare against GGUF Q4 of the same base model). + +Historically this has been caused by missing key-permutation steps in the diffusers→BFL state-dict conversion (e.g. `scale`/`shift` halves swapped). It's not a sign that the file itself is broken. + +### "no safetensors files found" or "size mismatch for weight" + +You probably pointed InvokeAI at the wrong subfolder — SDNQ pipelines are installed as the **whole repo root** (the folder that contains `model_index.json`), not at `transformer/` directly. The Model Manager's "Add Folder" flow expects the pipeline root. diff --git a/invokeai/app/invocations/flux2_denoise.py b/invokeai/app/invocations/flux2_denoise.py index 3b9d3d4ce89..4f11ea730e8 100644 --- a/invokeai/app/invocations/flux2_denoise.py +++ b/invokeai/app/invocations/flux2_denoise.py @@ -462,6 +462,7 @@ def _run_diffusion(self, context: InvocationContext) -> torch.Tensor: ModelFormat.BnbQuantizedLlmInt8b, ModelFormat.BnbQuantizednf4b, ModelFormat.GGUFQuantized, + ModelFormat.SDNQQuantized, ]: model_is_quantized = True else: diff --git a/invokeai/app/invocations/flux2_klein_model_loader.py b/invokeai/app/invocations/flux2_klein_model_loader.py index 2091fd380d7..d450794e3fb 100644 --- a/invokeai/app/invocations/flux2_klein_model_loader.py +++ b/invokeai/app/invocations/flux2_klein_model_loader.py @@ -94,14 +94,13 @@ class Flux2KleinModelLoaderInvocation(BaseInvocation): qwen3_source_model: Optional[ModelIdentifierField] = InputField( default=None, - description="Diffusers Flux2 Klein model to extract VAE and/or Qwen3 encoder from. " - "Use this if you don't have separate VAE/Qwen3 models. " + description="Diffusers or SDNQ-pipeline Flux2 Klein model to extract VAE and/or Qwen3 " + "encoder from. Use this if you don't have separate VAE/Qwen3 models. " "Ignored if both VAE and Qwen3 Encoder are provided separately.", input=Input.Direct, ui_model_base=BaseModelType.Flux2, ui_model_type=ModelType.Main, - ui_model_format=ModelFormat.Diffusers, - title="Qwen3 Source (Diffusers)", + title="Qwen3 Source", ) max_seq_len: Literal[256, 512] = InputField( @@ -114,9 +113,15 @@ def invoke(self, context: InvocationContext) -> Flux2KleinModelLoaderOutput: # Transformer always comes from the main model transformer = self.model.model_copy(update={"submodel_type": SubModelType.Transformer}) - # Check if main model is Diffusers format (can extract VAE directly) + # Check if main model is a pipeline-shaped config we can extract submodels from. + # Plain diffusers pipelines satisfy this; so do SDNQ-quantized pipeline folders (which + # ship the same submodels layout — transformer / vae / text_encoder / tokenizer). + # Single-file SDNQ/GGUF checkpoints don't have submodels populated and fall through to + # the standalone-encoder branch. main_config = context.models.get_config(self.model) - main_is_diffusers = main_config.format == ModelFormat.Diffusers + main_is_diffusers = main_config.format == ModelFormat.Diffusers or ( + main_config.format == ModelFormat.SDNQQuantized and bool(getattr(main_config, "submodels", None)) + ) # Determine VAE source # IMPORTANT: FLUX.2 Klein uses a 32-channel VAE (AutoencoderKLFlux2), not the 16-channel FLUX.1 VAE. @@ -173,13 +178,19 @@ def invoke(self, context: InvocationContext) -> Flux2KleinModelLoaderOutput: def _validate_diffusers_format( self, context: InvocationContext, model: ModelIdentifierField, model_name: str ) -> None: - """Validate that a model is in Diffusers format.""" + """Validate that a model exposes the diffusers-style submodel layout. Both plain diffusers + pipelines and SDNQ-quantized pipeline folders (which ship the same submodels) qualify; + single-file SDNQ FLUX.2 checkpoints don't have submodels populated and are still rejected. + """ config = context.models.get_config(model) - if config.format != ModelFormat.Diffusers: - raise ValueError( - f"The {model_name} model must be a Diffusers format model. " - f"The selected model '{config.name}' is in {config.format.value} format." - ) + if config.format == ModelFormat.Diffusers: + return + if config.format == ModelFormat.SDNQQuantized and getattr(config, "submodels", None): + return + raise ValueError( + f"The {model_name} model must be a Diffusers-style FLUX.2 pipeline (with VAE / Qwen3 " + f"submodels). The selected model '{config.name}' is in {config.format.value} format." + ) def _validate_qwen3_encoder_variant(self, context: InvocationContext, main_config) -> None: """Validate that the standalone Qwen3 encoder variant matches the FLUX.2 Klein variant. diff --git a/invokeai/app/invocations/flux_denoise.py b/invokeai/app/invocations/flux_denoise.py index 06147229232..cb6cd28e631 100644 --- a/invokeai/app/invocations/flux_denoise.py +++ b/invokeai/app/invocations/flux_denoise.py @@ -440,6 +440,7 @@ def _run_diffusion( ModelFormat.BnbQuantizedLlmInt8b, ModelFormat.BnbQuantizednf4b, ModelFormat.GGUFQuantized, + ModelFormat.SDNQQuantized, ]: model_is_quantized = True else: diff --git a/invokeai/app/invocations/flux_model_loader.py b/invokeai/app/invocations/flux_model_loader.py index c175ae7fedc..abe29b2fa87 100644 --- a/invokeai/app/invocations/flux_model_loader.py +++ b/invokeai/app/invocations/flux_model_loader.py @@ -15,6 +15,7 @@ ) from invokeai.backend.flux.util import get_flux_max_seq_length from invokeai.backend.model_manager.configs.base import Checkpoint_Config_Base +from invokeai.backend.model_manager.configs.main import Main_SDNQ_Diffusers_FLUX_Config from invokeai.backend.model_manager.taxonomy import BaseModelType, ModelType, SubModelType @@ -82,7 +83,7 @@ def invoke(self, context: InvocationContext) -> FluxModelLoaderOutput: t5_encoder = preprocess_t5_encoder_model_identifier(self.t5_encoder_model) transformer_config = context.models.get_config(transformer) - assert isinstance(transformer_config, Checkpoint_Config_Base) + assert isinstance(transformer_config, (Checkpoint_Config_Base, Main_SDNQ_Diffusers_FLUX_Config)) return FluxModelLoaderOutput( transformer=TransformerField(transformer=transformer, loras=[]), diff --git a/invokeai/app/invocations/flux_text_encoder.py b/invokeai/app/invocations/flux_text_encoder.py index 8b3b33fad1c..ba67731d64e 100644 --- a/invokeai/app/invocations/flux_text_encoder.py +++ b/invokeai/app/invocations/flux_text_encoder.py @@ -97,6 +97,7 @@ def _t5_encode(self, context: InvocationContext) -> torch.Tensor: ModelFormat.BnbQuantizedLlmInt8b, ModelFormat.BnbQuantizednf4b, ModelFormat.GGUFQuantized, + ModelFormat.SDNQQuantized, ]: model_is_quantized = True else: @@ -154,6 +155,18 @@ def _clip_encode(self, context: InvocationContext) -> torch.Tensor: cached_weights=cached_weights, ) ) + elif clip_text_encoder_config.format in [ModelFormat.SDNQQuantized]: + # SDNQ-quantized CLIP - apply LoRA as sidecar layers + exit_stack.enter_context( + LayerPatcher.apply_smart_model_patches( + model=clip_text_encoder, + patches=self._clip_lora_iterator(context), + prefix=FLUX_LORA_CLIP_PREFIX, + dtype=clip_text_encoder.dtype, + cached_weights=cached_weights, + force_sidecar_patching=True, + ) + ) else: # There are currently no supported CLIP quantized models. Add support here if needed. raise ValueError(f"Unsupported model format: {clip_text_encoder_config.format}") diff --git a/invokeai/app/invocations/flux_vae_decode.py b/invokeai/app/invocations/flux_vae_decode.py index c55dfb539ac..99f4fe64ffc 100644 --- a/invokeai/app/invocations/flux_vae_decode.py +++ b/invokeai/app/invocations/flux_vae_decode.py @@ -1,4 +1,5 @@ import torch +from diffusers.models.autoencoders.autoencoder_kl import AutoencoderKL from einops import rearrange from PIL import Image @@ -40,15 +41,29 @@ class FluxVaeDecodeInvocation(BaseInvocation, WithMetadata, WithBoard): ) def _vae_decode(self, vae_info: LoadedModel, latents: torch.Tensor) -> Image.Image: - assert isinstance(vae_info.model, AutoEncoder) - estimated_working_memory = estimate_vae_working_memory_flux( - operation="decode", image_tensor=latents, vae=vae_info.model - ) + assert isinstance(vae_info.model, (AutoEncoder, AutoencoderKL)) + + # Only estimate working memory for BFL AutoEncoder (diffusers VAE handles this internally) + if isinstance(vae_info.model, AutoEncoder): + estimated_working_memory = estimate_vae_working_memory_flux( + operation="decode", image_tensor=latents, vae=vae_info.model + ) + else: + estimated_working_memory = 0 + with vae_info.model_on_device(working_mem_bytes=estimated_working_memory) as (_, vae): - assert isinstance(vae, AutoEncoder) + assert isinstance(vae, (AutoEncoder, AutoencoderKL)) vae_dtype = next(iter(vae.parameters())).dtype latents = latents.to(device=TorchDevice.choose_torch_device(), dtype=vae_dtype) - img = vae.decode(latents) + + if isinstance(vae, AutoEncoder): + # BFL AutoEncoder returns tensor directly + img = vae.decode(latents) + else: + # Diffusers AutoencoderKL returns DecoderOutput with .sample attribute + # Scale latents for diffusers VAE (FLUX uses shift_factor and scale_factor) + latents = (latents / vae.config.scaling_factor) + vae.config.shift_factor + img = vae.decode(latents, return_dict=False)[0] img = img.clamp(-1, 1) img = rearrange(img[0], "c h w -> h w c") # noqa: F821 diff --git a/invokeai/app/invocations/z_image_denoise.py b/invokeai/app/invocations/z_image_denoise.py index c1e864ea179..358d060534b 100644 --- a/invokeai/app/invocations/z_image_denoise.py +++ b/invokeai/app/invocations/z_image_denoise.py @@ -438,7 +438,7 @@ def _run_diffusion(self, context: InvocationContext) -> torch.Tensor: # slower inference than direct patching, but is agnostic to the quantization format. if transformer_config.format in [ModelFormat.Diffusers, ModelFormat.Checkpoint]: model_is_quantized = False - elif transformer_config.format in [ModelFormat.GGUFQuantized]: + elif transformer_config.format in [ModelFormat.GGUFQuantized, ModelFormat.SDNQQuantized]: model_is_quantized = True else: raise ValueError(f"Unsupported Z-Image model format: {transformer_config.format}") diff --git a/invokeai/app/invocations/z_image_model_loader.py b/invokeai/app/invocations/z_image_model_loader.py index 4d746061dcc..8862c8086ae 100644 --- a/invokeai/app/invocations/z_image_model_loader.py +++ b/invokeai/app/invocations/z_image_model_loader.py @@ -126,10 +126,16 @@ def invoke(self, context: InvocationContext) -> ZImageModelLoaderOutput: def _validate_diffusers_format( self, context: InvocationContext, model: ModelIdentifierField, model_name: str ) -> None: - """Validate that a model is in Diffusers format.""" + """Validate that a model exposes the diffusers-style submodel layout (transformer / vae / + text_encoder / tokenizer subfolders). Plain diffusers Z-Image pipelines satisfy this; + SDNQ-quantized ZImagePipeline folders do too because they ship the same submodels. Single- + file SDNQ Z-Image checkpoints don't have submodels populated and must still be rejected.""" config = context.models.get_config(model) - if config.format != ModelFormat.Diffusers: - raise ValueError( - f"The {model_name} model must be a Diffusers format Z-Image model. " - f"The selected model '{config.name}' is in {config.format.value} format." - ) + if config.format == ModelFormat.Diffusers: + return + if config.format == ModelFormat.SDNQQuantized and getattr(config, "submodels", None): + return + raise ValueError( + f"The {model_name} model must be a Diffusers-style Z-Image pipeline (with VAE / Qwen3 " + f"submodels). The selected model '{config.name}' is in {config.format.value} format." + ) diff --git a/invokeai/app/util/t5_model_identifier.py b/invokeai/app/util/t5_model_identifier.py index a0d999920c8..4d0c2a9c994 100644 --- a/invokeai/app/util/t5_model_identifier.py +++ b/invokeai/app/util/t5_model_identifier.py @@ -8,6 +8,8 @@ def preprocess_t5_encoder_model_identifier(model_identifier: ModelIdentifierFiel """ if model_identifier.base == BaseModelType.Any: return model_identifier.model_copy(update={"submodel_type": SubModelType.TextEncoder2}) + elif model_identifier.base == BaseModelType.Flux: + return model_identifier.model_copy(update={"submodel_type": SubModelType.TextEncoder2}) elif model_identifier.base == BaseModelType.StableDiffusion3: return model_identifier.model_copy(update={"submodel_type": SubModelType.TextEncoder3}) else: @@ -20,6 +22,8 @@ def preprocess_t5_tokenizer_model_identifier(model_identifier: ModelIdentifierFi """ if model_identifier.base == BaseModelType.Any: return model_identifier.model_copy(update={"submodel_type": SubModelType.Tokenizer2}) + elif model_identifier.base == BaseModelType.Flux: + return model_identifier.model_copy(update={"submodel_type": SubModelType.Tokenizer2}) elif model_identifier.base == BaseModelType.StableDiffusion3: return model_identifier.model_copy(update={"submodel_type": SubModelType.Tokenizer3}) else: diff --git a/invokeai/backend/model_manager/configs/factory.py b/invokeai/backend/model_manager/configs/factory.py index 985cb982d30..5ee992270a7 100644 --- a/invokeai/backend/model_manager/configs/factory.py +++ b/invokeai/backend/model_manager/configs/factory.py @@ -83,12 +83,20 @@ Main_GGUF_FLUX_Config, Main_GGUF_QwenImage_Config, Main_GGUF_ZImage_Config, + Main_SDNQ_Diffusers_Flux2_Config, + Main_SDNQ_Diffusers_FLUX_Config, + Main_SDNQ_Diffusers_ZImage_Config, + Main_SDNQ_Flux2_Config, + Main_SDNQ_FLUX_Config, + Main_SDNQ_ZImage_Config, MainModelDefaultSettings, ) from invokeai.backend.model_manager.configs.qwen3_encoder import ( Qwen3Encoder_Checkpoint_Config, Qwen3Encoder_GGUF_Config, Qwen3Encoder_Qwen3Encoder_Config, + Qwen3Encoder_SDNQ_Config, + Qwen3Encoder_SDNQ_Folder_Config, ) from invokeai.backend.model_manager.configs.qwen_vl_encoder import ( QwenVLEncoder_Checkpoint_Config, @@ -100,7 +108,11 @@ T2IAdapter_Diffusers_SD1_Config, T2IAdapter_Diffusers_SDXL_Config, ) -from invokeai.backend.model_manager.configs.t5_encoder import T5Encoder_BnBLLMint8_Config, T5Encoder_T5Encoder_Config +from invokeai.backend.model_manager.configs.t5_encoder import ( + T5Encoder_BnBLLMint8_Config, + T5Encoder_SDNQ_Config, + T5Encoder_T5Encoder_Config, +) from invokeai.backend.model_manager.configs.text_llm import TextLLM_Diffusers_Config from invokeai.backend.model_manager.configs.textual_inversion import ( TI_File_SD1_Config, @@ -193,6 +205,12 @@ Annotated[Main_GGUF_FLUX_Config, Main_GGUF_FLUX_Config.get_tag()], Annotated[Main_GGUF_QwenImage_Config, Main_GGUF_QwenImage_Config.get_tag()], Annotated[Main_GGUF_ZImage_Config, Main_GGUF_ZImage_Config.get_tag()], + Annotated[Main_SDNQ_FLUX_Config, Main_SDNQ_FLUX_Config.get_tag()], + Annotated[Main_SDNQ_Diffusers_FLUX_Config, Main_SDNQ_Diffusers_FLUX_Config.get_tag()], + Annotated[Main_SDNQ_Flux2_Config, Main_SDNQ_Flux2_Config.get_tag()], + Annotated[Main_SDNQ_Diffusers_Flux2_Config, Main_SDNQ_Diffusers_Flux2_Config.get_tag()], + Annotated[Main_SDNQ_ZImage_Config, Main_SDNQ_ZImage_Config.get_tag()], + Annotated[Main_SDNQ_Diffusers_ZImage_Config, Main_SDNQ_Diffusers_ZImage_Config.get_tag()], # VAE - checkpoint format Annotated[VAE_Checkpoint_SD1_Config, VAE_Checkpoint_SD1_Config.get_tag()], Annotated[VAE_Checkpoint_SD2_Config, VAE_Checkpoint_SD2_Config.get_tag()], @@ -244,10 +262,13 @@ # T5 Encoder - all formats Annotated[T5Encoder_T5Encoder_Config, T5Encoder_T5Encoder_Config.get_tag()], Annotated[T5Encoder_BnBLLMint8_Config, T5Encoder_BnBLLMint8_Config.get_tag()], + Annotated[T5Encoder_SDNQ_Config, T5Encoder_SDNQ_Config.get_tag()], # Qwen3 Encoder Annotated[Qwen3Encoder_Qwen3Encoder_Config, Qwen3Encoder_Qwen3Encoder_Config.get_tag()], Annotated[Qwen3Encoder_Checkpoint_Config, Qwen3Encoder_Checkpoint_Config.get_tag()], Annotated[Qwen3Encoder_GGUF_Config, Qwen3Encoder_GGUF_Config.get_tag()], + Annotated[Qwen3Encoder_SDNQ_Config, Qwen3Encoder_SDNQ_Config.get_tag()], + Annotated[Qwen3Encoder_SDNQ_Folder_Config, Qwen3Encoder_SDNQ_Folder_Config.get_tag()], # Qwen VL Encoder (Qwen2.5-VL multimodal encoder for Qwen Image) Annotated[QwenVLEncoder_Diffusers_Config, QwenVLEncoder_Diffusers_Config.get_tag()], Annotated[QwenVLEncoder_Checkpoint_Config, QwenVLEncoder_Checkpoint_Config.get_tag()], diff --git a/invokeai/backend/model_manager/configs/main.py b/invokeai/backend/model_manager/configs/main.py index e1e408a3483..d2600ebd041 100644 --- a/invokeai/backend/model_manager/configs/main.py +++ b/invokeai/backend/model_manager/configs/main.py @@ -1,4 +1,5 @@ from abc import ABC +from pathlib import Path from typing import Any, Literal, Self from pydantic import BaseModel, ConfigDict, Field @@ -26,6 +27,7 @@ Flux2VariantType, FluxVariantType, ModelFormat, + ModelRepoVariant, ModelType, ModelVariantType, QwenImageVariantType, @@ -34,6 +36,7 @@ ZImageVariantType, ) from invokeai.backend.quantization.gguf.ggml_tensor import GGMLTensor +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor from invokeai.backend.stable_diffusion.schedulers.schedulers import SCHEDULER_NAME_VALUES DEFAULTS_PRECISION = Literal["fp16", "fp32"] @@ -122,6 +125,25 @@ def _has_ggml_tensors(state_dict: dict[str | int, Any]) -> bool: return any(isinstance(v, GGMLTensor) for v in state_dict.values()) +def _has_sdnq_tensors(state_dict: dict[str | int, Any]) -> bool: + """Check if state dict contains SDNQTensor instances.""" + return any(isinstance(v, SDNQTensor) for v in state_dict.values()) + + +def _has_sdnq_keys(state_dict: dict[str | int, Any]) -> bool: + """Check if state dict has SDNQ-style keys (weight + scale pairs). + + SDNQ quantized models store weights with associated scale tensors. + """ + keys = {k for k in state_dict.keys() if isinstance(k, str)} + for key in keys: + if key.endswith(".weight"): + base = key[:-7] + if f"{base}.scale" in keys: + return True + return False + + def _has_main_keys(state_dict: dict[str | int, Any]) -> bool: for key in state_dict.keys(): if isinstance(key, int): @@ -796,6 +818,10 @@ def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) - }, ) + # Reject SDNQ-quantized pipelines so Main_SDNQ_Diffusers_FLUX_Config matches instead. + if (mod.path / "transformer").is_dir() and _is_sdnq_folder(mod.path / "transformer"): + raise NotAMatchError("transformer is SDNQ-quantized; use Main_SDNQ_Diffusers_FLUX_Config") + variant = override_fields.pop("variant", None) or cls._get_variant_or_raise(mod) repo_variant = override_fields.pop("repo_variant", None) or cls._get_repo_variant_or_raise(mod) @@ -850,6 +876,13 @@ def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) - }, ) + # Reject SDNQ-quantized pipelines so the SDNQ-specific config matches them instead. + # Without this both configs accept the same folder and identification can latch onto + # the wrong one (the plain diffusers loader would then mis-read packed uint8 weights + # as bf16 and crash with size-mismatch errors at first inference). + if (mod.path / "transformer").is_dir() and _is_sdnq_folder(mod.path / "transformer"): + raise NotAMatchError("transformer is SDNQ-quantized; use Main_SDNQ_Diffusers_Flux2_Config") + variant = override_fields.pop("variant", None) or cls._get_variant_or_raise(mod) repo_variant = override_fields.pop("repo_variant", None) or cls._get_repo_variant_or_raise(mod) @@ -1408,3 +1441,561 @@ def _validate_looks_like_anima_model(cls, mod: ModelOnDisk) -> None: has_anima_keys = _has_anima_keys(mod.load_state_dict()) if not has_anima_keys: raise NotAMatchError("state dict does not look like an Anima model") + + +class Main_SDNQ_FLUX_Config(Checkpoint_Config_Base, Main_Config_Base, Config_Base): + """Model config for SDNQ-quantized FLUX transformer models.""" + + base: Literal[BaseModelType.Flux] = Field(default=BaseModelType.Flux) + format: Literal[ModelFormat.SDNQQuantized] = Field(default=ModelFormat.SDNQQuantized) + + variant: FluxVariantType = Field() + + @classmethod + def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self: + raise_if_not_file(mod) + + raise_for_override_fields(cls, override_fields) + + cls._validate_looks_like_main_model(mod) + + cls._validate_looks_like_sdnq_quantized(mod) + + variant = override_fields.get("variant") or cls._get_variant_or_raise(mod) + + return cls(**override_fields, variant=variant) + + @classmethod + def _get_variant_or_raise(cls, mod: ModelOnDisk) -> FluxVariantType: + state_dict = mod.load_state_dict() + variant = _get_flux_variant(state_dict) + + if variant is None: + raise NotAMatchError("unable to determine model variant from state dict") + + return variant + + @classmethod + def _validate_looks_like_main_model(cls, mod: ModelOnDisk) -> None: + has_main_model_keys = _has_main_keys(mod.load_state_dict()) + if not has_main_model_keys: + raise NotAMatchError("state dict does not look like a main model") + + @classmethod + def _validate_looks_like_sdnq_quantized(cls, mod: ModelOnDisk) -> None: + state_dict = mod.load_state_dict() + if not _has_sdnq_keys(state_dict) and not _has_sdnq_tensors(state_dict): + raise NotAMatchError("state dict does not look like SDNQ quantized") + + +class Main_SDNQ_Flux2_Config(Checkpoint_Config_Base, Main_Config_Base, Config_Base): + """Model config for SDNQ-quantized FLUX.2 transformer models (e.g. Klein 4B / 9B).""" + + base: Literal[BaseModelType.Flux2] = Field(default=BaseModelType.Flux2) + format: Literal[ModelFormat.SDNQQuantized] = Field(default=ModelFormat.SDNQQuantized) + + variant: Flux2VariantType = Field() + + @classmethod + def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self: + raise_if_not_file(mod) + + raise_for_override_fields(cls, override_fields) + + cls._validate_looks_like_main_model(mod) + + cls._validate_is_flux2(mod) + + cls._validate_looks_like_sdnq_quantized(mod) + + variant = override_fields.pop("variant", None) or cls._get_variant_or_raise(mod) + + return cls(**override_fields, variant=variant) + + @classmethod + def _validate_looks_like_main_model(cls, mod: ModelOnDisk) -> None: + has_main_model_keys = _has_main_keys(mod.load_state_dict()) + if not has_main_model_keys: + raise NotAMatchError("state dict does not look like a main model") + + @classmethod + def _validate_is_flux2(cls, mod: ModelOnDisk) -> None: + state_dict = mod.load_state_dict() + if not _is_flux2_model(state_dict): + raise NotAMatchError("state dict does not look like a FLUX.2 model") + + @classmethod + def _validate_looks_like_sdnq_quantized(cls, mod: ModelOnDisk) -> None: + state_dict = mod.load_state_dict() + if not _has_sdnq_keys(state_dict) and not _has_sdnq_tensors(state_dict): + raise NotAMatchError("state dict does not look like SDNQ quantized") + + @classmethod + def _get_variant_or_raise(cls, mod: ModelOnDisk) -> Flux2VariantType: + state_dict = mod.load_state_dict() + variant = _get_flux2_variant(state_dict) + + if variant is None: + raise NotAMatchError("unable to determine FLUX.2 model variant from state dict") + + if variant == Flux2VariantType.Klein9B and _filename_suggests_base(mod.name): + return Flux2VariantType.Klein9BBase + if variant == Flux2VariantType.Klein4B and _filename_suggests_base(mod.name): + return Flux2VariantType.Klein4BBase + + return variant + + +class Main_SDNQ_ZImage_Config(Checkpoint_Config_Base, Main_Config_Base, Config_Base): + """Model config for SDNQ-quantized Z-Image transformer models.""" + + base: Literal[BaseModelType.ZImage] = Field(default=BaseModelType.ZImage) + format: Literal[ModelFormat.SDNQQuantized] = Field(default=ModelFormat.SDNQQuantized) + variant: ZImageVariantType = Field() + + @classmethod + def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self: + raise_if_not_file(mod) + + raise_for_override_fields(cls, override_fields) + + cls._validate_looks_like_z_image_model(mod) + + cls._validate_looks_like_sdnq_quantized(mod) + + variant = override_fields.pop("variant", None) or ZImageVariantType.Turbo + + return cls(**override_fields, variant=variant) + + @classmethod + def _validate_looks_like_z_image_model(cls, mod: ModelOnDisk) -> None: + has_z_image_keys = _has_z_image_keys(mod.load_state_dict()) + if not has_z_image_keys: + raise NotAMatchError("state dict does not look like a Z-Image model") + + @classmethod + def _validate_looks_like_sdnq_quantized(cls, mod: ModelOnDisk) -> None: + state_dict = mod.load_state_dict() + if not _has_sdnq_keys(state_dict) and not _has_sdnq_tensors(state_dict): + raise NotAMatchError("state dict does not look like SDNQ quantized") + + +class Main_SDNQ_Diffusers_Flux2_Config(Main_Config_Base, Config_Base): + """Model config for SDNQ-quantized FLUX.2 models in diffusers format + (Flux2KleinPipeline / Flux2Pipeline folder with transformer/, text_encoder/, vae/, ...).""" + + base: Literal[BaseModelType.Flux2] = Field(default=BaseModelType.Flux2) + format: Literal[ModelFormat.SDNQQuantized] = Field(default=ModelFormat.SDNQQuantized) + + variant: Flux2VariantType = Field() + repo_variant: ModelRepoVariant = Field(default=ModelRepoVariant.Default) + submodels: dict[SubModelType, SubmodelDefinition] | None = Field( + description="Loadable submodels in this model", + default=None, + ) + + @classmethod + def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self: + raise_if_not_dir(mod) + + raise_for_override_fields(cls, override_fields) + + cls._validate_looks_like_flux2_diffusers(mod) + + cls._validate_has_sdnq_transformer(mod) + + variant = override_fields.get("variant") or cls._get_variant_or_raise(mod) + repo_variant = override_fields.get("repo_variant") or cls._get_repo_variant(mod) + submodels = override_fields.get("submodels") or cls._get_submodels(mod) + + return cls(**override_fields, variant=variant, repo_variant=repo_variant, submodels=submodels) + + @classmethod + def _validate_looks_like_flux2_diffusers(cls, mod: ModelOnDisk) -> None: + raise_for_class_name( + common_config_paths(mod.path), + { + "Flux2Pipeline", + "Flux2KleinPipeline", + "Flux2Transformer2DModel", + }, + ) + + @classmethod + def _validate_has_sdnq_transformer(cls, mod: ModelOnDisk) -> None: + transformer_path = mod.path / "transformer" + if not transformer_path.is_dir(): + raise NotAMatchError("no transformer subfolder found") + + if not _is_sdnq_folder(transformer_path): + raise NotAMatchError("transformer is not SDNQ quantized") + + @classmethod + def _get_variant_or_raise(cls, mod: ModelOnDisk) -> Flux2VariantType: + """Determine the Flux2 variant from the transformer config + filename heuristic.""" + transformer_config = get_config_dict_or_raise(mod.path / "transformer" / "config.json") + + hidden_size = transformer_config.get("attention_head_dim", 128) * transformer_config.get( + "num_attention_heads", 24 + ) + joint_attention_dim = transformer_config.get("joint_attention_dim", 7680) + + # Klein 4B uses Qwen3-4B encoder → joint_attention_dim = 3 × 2560 = 7680 + # Klein 9B uses Qwen3-8B encoder → joint_attention_dim = 3 × 4096 = 12288 + # hidden_size 3072 → 4B variant, 4096 → 9B variant + if hidden_size == 4096 or joint_attention_dim == 12288: + variant = Flux2VariantType.Klein9B + else: + variant = Flux2VariantType.Klein4B + + if _filename_suggests_base(mod.name): + if variant == Flux2VariantType.Klein9B: + return Flux2VariantType.Klein9BBase + if variant == Flux2VariantType.Klein4B: + return Flux2VariantType.Klein4BBase + return variant + + @classmethod + def _get_repo_variant(cls, mod: ModelOnDisk) -> ModelRepoVariant: + weight_files = list(mod.path.glob("**/*.safetensors")) + weight_files.extend(list(mod.path.glob("**/*.bin"))) + for x in weight_files: + if ".fp16" in x.suffixes: + return ModelRepoVariant.FP16 + if "openvino_model" in x.name: + return ModelRepoVariant.OpenVINO + if "flax_model" in x.name: + return ModelRepoVariant.Flax + if x.suffix == ".onnx": + return ModelRepoVariant.ONNX + return ModelRepoVariant.Default + + @classmethod + def _get_submodels(cls, mod: ModelOnDisk) -> dict[SubModelType, SubmodelDefinition]: + config = get_config_dict_or_raise(common_config_paths(mod.path)) + + submodels: dict[SubModelType, SubmodelDefinition] = {} + + for key, value in config.items(): + if key.startswith("_") or not (isinstance(value, list) and len(value) == 2): + continue + + _library_name, class_name = value + + if class_name is None: + continue + + match class_name: + case "Flux2Transformer2DModel": + submodels[SubModelType.Transformer] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.Main, + variant=None, + ) + case "Qwen3ForCausalLM": + submodels[SubModelType.TextEncoder] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.Qwen3Encoder, + variant=None, + ) + case "Qwen2Tokenizer": + submodels[SubModelType.Tokenizer] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.Qwen3Encoder, + variant=None, + ) + case "AutoencoderKLFlux2" | "AutoencoderKL": + submodels[SubModelType.VAE] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.VAE, + variant=None, + ) + case _: + pass + + return submodels + + +class Main_SDNQ_Diffusers_ZImage_Config(Main_Config_Base, Config_Base): + """Model config for SDNQ-quantized Z-Image models in diffusers format (full ZImagePipeline folder).""" + + base: Literal[BaseModelType.ZImage] = Field(default=BaseModelType.ZImage) + format: Literal[ModelFormat.SDNQQuantized] = Field(default=ModelFormat.SDNQQuantized) + variant: ZImageVariantType = Field() + + repo_variant: ModelRepoVariant = Field(default=ModelRepoVariant.Default) + submodels: dict[SubModelType, SubmodelDefinition] | None = Field( + description="Loadable submodels in this model", + default=None, + ) + + @classmethod + def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self: + raise_if_not_dir(mod) + + raise_for_override_fields(cls, override_fields) + + cls._validate_looks_like_z_image_diffusers(mod) + + cls._validate_has_sdnq_transformer(mod) + + variant = override_fields.get("variant") or cls._get_variant_or_default(mod) + + repo_variant = override_fields.get("repo_variant") or cls._get_repo_variant(mod) + + submodels = override_fields.get("submodels") or cls._get_submodels(mod) + + return cls(**override_fields, variant=variant, repo_variant=repo_variant, submodels=submodels) + + @classmethod + def _get_variant_or_default(cls, mod: ModelOnDisk) -> ZImageVariantType: + """Determine Z-Image variant from the scheduler config (same heuristic as the unquantized diffusers config). + + Turbo (distilled) uses shift = 3.0, ZBase (undistilled) uses shift = 6.0. + """ + try: + scheduler_config = get_config_dict_or_raise(mod.path / "scheduler" / "scheduler_config.json") + shift = scheduler_config.get("shift", 3.0) + except NotAMatchError: + return ZImageVariantType.Turbo + return ZImageVariantType.ZBase if shift >= 5.0 else ZImageVariantType.Turbo + + @classmethod + def _validate_looks_like_z_image_diffusers(cls, mod: ModelOnDisk) -> None: + raise_for_class_name( + common_config_paths(mod.path), + { + "ZImagePipeline", + "ZImageTransformer2DModel", + }, + ) + + @classmethod + def _validate_has_sdnq_transformer(cls, mod: ModelOnDisk) -> None: + transformer_path = mod.path / "transformer" + if not transformer_path.is_dir(): + raise NotAMatchError("no transformer subfolder found") + + if not _is_sdnq_folder(transformer_path): + raise NotAMatchError("transformer is not SDNQ quantized") + + @classmethod + def _get_repo_variant(cls, mod: ModelOnDisk) -> ModelRepoVariant: + weight_files = list(mod.path.glob("**/*.safetensors")) + weight_files.extend(list(mod.path.glob("**/*.bin"))) + for x in weight_files: + if ".fp16" in x.suffixes: + return ModelRepoVariant.FP16 + if "openvino_model" in x.name: + return ModelRepoVariant.OpenVINO + if "flax_model" in x.name: + return ModelRepoVariant.Flax + if x.suffix == ".onnx": + return ModelRepoVariant.ONNX + return ModelRepoVariant.Default + + @classmethod + def _get_submodels(cls, mod: ModelOnDisk) -> dict[SubModelType, SubmodelDefinition]: + config = get_config_dict_or_raise(common_config_paths(mod.path)) + + submodels: dict[SubModelType, SubmodelDefinition] = {} + + for key, value in config.items(): + if key.startswith("_") or not (isinstance(value, list) and len(value) == 2): + continue + + _library_name, class_name = value + + if class_name is None: + continue + + match class_name: + case "ZImageTransformer2DModel": + submodels[SubModelType.Transformer] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.Main, + variant=None, + ) + case "Qwen3ForCausalLM": + submodels[SubModelType.TextEncoder] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.Qwen3Encoder, + variant=None, + ) + case "Qwen2Tokenizer": + submodels[SubModelType.Tokenizer] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.Qwen3Encoder, + variant=None, + ) + case "AutoencoderKL": + submodels[SubModelType.VAE] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.VAE, + variant=None, + ) + case _: + pass + + return submodels + + +def _is_sdnq_folder(folder_path: Path) -> bool: + """Check if a folder contains SDNQ-quantized model weights by checking quantization_config.json.""" + import json + + quant_config_path = folder_path / "quantization_config.json" + if quant_config_path.exists(): + try: + with open(quant_config_path, "r", encoding="utf-8") as f: + quant_config = json.load(f) + if quant_config.get("quant_method") == "sdnq": + return True + except (json.JSONDecodeError, OSError): + pass + return False + + +class Main_SDNQ_Diffusers_FLUX_Config(Main_Config_Base, Config_Base): + """Model config for SDNQ-quantized FLUX models in diffusers format (folder with transformer, text_encoder, etc.).""" + + base: Literal[BaseModelType.Flux] = Field(default=BaseModelType.Flux) + format: Literal[ModelFormat.SDNQQuantized] = Field(default=ModelFormat.SDNQQuantized) + + variant: FluxVariantType = Field() + repo_variant: ModelRepoVariant = Field(default=ModelRepoVariant.Default) + submodels: dict[SubModelType, SubmodelDefinition] | None = Field( + description="Loadable submodels in this model", + default=None, + ) + + @classmethod + def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self: + raise_if_not_dir(mod) + + raise_for_override_fields(cls, override_fields) + + cls._validate_looks_like_flux_diffusers(mod) + + cls._validate_has_sdnq_transformer(mod) + + variant = override_fields.get("variant") or cls._get_variant_or_raise(mod) + + repo_variant = override_fields.get("repo_variant") or cls._get_repo_variant(mod) + + submodels = override_fields.get("submodels") or cls._get_submodels(mod) + + return cls(**override_fields, variant=variant, repo_variant=repo_variant, submodels=submodels) + + @classmethod + def _validate_looks_like_flux_diffusers(cls, mod: ModelOnDisk) -> None: + """Check if this looks like a Flux diffusers model by checking for FluxPipeline or FluxTransformer2DModel.""" + raise_for_class_name( + common_config_paths(mod.path), + { + "FluxPipeline", + "FluxTransformer2DModel", + }, + ) + + @classmethod + def _validate_has_sdnq_transformer(cls, mod: ModelOnDisk) -> None: + """Check if the transformer subfolder contains SDNQ quantization.""" + transformer_path = mod.path / "transformer" + if not transformer_path.is_dir(): + raise NotAMatchError("no transformer subfolder found") + + if not _is_sdnq_folder(transformer_path): + raise NotAMatchError("transformer is not SDNQ quantized") + + @classmethod + def _get_variant_or_raise(cls, mod: ModelOnDisk) -> FluxVariantType: + """Determine the Flux variant from the transformer config.""" + transformer_config = get_config_dict_or_raise(mod.path / "transformer" / "config.json") + + # Check for guidance_embeds to determine if it's Dev or Schnell + guidance_embeds = transformer_config.get("guidance_embeds", False) + in_channels = transformer_config.get("in_channels", 64) + + if guidance_embeds and in_channels == 384: + return FluxVariantType.DevFill + elif guidance_embeds: + return FluxVariantType.Dev + else: + return FluxVariantType.Schnell + + @classmethod + def _get_repo_variant(cls, mod: ModelOnDisk) -> ModelRepoVariant: + """Determine the repo variant from the model files.""" + weight_files = list(mod.path.glob("**/*.safetensors")) + weight_files.extend(list(mod.path.glob("**/*.bin"))) + for x in weight_files: + if ".fp16" in x.suffixes: + return ModelRepoVariant.FP16 + if "openvino_model" in x.name: + return ModelRepoVariant.OpenVINO + if "flax_model" in x.name: + return ModelRepoVariant.Flax + if x.suffix == ".onnx": + return ModelRepoVariant.ONNX + return ModelRepoVariant.Default + + @classmethod + def _get_submodels(cls, mod: ModelOnDisk) -> dict[SubModelType, SubmodelDefinition]: + """Extract submodels from model_index.json for Flux SDNQ diffusers format.""" + config = get_config_dict_or_raise(common_config_paths(mod.path)) + + submodels: dict[SubModelType, SubmodelDefinition] = {} + + for key, value in config.items(): + # Skip metadata fields and invalid entries + if key.startswith("_") or not (isinstance(value, list) and len(value) == 2): + continue + + _library_name, class_name = value + + # Skip null entries + if class_name is None: + continue + + match class_name: + case "FluxTransformer2DModel": + submodels[SubModelType.Transformer] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.Main, + variant=None, + ) + case "CLIPTextModel": + submodels[SubModelType.TextEncoder] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.CLIPEmbed, + variant=None, + ) + case "T5EncoderModel": + submodels[SubModelType.TextEncoder2] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.T5Encoder, + variant=None, + ) + case "AutoencoderKL": + submodels[SubModelType.VAE] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.VAE, + variant=None, + ) + case "CLIPTokenizer": + submodels[SubModelType.Tokenizer] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.CLIPEmbed, + variant=None, + ) + case "T5TokenizerFast": + submodels[SubModelType.Tokenizer2] = SubmodelDefinition( + path_or_prefix=(mod.path / key).resolve().as_posix(), + model_type=ModelType.T5Encoder, + variant=None, + ) + case _: + pass + + return submodels diff --git a/invokeai/backend/model_manager/configs/qwen3_encoder.py b/invokeai/backend/model_manager/configs/qwen3_encoder.py index 308539aa354..11121a0d4c1 100644 --- a/invokeai/backend/model_manager/configs/qwen3_encoder.py +++ b/invokeai/backend/model_manager/configs/qwen3_encoder.py @@ -14,6 +14,7 @@ from invokeai.backend.model_manager.model_on_disk import ModelOnDisk from invokeai.backend.model_manager.taxonomy import BaseModelType, ModelFormat, ModelType, Qwen3VariantType from invokeai.backend.quantization.gguf.ggml_tensor import GGMLTensor +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor def _has_qwen3_keys(state_dict: dict[str | int, Any]) -> bool: @@ -46,6 +47,22 @@ def _has_ggml_tensors(state_dict: dict[str | int, Any]) -> bool: return any(isinstance(v, GGMLTensor) for v in state_dict.values()) +def _has_sdnq_tensors(state_dict: dict[str | int, Any]) -> bool: + """Check if state dict contains SDNQTensor instances.""" + return any(isinstance(v, SDNQTensor) for v in state_dict.values()) + + +def _has_sdnq_keys(state_dict: dict[str | int, Any]) -> bool: + """Check if state dict has SDNQ-style keys (weight + scale pairs).""" + keys = {k for k in state_dict.keys() if isinstance(k, str)} + for key in keys: + if key.endswith(".weight"): + base = key[:-7] + if f"{base}.scale" in keys: + return True + return False + + def _get_qwen3_variant_from_state_dict(state_dict: dict[str | int, Any]) -> Optional[Qwen3VariantType]: """Determine Qwen3 variant (0.6B, 4B, or 8B) from state dict based on hidden_size. @@ -281,3 +298,115 @@ def _validate_looks_like_gguf_quantized(cls, mod: ModelOnDisk) -> None: has_ggml = _has_ggml_tensors(mod.load_state_dict()) if not has_ggml: raise NotAMatchError("state dict does not look like GGUF quantized") + + +class Qwen3Encoder_SDNQ_Config(Checkpoint_Config_Base, Config_Base): + """Configuration for SDNQ-quantized Qwen3 Encoder models (single file).""" + + base: Literal[BaseModelType.Any] = Field(default=BaseModelType.Any) + type: Literal[ModelType.Qwen3Encoder] = Field(default=ModelType.Qwen3Encoder) + format: Literal[ModelFormat.SDNQQuantized] = Field(default=ModelFormat.SDNQQuantized) + cpu_only: bool | None = Field(default=None, description="Whether this model should run on CPU only") + variant: Qwen3VariantType = Field(description="Qwen3 model size variant (4B or 8B)") + + @classmethod + def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self: + raise_if_not_file(mod) + + raise_for_override_fields(cls, override_fields) + + cls._validate_looks_like_qwen3_model(mod) + + cls._validate_looks_like_sdnq_quantized(mod) + + variant = cls._get_variant_or_default(mod) + + return cls(variant=variant, **override_fields) + + @classmethod + def _get_variant_or_default(cls, mod: ModelOnDisk) -> Qwen3VariantType: + """Get variant from state dict, defaulting to 4B if unknown.""" + state_dict = mod.load_state_dict() + variant = _get_qwen3_variant_from_state_dict(state_dict) + return variant if variant is not None else Qwen3VariantType.Qwen3_4B + + @classmethod + def _validate_looks_like_qwen3_model(cls, mod: ModelOnDisk) -> None: + has_qwen3 = _has_qwen3_keys(mod.load_state_dict()) + if not has_qwen3: + raise NotAMatchError("state dict does not look like a Qwen3 model") + + @classmethod + def _validate_looks_like_sdnq_quantized(cls, mod: ModelOnDisk) -> None: + state_dict = mod.load_state_dict() + if not _has_sdnq_tensors(state_dict) and not _has_sdnq_keys(state_dict): + raise NotAMatchError("state dict does not look like SDNQ quantized") + + +class Qwen3Encoder_SDNQ_Folder_Config(Config_Base): + """Configuration for folder-based SDNQ-quantized Qwen3 Encoder models. + + Used for SDNQ bundles where the text_encoder is a folder containing + quantization_config.json and safetensors files with SDNQ keys. + """ + + base: Literal[BaseModelType.Any] = Field(default=BaseModelType.Any) + type: Literal[ModelType.Qwen3Encoder] = Field(default=ModelType.Qwen3Encoder) + format: Literal[ModelFormat.SDNQQuantized] = Field(default=ModelFormat.SDNQQuantized) + cpu_only: bool | None = Field(default=None, description="Whether this model should run on CPU only") + variant: Qwen3VariantType = Field(description="Qwen3 model size variant (4B or 8B)") + + @classmethod + def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self: + raise_if_not_dir(mod) + + raise_for_override_fields(cls, override_fields) + + matched = False + + # Check for quantization_config.json with quant_method="sdnq" + quant_config_path = mod.path / "quantization_config.json" + if quant_config_path.exists(): + with open(quant_config_path, "r", encoding="utf-8") as f: + quant_config = json.load(f) + if quant_config.get("quant_method") == "sdnq": + matched = True + + # Fallback: check if safetensors files have SDNQ-style keys + if not matched: + safetensors_files = list(mod.path.glob("*.safetensors")) + if safetensors_files: + state_dict = mod.load_state_dict() + if _has_sdnq_keys(state_dict): + matched = True + + if not matched: + raise NotAMatchError("directory does not look like an SDNQ-quantized Qwen3 encoder") + + variant = cls._get_variant_from_dir(mod) + return cls(variant=variant, **override_fields) + + @classmethod + def _get_variant_from_dir(cls, mod: ModelOnDisk) -> Qwen3VariantType: + """Determine variant from config.json hidden_size, defaulting to 4B.""" + QWEN3_06B_HIDDEN_SIZE = 1024 + QWEN3_4B_HIDDEN_SIZE = 2560 + QWEN3_8B_HIDDEN_SIZE = 4096 + + for cfg_path in (mod.path / "config.json", mod.path / "text_encoder" / "config.json"): + if not cfg_path.exists(): + continue + try: + with open(cfg_path, "r", encoding="utf-8") as f: + cfg = json.load(f) + except (json.JSONDecodeError, OSError): + continue + hidden_size = cfg.get("hidden_size") + if hidden_size == QWEN3_8B_HIDDEN_SIZE: + return Qwen3VariantType.Qwen3_8B + if hidden_size == QWEN3_4B_HIDDEN_SIZE: + return Qwen3VariantType.Qwen3_4B + if hidden_size == QWEN3_06B_HIDDEN_SIZE: + return Qwen3VariantType.Qwen3_06B + break + return Qwen3VariantType.Qwen3_4B diff --git a/invokeai/backend/model_manager/configs/t5_encoder.py b/invokeai/backend/model_manager/configs/t5_encoder.py index 2da417b10a5..60684cea5fb 100644 --- a/invokeai/backend/model_manager/configs/t5_encoder.py +++ b/invokeai/backend/model_manager/configs/t5_encoder.py @@ -1,6 +1,8 @@ +import json from typing import Any, Literal, Self from pydantic import Field +from safetensors import safe_open from invokeai.backend.model_manager.configs.base import Config_Base from invokeai.backend.model_manager.configs.identification_utils import ( @@ -14,6 +16,20 @@ from invokeai.backend.model_manager.taxonomy import BaseModelType, ModelFormat, ModelType +def _safetensors_dir_has_sdnq_keys(directory) -> bool: + """Return True if any safetensors file in ``directory`` looks SDNQ-quantized (weight + matching scale).""" + for st_file in sorted(directory.glob("*.safetensors")): + try: + with safe_open(st_file, framework="pt") as f: + keys = set(f.keys()) + except Exception: + continue + for key in keys: + if key.endswith(".weight") and f"{key[:-7]}.scale" in keys: + return True + return False + + class T5Encoder_T5Encoder_Config(Config_Base): """Configuration for T5 Encoder models in a bespoke, diffusers-like format. The model weights are expected to be in a folder called text_encoder_2 inside the model directory, with a config file named model.safetensors.index.json.""" @@ -80,3 +96,65 @@ def raise_if_state_dict_doesnt_look_like_bnb_quantized(cls, mod: ModelOnDisk) -> has_scb_key_suffix = state_dict_has_any_keys_ending_with(mod.load_state_dict(), "SCB") if not has_scb_key_suffix: raise NotAMatchError("state dict does not look like bnb quantized llm_int8") + + +class T5Encoder_SDNQ_Config(Config_Base): + """Configuration for SDNQ-quantized T5 Encoder models. + + Matches two layouts: + + 1. **Standalone T5 bundle**: ``mod.path`` is the pipeline-style root, with + ``text_encoder_2/`` (and usually ``tokenizer_2/``) as subfolders. + 2. **Inline submodel**: ``mod.path`` *is* the ``text_encoder_2`` folder itself — + this is how a parent FluxPipeline / similar config registers its T5 submodel + (``submodels[TextEncoder2].path_or_prefix`` points straight at the folder). + + In both cases, the SDNQ-quantized state lives next to a ``config.json`` declaring + ``T5EncoderModel`` and is signalled either by ``quantization_config.json`` with + ``quant_method == "sdnq"`` or by SDNQ-style ``weight`` + ``scale`` key pairs. + """ + + base: Literal[BaseModelType.Any] = Field(default=BaseModelType.Any) + type: Literal[ModelType.T5Encoder] = Field(default=ModelType.T5Encoder) + format: Literal[ModelFormat.SDNQQuantized] = Field(default=ModelFormat.SDNQQuantized) + cpu_only: bool | None = Field(default=None, description="Whether this model should run on CPU only") + + @classmethod + def from_model_on_disk(cls, mod: ModelOnDisk, override_fields: dict[str, Any]) -> Self: + raise_if_not_dir(mod) + + raise_for_override_fields(cls, override_fields) + + te_dir = cls._locate_text_encoder_dir(mod) + raise_for_class_name(te_dir / "config.json", "T5EncoderModel") + + cls._raise_if_not_sdnq_quantized(te_dir) + + return cls(**override_fields) + + @classmethod + def _locate_text_encoder_dir(cls, mod: ModelOnDisk): + """Return the directory that actually holds T5's config.json + safetensors.""" + nested = mod.path / "text_encoder_2" + if (nested / "config.json").exists(): + return nested + if (mod.path / "config.json").exists(): + return mod.path + raise NotAMatchError("no text_encoder_2/config.json or config.json at model root") + + @classmethod + def _raise_if_not_sdnq_quantized(cls, te_dir) -> None: + quant_config_path = te_dir / "quantization_config.json" + if quant_config_path.exists(): + try: + with open(quant_config_path, "r", encoding="utf-8") as f: + quant_config = json.load(f) + except (OSError, ValueError): + quant_config = {} + if quant_config.get("quant_method") == "sdnq": + return + + if _safetensors_dir_has_sdnq_keys(te_dir): + return + + raise NotAMatchError("text_encoder_2 does not look like an SDNQ-quantized T5 encoder") diff --git a/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_conv2d.py b/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_conv2d.py index eac3549b5ab..a6ecf84dde3 100644 --- a/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_conv2d.py +++ b/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_conv2d.py @@ -8,6 +8,7 @@ add_nullable_tensors, ) from invokeai.backend.quantization.gguf.ggml_tensor import GGMLTensor +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor class CustomConv2d(torch.nn.Conv2d, CustomModuleMixin): @@ -17,7 +18,7 @@ def _cast_tensor_for_input(self, tensor: torch.Tensor | None, input: torch.Tenso tensor is not None and input.is_floating_point() and tensor.is_floating_point() - and not isinstance(tensor, GGMLTensor) + and not isinstance(tensor, (GGMLTensor, SDNQTensor)) and tensor.dtype != input.dtype ): tensor = tensor.to(dtype=input.dtype) @@ -57,13 +58,13 @@ def forward(self, input: torch.Tensor) -> torch.Tensor: elif input.is_floating_point() and ( ( self.weight.is_floating_point() - and not isinstance(self.weight, GGMLTensor) + and not isinstance(self.weight, (GGMLTensor, SDNQTensor)) and self.weight.dtype != input.dtype ) or ( self.bias is not None and self.bias.is_floating_point() - and not isinstance(self.bias, GGMLTensor) + and not isinstance(self.bias, (GGMLTensor, SDNQTensor)) and self.bias.dtype != input.dtype ) ): diff --git a/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_invoke_linear_8_bit_lt.py b/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_invoke_linear_8_bit_lt.py index 0f538caa5a4..efc2d3738c5 100644 --- a/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_invoke_linear_8_bit_lt.py +++ b/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_invoke_linear_8_bit_lt.py @@ -11,6 +11,7 @@ from invokeai.backend.patches.layers.param_shape_utils import get_param_shape from invokeai.backend.quantization.bnb_llm_int8 import InvokeLinear8bitLt from invokeai.backend.quantization.gguf.ggml_tensor import GGMLTensor +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor class CustomInvokeLinear8bitLt(InvokeLinear8bitLt, CustomModuleMixin): @@ -20,7 +21,7 @@ def _cast_tensor_for_input(self, tensor: torch.Tensor | None, input: torch.Tenso tensor is not None and input.is_floating_point() and tensor.is_floating_point() - and not isinstance(tensor, GGMLTensor) + and not isinstance(tensor, (GGMLTensor, SDNQTensor)) and tensor.dtype != input.dtype ): tensor = tensor.to(dtype=input.dtype) diff --git a/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_invoke_linear_nf4.py b/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_invoke_linear_nf4.py index 82596901704..5f47ce41a4c 100644 --- a/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_invoke_linear_nf4.py +++ b/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_invoke_linear_nf4.py @@ -13,6 +13,7 @@ from invokeai.backend.patches.layers.param_shape_utils import get_param_shape from invokeai.backend.quantization.bnb_nf4 import InvokeLinearNF4 from invokeai.backend.quantization.gguf.ggml_tensor import GGMLTensor +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor class CustomInvokeLinearNF4(InvokeLinearNF4, CustomModuleMixin): @@ -22,7 +23,7 @@ def _cast_tensor_for_input(self, tensor: torch.Tensor | None, input: torch.Tenso tensor is not None and input.is_floating_point() and tensor.is_floating_point() - and not isinstance(tensor, GGMLTensor) + and not isinstance(tensor, (GGMLTensor, SDNQTensor)) and tensor.dtype != input.dtype ): tensor = tensor.to(dtype=input.dtype) diff --git a/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_linear.py b/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_linear.py index 77227583cd9..451c5a9c86b 100644 --- a/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_linear.py +++ b/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_linear.py @@ -10,6 +10,7 @@ from invokeai.backend.patches.layers.flux_control_lora_layer import FluxControlLoRALayer from invokeai.backend.patches.layers.lora_layer import LoRALayer from invokeai.backend.quantization.gguf.ggml_tensor import GGMLTensor +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor def linear_lora_forward(input: torch.Tensor, lora_layer: LoRALayer, lora_weight: float) -> torch.Tensor: @@ -82,7 +83,7 @@ def _cast_tensor_for_input(self, tensor: torch.Tensor | None, input: torch.Tenso tensor is not None and input.is_floating_point() and tensor.is_floating_point() - and not isinstance(tensor, GGMLTensor) + and not isinstance(tensor, (GGMLTensor, SDNQTensor)) and tensor.dtype != input.dtype ): tensor = tensor.to(dtype=input.dtype) @@ -111,7 +112,7 @@ def forward(self, input: torch.Tensor) -> torch.Tensor: or ( self.bias is not None and self.bias.is_floating_point() - and not isinstance(self.bias, GGMLTensor) + and not isinstance(self.bias, (GGMLTensor, SDNQTensor)) and self.bias.dtype != input.dtype ) ): diff --git a/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_module_mixin.py b/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_module_mixin.py index 08ad15c4b6f..9411b0d5b2b 100644 --- a/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_module_mixin.py +++ b/invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_module_mixin.py @@ -5,6 +5,7 @@ from invokeai.backend.patches.layers.base_layer_patch import BaseLayerPatch from invokeai.backend.patches.layers.param_shape_utils import get_param_shape from invokeai.backend.quantization.gguf.ggml_tensor import GGMLTensor +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor class CustomModuleMixin: @@ -58,6 +59,8 @@ def _aggregate_patch_parameters( # Move to device and dequantize here. Doing it in the patch layer can result in redundant casts / # dequantizations. orig_params[param_name] = param.to(device=device).get_dequantized_tensor() + elif type(param) is SDNQTensor: + orig_params[param_name] = param.to(device=device).get_dequantized_tensor() else: orig_params[param_name] = torch.empty(get_param_shape(param), device="meta") diff --git a/invokeai/backend/model_manager/load/model_loaders/flux.py b/invokeai/backend/model_manager/load/model_loaders/flux.py index c385a5dd819..03ec7a7aa19 100644 --- a/invokeai/backend/model_manager/load/model_loaders/flux.py +++ b/invokeai/backend/model_manager/load/model_loaders/flux.py @@ -6,6 +6,7 @@ import accelerate import torch +from diffusers import AutoencoderKL from safetensors.torch import load_file from transformers import ( AutoConfig, @@ -49,8 +50,16 @@ Main_Checkpoint_FLUX_Config, Main_GGUF_Flux2_Config, Main_GGUF_FLUX_Config, + Main_SDNQ_Diffusers_Flux2_Config, + Main_SDNQ_Diffusers_FLUX_Config, + Main_SDNQ_Flux2_Config, + Main_SDNQ_FLUX_Config, +) +from invokeai.backend.model_manager.configs.t5_encoder import ( + T5Encoder_BnBLLMint8_Config, + T5Encoder_SDNQ_Config, + T5Encoder_T5Encoder_Config, ) -from invokeai.backend.model_manager.configs.t5_encoder import T5Encoder_BnBLLMint8_Config, T5Encoder_T5Encoder_Config from invokeai.backend.model_manager.configs.vae import VAE_Checkpoint_Config_Base, VAE_Checkpoint_Flux2_Config from invokeai.backend.model_manager.load.load_default import ModelLoader from invokeai.backend.model_manager.load.model_loader_registry import ModelLoaderRegistry @@ -68,6 +77,7 @@ ) from invokeai.backend.quantization.gguf.loaders import gguf_sd_loader from invokeai.backend.quantization.gguf.utils import TORCH_COMPATIBLE_QTYPES +from invokeai.backend.quantization.sdnq.loaders import sdnq_sd_loader from invokeai.backend.util.silence_warnings import SilenceWarnings try: @@ -504,6 +514,54 @@ def _load_model( ) +@ModelLoaderRegistry.register(base=BaseModelType.Any, type=ModelType.T5Encoder, format=ModelFormat.SDNQQuantized) +class T5EncoderSDNQLoader(ModelLoader): + """Class to load SDNQ-quantized T5 Encoder models.""" + + def _load_model( + self, + config: AnyModelConfig, + submodel_type: Optional[SubModelType] = None, + ) -> AnyModel: + if not isinstance(config, T5Encoder_SDNQ_Config): + raise ValueError("Only T5Encoder_SDNQ_Config models are supported here.") + + match submodel_type: + case SubModelType.Tokenizer2 | SubModelType.Tokenizer3: + return T5TokenizerFast.from_pretrained( + Path(config.path) / "tokenizer_2", max_length=512, local_files_only=True + ) + case SubModelType.TextEncoder2 | SubModelType.TextEncoder3: + return self._load_text_encoder(config) + + raise ValueError( + f"Only Tokenizer and TextEncoder submodels are currently supported. Received: {submodel_type.value if submodel_type else 'None'}" + ) + + def _load_text_encoder(self, config: T5Encoder_SDNQ_Config) -> AnyModel: + # Two layouts: either config.path is the pipeline root (T5 lives under text_encoder_2/), + # or config.path is the text_encoder_2 folder itself (FluxPipeline submodel case). + base = Path(config.path) + nested = base / "text_encoder_2" + te_dir = nested if (nested / "config.json").exists() else base + + model_config = AutoConfig.from_pretrained(te_dir, local_files_only=True) + with accelerate.init_empty_weights(): + model = AutoModelForTextEncoding.from_config(model_config) + + sd = sdnq_sd_loader(te_dir, compute_dtype=torch.bfloat16) + + # T5's embed_tokens and shared point to the same parameter; the SDNQ state dict only carries one of them. + missing_keys, unexpected_keys = model.load_state_dict(sd, strict=False, assign=True) + assert len(unexpected_keys) == 0, f"Unexpected keys loading SDNQ T5: {unexpected_keys}" + assert set(missing_keys) <= {"encoder.embed_tokens.weight"}, ( + f"Unexpected missing keys loading SDNQ T5: {missing_keys}" + ) + if "encoder.embed_tokens.weight" in missing_keys: + model.encoder.embed_tokens.weight = model.shared.weight + return model + + @ModelLoaderRegistry.register(base=BaseModelType.Flux, type=ModelType.Main, format=ModelFormat.Checkpoint) class FluxCheckpointModel(ModelLoader): """Class to load main models.""" @@ -1116,6 +1174,170 @@ def _dequantize_fp8_weights(self, sd: dict) -> dict: return sd +@ModelLoaderRegistry.register(base=BaseModelType.Flux2, type=ModelType.Main, format=ModelFormat.SDNQQuantized) +class Flux2SDNQCheckpointModel(ModelLoader): + """Class to load SDNQ-quantized FLUX.2 transformer models (e.g. Klein 4B / 9B). + + The checkpoint is expected to be in diffusers layout (i.e. the same key naming as + Flux2Transformer2DModel.state_dict()), since SDNQ tooling typically operates on + diffusers state dicts. BFL-layout SDNQ FLUX.2 checkpoints are not supported here. + """ + + def _load_model( + self, + config: AnyModelConfig, + submodel_type: Optional[SubModelType] = None, + ) -> AnyModel: + if not isinstance(config, (Main_SDNQ_Flux2_Config, Main_SDNQ_Diffusers_Flux2_Config)): + raise ValueError( + "Only Main_SDNQ_Flux2_Config or Main_SDNQ_Diffusers_Flux2_Config models are supported here." + ) + + # Single-file SDNQ FLUX.2 checkpoints only ship the transformer. + if isinstance(config, Main_SDNQ_Flux2_Config): + if submodel_type == SubModelType.Transformer: + return self._load_from_singlefile(config) + raise ValueError( + f"Single-file SDNQ FLUX.2 checkpoints only provide the Transformer submodel. " + f"Received: {submodel_type.value if submodel_type else 'None'}" + ) + + # Full Flux2 pipeline folder — dispatch each submodel from its own subfolder. + match submodel_type: + case SubModelType.Transformer: + return self._load_transformer_from_folder(config) + case SubModelType.TextEncoder: + return self._load_text_encoder(config) + case SubModelType.Tokenizer: + return self._load_tokenizer(config) + case SubModelType.VAE: + return self._load_vae(config) + + raise ValueError( + f"Unsupported submodel type for SDNQ FLUX.2 pipeline: {submodel_type.value if submodel_type else 'None'}" + ) + + def _load_transformer_from_folder(self, config: Main_SDNQ_Diffusers_Flux2_Config) -> AnyModel: + from diffusers import Flux2Transformer2DModel + + model_path = Path(config.path) + transformer_path = model_path / "transformer" if (model_path / "transformer").is_dir() else model_path + + with accelerate.init_empty_weights(): + model = Flux2Transformer2DModel.from_config( + Flux2Transformer2DModel.load_config(transformer_path, local_files_only=True) + ) + + sd = sdnq_sd_loader(transformer_path, compute_dtype=torch.bfloat16) + model.load_state_dict(sd, assign=True, strict=False) + return model + + def _load_text_encoder(self, config: Main_SDNQ_Diffusers_Flux2_Config) -> AnyModel: + from transformers import AutoConfig, Qwen3ForCausalLM + + te_dir = Path(config.path) / "text_encoder" + te_config = AutoConfig.from_pretrained(te_dir, local_files_only=True) + with accelerate.init_empty_weights(): + model = Qwen3ForCausalLM(te_config) + + sd = sdnq_sd_loader(te_dir, compute_dtype=torch.bfloat16) + missing, unexpected = model.load_state_dict(sd, assign=True, strict=False) + if unexpected: + raise ValueError(f"Unexpected keys loading SDNQ Qwen3 text encoder: {unexpected}") + if missing and missing != ["lm_head.weight"]: + raise ValueError(f"Unexpected missing keys loading SDNQ Qwen3 text encoder: {missing}") + if missing == ["lm_head.weight"]: + model.lm_head.weight = model.model.embed_tokens.weight + return model + + def _load_tokenizer(self, config: Main_SDNQ_Diffusers_Flux2_Config) -> AnyModel: + from transformers import AutoTokenizer + + tok_dir = Path(config.path) / "tokenizer" + return AutoTokenizer.from_pretrained(tok_dir, local_files_only=True) + + def _load_vae(self, config: Main_SDNQ_Diffusers_Flux2_Config) -> AnyModel: + # FLUX.2 Klein uses AutoencoderKLFlux2 (not the generic AutoencoderKL). Both ship as + # plain bf16 in this pipeline (the VAE itself isn't SDNQ-quantized). + from diffusers import AutoencoderKL, AutoencoderKLFlux2 + + vae_dir = Path(config.path) / "vae" + # Pick the right class based on what the on-disk config.json declares. + try: + cls_name = AutoencoderKL.load_config(vae_dir, local_files_only=True).get("_class_name", "") + except Exception: + cls_name = "" + if cls_name == "AutoencoderKLFlux2": + return AutoencoderKLFlux2.from_pretrained(vae_dir, local_files_only=True) + return AutoencoderKL.from_pretrained(vae_dir, local_files_only=True) + + def _load_from_singlefile(self, config: Main_SDNQ_Flux2_Config) -> AnyModel: + from diffusers import Flux2Transformer2DModel + + model_path = Path(config.path) + + sd = sdnq_sd_loader(model_path, compute_dtype=torch.bfloat16) + + # Detect architecture from state dict shapes. SDNQTensor.shape returns the + # *dequantized* shape, so this works identically to the fp16 path. + double_block_indices = [ + int(k.split(".")[1]) for k in sd.keys() if isinstance(k, str) and k.startswith("transformer_blocks.") + ] + single_block_indices = [ + int(k.split(".")[1]) for k in sd.keys() if isinstance(k, str) and k.startswith("single_transformer_blocks.") + ] + num_layers = max(double_block_indices) + 1 if double_block_indices else 5 + num_single_layers = max(single_block_indices) + 1 if single_block_indices else 20 + + context_embedder_weight = sd.get("context_embedder.weight") + if context_embedder_weight is not None: + hidden_size = context_embedder_weight.shape[0] + joint_attention_dim = context_embedder_weight.shape[1] + else: + hidden_size = 3072 + joint_attention_dim = 7680 + + x_embedder_weight = sd.get("x_embedder.weight") + in_channels = x_embedder_weight.shape[1] if x_embedder_weight is not None else 128 + + attention_head_dim = 128 + num_attention_heads = hidden_size // attention_head_dim + + has_guidance = "time_guidance_embed.guidance_embedder.linear_1.weight" in sd + + with SilenceWarnings(): + with accelerate.init_empty_weights(): + model = Flux2Transformer2DModel( + in_channels=in_channels, + out_channels=in_channels, + num_layers=num_layers, + num_single_layers=num_single_layers, + attention_head_dim=attention_head_dim, + num_attention_heads=num_attention_heads, + joint_attention_dim=joint_attention_dim, + patch_size=1, + ) + + # Klein variants ship without guidance embeddings — zero-fill from the timestep + # embedder dimensions so load_state_dict has a tensor for those slots. + if not has_guidance: + timestep_linear1 = sd.get("time_guidance_embed.timestep_embedder.linear_1.weight") + if timestep_linear1 is not None: + out_features, in_features = timestep_linear1.shape[0], timestep_linear1.shape[1] + sd["time_guidance_embed.guidance_embedder.linear_1.weight"] = torch.zeros( + out_features, in_features, dtype=torch.bfloat16 + ) + timestep_linear2 = sd.get("time_guidance_embed.timestep_embedder.linear_2.weight") + if timestep_linear2 is not None: + out2, in2 = timestep_linear2.shape[0], timestep_linear2.shape[1] + sd["time_guidance_embed.guidance_embedder.linear_2.weight"] = torch.zeros( + out2, in2, dtype=torch.bfloat16 + ) + + model.load_state_dict(sd, assign=True) + return model + + @ModelLoaderRegistry.register(base=BaseModelType.Flux2, type=ModelType.Main, format=ModelFormat.GGUFQuantized) class Flux2GGUFCheckpointModel(ModelLoader): """Class to load GGUF-quantized FLUX.2 transformer models.""" @@ -1475,3 +1697,379 @@ def _load_model( model.load_state_dict(sd, assign=True) model.to(dtype=torch.bfloat16) return model + + +def _is_sdnq_folder(folder_path: Path) -> bool: + """Check if a folder contains SDNQ-quantized model weights.""" + import json + + quant_config_path = folder_path / "quantization_config.json" + if quant_config_path.exists(): + try: + with open(quant_config_path, "r", encoding="utf-8") as f: + quant_config = json.load(f) + if quant_config.get("quant_method") == "sdnq": + return True + except (json.JSONDecodeError, OSError): + pass + return False + + +@ModelLoaderRegistry.register(base=BaseModelType.Flux, type=ModelType.Main, format=ModelFormat.SDNQQuantized) +class FluxSDNQDiffusersModel(ModelLoader): + """Class to load SDNQ-quantized Flux models in diffusers format.""" + + def _load_model( + self, + config: AnyModelConfig, + submodel_type: Optional[SubModelType] = None, + ) -> AnyModel: + print( + f"[SDNQ] FluxSDNQDiffusersModel._load_model called with config={type(config).__name__}, submodel={submodel_type}" + ) + # Handle single-file SDNQ checkpoint (Main_SDNQ_FLUX_Config) + if isinstance(config, Main_SDNQ_FLUX_Config): + if submodel_type == SubModelType.Transformer: + return self._load_sdnq_transformer_checkpoint(config) + raise ValueError( + f"Only Transformer submodels are supported for checkpoint format. Received: {submodel_type}" + ) + + # Handle diffusers-format SDNQ model (Main_SDNQ_Diffusers_FLUX_Config) + if not isinstance(config, Main_SDNQ_Diffusers_FLUX_Config): + raise ValueError(f"Expected Main_SDNQ_Diffusers_FLUX_Config, got {type(config).__name__}") + + if submodel_type is None: + raise ValueError("A submodel type must be provided when loading main pipelines.") + + model_path = Path(config.path) + submodel_path = model_path / submodel_type.value + + match submodel_type: + case SubModelType.Transformer: + return self._load_sdnq_transformer(submodel_path, config) + case SubModelType.TextEncoder: + return self._load_text_encoder(submodel_path) + case SubModelType.TextEncoder2: + return self._load_text_encoder_2(submodel_path) + case SubModelType.Tokenizer: + return CLIPTokenizer.from_pretrained(submodel_path, local_files_only=True) + case SubModelType.Tokenizer2: + return T5TokenizerFast.from_pretrained(submodel_path, max_length=512, local_files_only=True) + case SubModelType.VAE: + return self._load_vae(submodel_path) + case _: + raise ValueError(f"Unsupported submodel type: {submodel_type}") + + def _load_sdnq_transformer_checkpoint(self, config: Main_SDNQ_FLUX_Config) -> AnyModel: + """Load SDNQ transformer from single-file checkpoint.""" + model_path = Path(config.path) + + with accelerate.init_empty_weights(): + model = Flux(get_flux_transformers_params(config.variant)) + + sd = sdnq_sd_loader(model_path, compute_dtype=torch.bfloat16) + + # Handle ComfyUI bundle format + if "model.diffusion_model.double_blocks.0.img_attn.norm.key_norm.scale" in sd: + sd = convert_bundle_to_flux_transformer_checkpoint(sd) + + model.load_state_dict(sd, assign=True) + return model + + def _load_sdnq_transformer(self, transformer_path: Path, config: Main_SDNQ_Diffusers_FLUX_Config) -> AnyModel: + """Load SDNQ-quantized transformer from diffusers folder.""" + print(f"[SDNQ] _load_sdnq_transformer called for {transformer_path}") + with accelerate.init_empty_weights(): + model = Flux(get_flux_transformers_params(config.variant)) + + sd = sdnq_sd_loader(transformer_path, compute_dtype=torch.bfloat16) + + # Convert from diffusers format to BFL format + sd = self._convert_diffusers_sd_to_bfl(sd) + + model.load_state_dict(sd, assign=True) + return model + + def _convert_diffusers_sd_to_bfl(self, sd: dict[str, torch.Tensor]) -> dict[str, torch.Tensor]: + """Convert a Flux transformer state dict from diffusers format to BFL format. + + Note: For SDNQTensor objects, Q/K/V tensors are dequantized before fusion since + torch.cat doesn't work with quantized tensors. Other layers retain quantization. + """ + from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor + + # Helper to dequantize SDNQTensor or return as-is + def maybe_dequantize(t: torch.Tensor) -> torch.Tensor: + if isinstance(t, SDNQTensor): + return t.get_dequantized_tensor() + return t + + # Helper to fuse weights, handling SDNQTensor + def fuse_weights(*tensors: torch.Tensor) -> torch.Tensor: + dequantized = [maybe_dequantize(t) for t in tensors] + return torch.cat(dequantized, dim=0) + + def _swap_scale_shift_halves(t: torch.Tensor) -> torch.Tensor: + """Swap the (scale, shift) halves along dim 0 to (shift, scale). + + diffusers' AdaLayerNormContinuous packs (scale, shift); BFL's LastLayer expects + (shift, scale). Same memory, different interpretation — without this swap the final + normalisation modulation is permuted and the output is high-frequency noise. + """ + t = maybe_dequantize(t) + if t.dim() < 1 or t.shape[0] % 2 != 0: + return t + scale, shift = t.chunk(2, dim=0) + return torch.cat([shift, scale], dim=0) + + # Make a shallow copy so we can pop keys + sd = sd.copy() + new_sd: dict[str, torch.Tensor] = {} + + # Basic 1-to-1 key conversions + basic_key_map = { + # txt_in keys + "context_embedder.bias": "txt_in.bias", + "context_embedder.weight": "txt_in.weight", + # guidance_in MLPEmbedder keys + "time_text_embed.guidance_embedder.linear_1.bias": "guidance_in.in_layer.bias", + "time_text_embed.guidance_embedder.linear_1.weight": "guidance_in.in_layer.weight", + "time_text_embed.guidance_embedder.linear_2.bias": "guidance_in.out_layer.bias", + "time_text_embed.guidance_embedder.linear_2.weight": "guidance_in.out_layer.weight", + # vector_in MLPEmbedder keys + "time_text_embed.text_embedder.linear_1.bias": "vector_in.in_layer.bias", + "time_text_embed.text_embedder.linear_1.weight": "vector_in.in_layer.weight", + "time_text_embed.text_embedder.linear_2.bias": "vector_in.out_layer.bias", + "time_text_embed.text_embedder.linear_2.weight": "vector_in.out_layer.weight", + # time_in MLPEmbedder keys + "time_text_embed.timestep_embedder.linear_1.bias": "time_in.in_layer.bias", + "time_text_embed.timestep_embedder.linear_1.weight": "time_in.in_layer.weight", + "time_text_embed.timestep_embedder.linear_2.bias": "time_in.out_layer.bias", + "time_text_embed.timestep_embedder.linear_2.weight": "time_in.out_layer.weight", + # img_in keys + "x_embedder.bias": "img_in.bias", + "x_embedder.weight": "img_in.weight", + # final_layer keys + "proj_out.bias": "final_layer.linear.bias", + "proj_out.weight": "final_layer.linear.weight", + # norm_out.linear is the final AdaLayerNormContinuous. diffusers packs the linear + # output as (scale, shift); BFL's LastLayer packs as (shift, scale). Swap the + # halves of the weight and bias to keep the math correct. + "norm_out.linear.bias": "final_layer.adaLN_modulation.1.bias", + "norm_out.linear.weight": "final_layer.adaLN_modulation.1.weight", + } + # Keys whose first-axis halves (scale, shift) must be swapped to (shift, scale) for BFL. + SWAP_SCALE_SHIFT_KEYS = { + "norm_out.linear.bias", + "norm_out.linear.weight", + } + for old_key, new_key in basic_key_map.items(): + v = sd.pop(old_key, None) + if v is not None: + if old_key in SWAP_SCALE_SHIFT_KEYS: + v = _swap_scale_shift_halves(v) + new_sd[new_key] = v + + # Handle the double_blocks (19 blocks for FLUX) + block_index = 0 + while f"transformer_blocks.{block_index}.attn.add_q_proj.bias" in sd: + from_prefix = f"transformer_blocks.{block_index}" + to_prefix = f"double_blocks.{block_index}" + + # txt_attn.qkv (fuse add_q, add_k, add_v) + new_sd[f"{to_prefix}.txt_attn.qkv.bias"] = fuse_weights( + sd.pop(f"{from_prefix}.attn.add_q_proj.bias"), + sd.pop(f"{from_prefix}.attn.add_k_proj.bias"), + sd.pop(f"{from_prefix}.attn.add_v_proj.bias"), + ) + new_sd[f"{to_prefix}.txt_attn.qkv.weight"] = fuse_weights( + sd.pop(f"{from_prefix}.attn.add_q_proj.weight"), + sd.pop(f"{from_prefix}.attn.add_k_proj.weight"), + sd.pop(f"{from_prefix}.attn.add_v_proj.weight"), + ) + + # img_attn.qkv (fuse to_q, to_k, to_v) + new_sd[f"{to_prefix}.img_attn.qkv.bias"] = fuse_weights( + sd.pop(f"{from_prefix}.attn.to_q.bias"), + sd.pop(f"{from_prefix}.attn.to_k.bias"), + sd.pop(f"{from_prefix}.attn.to_v.bias"), + ) + new_sd[f"{to_prefix}.img_attn.qkv.weight"] = fuse_weights( + sd.pop(f"{from_prefix}.attn.to_q.weight"), + sd.pop(f"{from_prefix}.attn.to_k.weight"), + sd.pop(f"{from_prefix}.attn.to_v.weight"), + ) + + # 1-to-1 key mappings for double block + double_block_key_map = { + # img_attn + "attn.norm_k.weight": "img_attn.norm.key_norm.scale", + "attn.norm_q.weight": "img_attn.norm.query_norm.scale", + "attn.to_out.0.weight": "img_attn.proj.weight", + "attn.to_out.0.bias": "img_attn.proj.bias", + # img_mlp + "ff.net.0.proj.weight": "img_mlp.0.weight", + "ff.net.0.proj.bias": "img_mlp.0.bias", + "ff.net.2.weight": "img_mlp.2.weight", + "ff.net.2.bias": "img_mlp.2.bias", + # img_mod + "norm1.linear.weight": "img_mod.lin.weight", + "norm1.linear.bias": "img_mod.lin.bias", + # txt_attn + "attn.norm_added_q.weight": "txt_attn.norm.query_norm.scale", + "attn.norm_added_k.weight": "txt_attn.norm.key_norm.scale", + "attn.to_add_out.weight": "txt_attn.proj.weight", + "attn.to_add_out.bias": "txt_attn.proj.bias", + # txt_mlp + "ff_context.net.0.proj.weight": "txt_mlp.0.weight", + "ff_context.net.0.proj.bias": "txt_mlp.0.bias", + "ff_context.net.2.weight": "txt_mlp.2.weight", + "ff_context.net.2.bias": "txt_mlp.2.bias", + # txt_mod + "norm1_context.linear.weight": "txt_mod.lin.weight", + "norm1_context.linear.bias": "txt_mod.lin.bias", + } + for from_key, to_key in double_block_key_map.items(): + v = sd.pop(f"{from_prefix}.{from_key}", None) + if v is not None: + new_sd[f"{to_prefix}.{to_key}"] = v + + block_index += 1 + + # Handle the single_blocks (38 blocks for FLUX) + block_index = 0 + while f"single_transformer_blocks.{block_index}.attn.to_q.bias" in sd: + from_prefix = f"single_transformer_blocks.{block_index}" + to_prefix = f"single_blocks.{block_index}" + + # linear1 (fuse to_q, to_k, to_v, proj_mlp) + new_sd[f"{to_prefix}.linear1.bias"] = fuse_weights( + sd.pop(f"{from_prefix}.attn.to_q.bias"), + sd.pop(f"{from_prefix}.attn.to_k.bias"), + sd.pop(f"{from_prefix}.attn.to_v.bias"), + sd.pop(f"{from_prefix}.proj_mlp.bias"), + ) + new_sd[f"{to_prefix}.linear1.weight"] = fuse_weights( + sd.pop(f"{from_prefix}.attn.to_q.weight"), + sd.pop(f"{from_prefix}.attn.to_k.weight"), + sd.pop(f"{from_prefix}.attn.to_v.weight"), + sd.pop(f"{from_prefix}.proj_mlp.weight"), + ) + + # 1-to-1 key mappings for single block + single_block_key_map = { + # linear2 + "proj_out.weight": "linear2.weight", + "proj_out.bias": "linear2.bias", + # modulation + "norm.linear.weight": "modulation.lin.weight", + "norm.linear.bias": "modulation.lin.bias", + # norm + "attn.norm_k.weight": "norm.key_norm.scale", + "attn.norm_q.weight": "norm.query_norm.scale", + } + for from_key, to_key in single_block_key_map.items(): + v = sd.pop(f"{from_prefix}.{from_key}", None) + if v is not None: + new_sd[f"{to_prefix}.{to_key}"] = v + + block_index += 1 + + # Any remaining keys that weren't converted - just pass through + for k, v in sd.items(): + if k not in new_sd: + new_sd[k] = v + + return new_sd + + def _load_text_encoder(self, text_encoder_path: Path) -> AnyModel: + """Load text encoder (CLIP) - SDNQ or normal.""" + if _is_sdnq_folder(text_encoder_path): + # SDNQ CLIP - need custom loading + return self._load_sdnq_clip(text_encoder_path) + # Normal CLIP + return CLIPTextModel.from_pretrained(text_encoder_path, local_files_only=True) + + def _load_text_encoder_2(self, text_encoder_path: Path) -> AnyModel: + """Load text encoder 2 (T5) - SDNQ or normal.""" + if _is_sdnq_folder(text_encoder_path): + # SDNQ T5 - need custom loading + return self._load_sdnq_t5(text_encoder_path) + # Normal T5 + return T5EncoderModel.from_pretrained( + text_encoder_path, + torch_dtype="auto", + low_cpu_mem_usage=True, + local_files_only=True, + ) + + def _load_sdnq_clip(self, clip_path: Path) -> AnyModel: + """Load SDNQ-quantized CLIP text encoder.""" + from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor + + # Load SDNQ state dict + sd = sdnq_sd_loader(clip_path, compute_dtype=torch.bfloat16) + + # Load config and create model + model_config = AutoConfig.from_pretrained(clip_path, local_files_only=True) + with accelerate.init_empty_weights(): + model = CLIPTextModel(model_config) + + model.load_state_dict(sd, strict=False, assign=True) + + # Dequantize embedding layer + if hasattr(model, "text_model") and hasattr(model.text_model, "embeddings"): + embed_weight = model.text_model.embeddings.token_embedding.weight + if isinstance(embed_weight, SDNQTensor): + dequantized = embed_weight.get_dequantized_tensor() + model.text_model.embeddings.token_embedding.weight = torch.nn.Parameter( + dequantized, requires_grad=False + ) + + return model + + def _load_sdnq_t5(self, t5_path: Path) -> AnyModel: + """Load SDNQ-quantized T5 text encoder.""" + from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor + + # Load SDNQ state dict + sd = sdnq_sd_loader(t5_path, compute_dtype=torch.bfloat16) + + # Load config and create model + model_config = AutoConfig.from_pretrained(t5_path, local_files_only=True) + with accelerate.init_empty_weights(): + model = AutoModelForTextEncoding.from_config(model_config) + + model.load_state_dict(sd, strict=False, assign=True) + + # Dequantize shared embedding + if hasattr(model, "shared") and isinstance(model.shared.weight, SDNQTensor): + dequantized = model.shared.weight.get_dequantized_tensor() + model.shared.weight = torch.nn.Parameter(dequantized, requires_grad=False) + + # Re-tie weights after dequantization + if hasattr(model, "encoder") and hasattr(model.encoder, "embed_tokens"): + if model.encoder.embed_tokens.weight is not model.shared.weight: + model.encoder.embed_tokens.weight = model.shared.weight + + return model + + def _load_vae(self, vae_path: Path) -> AnyModel: + """Load VAE - SDNQ or normal.""" + if _is_sdnq_folder(vae_path): + return self._load_sdnq_vae(vae_path) + # Normal VAE + return AutoencoderKL.from_pretrained(vae_path, local_files_only=True) + + def _load_sdnq_vae(self, vae_path: Path) -> AnyModel: + """Load SDNQ-quantized VAE.""" + # Load SDNQ state dict + sd = sdnq_sd_loader(vae_path, compute_dtype=torch.bfloat16) + + # Load config and create model + with accelerate.init_empty_weights(): + model = AutoencoderKL.from_config(AutoencoderKL.load_config(vae_path, local_files_only=True)) + + model.load_state_dict(sd, strict=False, assign=True) + return model diff --git a/invokeai/backend/model_manager/load/model_loaders/vae.py b/invokeai/backend/model_manager/load/model_loaders/vae.py index 720821f3af8..60df70946f5 100644 --- a/invokeai/backend/model_manager/load/model_loaders/vae.py +++ b/invokeai/backend/model_manager/load/model_loaders/vae.py @@ -1,9 +1,12 @@ # Copyright (c) 2024, Lincoln D. Stein and the InvokeAI Development Team """Class for VAE model loading in InvokeAI.""" +from pathlib import Path from typing import Optional +import accelerate from diffusers.models.autoencoders.autoencoder_kl import AutoencoderKL +from safetensors import safe_open from invokeai.backend.model_manager.configs.factory import AnyModelConfig from invokeai.backend.model_manager.configs.vae import ( @@ -20,6 +23,28 @@ ModelType, SubModelType, ) +from invokeai.backend.quantization.sdnq.loaders import sdnq_sd_loader + + +def _is_sdnq_vae_folder(path: Path) -> bool: + """Check if a VAE folder contains SDNQ-quantized weights.""" + model_file = path / "diffusion_pytorch_model.safetensors" + if not model_file.exists(): + model_file = path / "model.safetensors" + if not model_file.exists(): + return False + + try: + with safe_open(model_file, framework="pt", device="cpu") as f: + keys = set(f.keys()) + for key in keys: + if key.endswith(".weight"): + base = key[:-7] + if f"{base}.scale" in keys: + return True + except Exception: + pass + return False @ModelLoaderRegistry.register(base=BaseModelType.Any, type=ModelType.VAE, format=ModelFormat.Diffusers) @@ -46,8 +71,14 @@ def _load_model( config.path, torch_dtype=self._torch_dtype, ) - else: - return super()._load_model(config, submodel_type) + + model_path = Path(config.path) + + # Check if this is an SDNQ-quantized VAE folder + if model_path.is_dir() and _is_sdnq_vae_folder(model_path): + return self._load_sdnq_vae(model_path) + + return super()._load_model(config, submodel_type) def _load_qwen_image_vae(self, config: VAE_Checkpoint_QwenImage_Config) -> AnyModel: """Load a Qwen Image VAE from a single safetensors file. @@ -78,3 +109,21 @@ def _load_qwen_image_vae(self, config: VAE_Checkpoint_QwenImage_Config) -> AnyMo model.load_state_dict(sd, strict=True, assign=True) model.eval() return model + + def _load_sdnq_vae(self, model_path: Path) -> AnyModel: + """Load SDNQ-quantized VAE with on-the-fly dequantization.""" + # Find the safetensors file + model_file = model_path / "diffusion_pytorch_model.safetensors" + if not model_file.exists(): + model_file = model_path / "model.safetensors" + + # Load SDNQ state dict + sd = sdnq_sd_loader(model_file, compute_dtype=self._torch_dtype) + + # Create empty model from config + with accelerate.init_empty_weights(): + model = AutoencoderKL.from_config(AutoencoderKL.load_config(model_path, local_files_only=True)) + + # Load state dict with SDNQTensor objects + model.load_state_dict(sd, strict=False, assign=True) + return model diff --git a/invokeai/backend/model_manager/load/model_loaders/z_image.py b/invokeai/backend/model_manager/load/model_loaders/z_image.py index 6c2102933af..057d938ff2f 100644 --- a/invokeai/backend/model_manager/load/model_loaders/z_image.py +++ b/invokeai/backend/model_manager/load/model_loaders/z_image.py @@ -11,11 +11,18 @@ from invokeai.backend.model_manager.configs.base import Checkpoint_Config_Base, Diffusers_Config_Base from invokeai.backend.model_manager.configs.controlnet import ControlNet_Checkpoint_ZImage_Config from invokeai.backend.model_manager.configs.factory import AnyModelConfig -from invokeai.backend.model_manager.configs.main import Main_Checkpoint_ZImage_Config, Main_GGUF_ZImage_Config +from invokeai.backend.model_manager.configs.main import ( + Main_Checkpoint_ZImage_Config, + Main_GGUF_ZImage_Config, + Main_SDNQ_Diffusers_ZImage_Config, + Main_SDNQ_ZImage_Config, +) from invokeai.backend.model_manager.configs.qwen3_encoder import ( Qwen3Encoder_Checkpoint_Config, Qwen3Encoder_GGUF_Config, Qwen3Encoder_Qwen3Encoder_Config, + Qwen3Encoder_SDNQ_Config, + Qwen3Encoder_SDNQ_Folder_Config, ) from invokeai.backend.model_manager.load.load_default import ModelLoader from invokeai.backend.model_manager.load.model_loader_registry import ModelLoaderRegistry @@ -28,6 +35,7 @@ SubModelType, ) from invokeai.backend.quantization.gguf.loaders import gguf_sd_loader +from invokeai.backend.quantization.sdnq.loaders import sdnq_sd_loader from invokeai.backend.util.devices import TorchDevice @@ -141,17 +149,25 @@ def _load_model( raise Exception("A submodel type must be provided when loading main pipelines.") model_path = Path(config.path) + submodel_path = model_path / submodel_type.value + + # Check if submodel folder has SDNQ quantization - if so, use SDNQ loader + if self._is_sdnq_folder(submodel_path): + if submodel_type == SubModelType.TextEncoder: + return self._load_sdnq_text_encoder(submodel_path) + elif submodel_type == SubModelType.Transformer: + return self._load_sdnq_transformer(submodel_path) + load_class = self.get_hf_load_class(model_path, submodel_type) repo_variant = config.repo_variant if isinstance(config, Diffusers_Config_Base) else None variant = repo_variant.value if repo_variant else None - model_path = model_path / submodel_type.value # Z-Image prefers bfloat16, but use safe dtype based on target device capabilities. target_device = TorchDevice.choose_torch_device() dtype = TorchDevice.choose_bfloat16_safe_dtype(target_device) try: result: AnyModel = load_class.from_pretrained( - model_path, + submodel_path, torch_dtype=dtype, variant=variant, ) @@ -159,13 +175,184 @@ def _load_model( if variant and "no file named" in str( e ): # try without the variant, just in case user's preferences changed - result = load_class.from_pretrained(model_path, torch_dtype=dtype) + result = load_class.from_pretrained(submodel_path, torch_dtype=dtype) else: raise e result = self._apply_fp8_layerwise_casting(result, config, submodel_type) return result + def _is_sdnq_folder(self, folder_path: Path) -> bool: + """Check if a folder contains SDNQ-quantized model weights.""" + import json + + quant_config_path = folder_path / "quantization_config.json" + if quant_config_path.exists(): + with open(quant_config_path, "r", encoding="utf-8") as f: + quant_config = json.load(f) + if quant_config.get("quant_method") == "sdnq": + return True + return False + + def _load_sdnq_text_encoder(self, text_encoder_path: Path) -> AnyModel: + """Load SDNQ-quantized text encoder from folder.""" + from transformers import Qwen3Config, Qwen3ForCausalLM + + from invokeai.backend.quantization.sdnq.loaders import sdnq_sd_loader + from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor + from invokeai.backend.util.logging import InvokeAILogger + + logger = InvokeAILogger.get_logger(self.__class__.__name__) + logger.info(f"Loading SDNQ-quantized text encoder from {text_encoder_path}") + + target_device = TorchDevice.choose_torch_device() + compute_dtype = TorchDevice.choose_bfloat16_safe_dtype(target_device) + + # Load the SDNQ state dict + sd = sdnq_sd_loader(text_encoder_path, compute_dtype=compute_dtype) + + # Determine Qwen model configuration from state dict + layer_count = 0 + for key in sd.keys(): + if isinstance(key, str) and key.startswith("model.layers."): + parts = key.split(".") + if len(parts) > 2: + try: + layer_idx = int(parts[2]) + layer_count = max(layer_count, layer_idx + 1) + except ValueError: + pass + + # Get hidden size from embed_tokens weight shape + embed_weight = sd.get("model.embed_tokens.weight") + if embed_weight is None: + raise ValueError("Could not find model.embed_tokens.weight in state dict") + + embed_shape = embed_weight.shape if hasattr(embed_weight, "shape") else embed_weight.tensor_shape + hidden_size = embed_shape[1] + vocab_size = embed_shape[0] + + # Detect attention configuration from layer 0 weights + q_proj_weight = sd.get("model.layers.0.self_attn.q_proj.weight") + k_proj_weight = sd.get("model.layers.0.self_attn.k_proj.weight") + gate_proj_weight = sd.get("model.layers.0.mlp.gate_proj.weight") + + if q_proj_weight is None or k_proj_weight is None or gate_proj_weight is None: + raise ValueError("Could not find attention/mlp weights in state dict") + + q_shape = q_proj_weight.shape if hasattr(q_proj_weight, "shape") else q_proj_weight.tensor_shape + k_shape = k_proj_weight.shape if hasattr(k_proj_weight, "shape") else k_proj_weight.tensor_shape + gate_shape = gate_proj_weight.shape if hasattr(gate_proj_weight, "shape") else gate_proj_weight.tensor_shape + + head_dim = 128 + num_attention_heads = q_shape[0] // head_dim + num_kv_heads = k_shape[0] // head_dim + intermediate_size = gate_shape[0] + + logger.info( + f"Qwen3 SDNQ config: layers={layer_count}, hidden={hidden_size}, " + f"heads={num_attention_heads}, kv_heads={num_kv_heads}" + ) + + qwen_config = Qwen3Config( + vocab_size=vocab_size, + hidden_size=hidden_size, + intermediate_size=intermediate_size, + num_hidden_layers=layer_count, + num_attention_heads=num_attention_heads, + num_key_value_heads=num_kv_heads, + head_dim=head_dim, + max_position_embeddings=40960, + rms_norm_eps=1e-6, + tie_word_embeddings=True, + rope_theta=1000000.0, + use_sliding_window=False, + attention_bias=False, + attention_dropout=0.0, + torch_dtype=compute_dtype, + ) + + with accelerate.init_empty_weights(): + model = Qwen3ForCausalLM(qwen_config) + + model.load_state_dict(sd, strict=False, assign=True) + + # Dequantize embed_tokens weight for embedding lookups + embed_tokens_weight = model.model.embed_tokens.weight + if isinstance(embed_tokens_weight, SDNQTensor): + dequantized = embed_tokens_weight.get_dequantized_tensor() + model.model.embed_tokens.weight = torch.nn.Parameter(dequantized, requires_grad=False) + logger.info("Dequantized embed_tokens weight for embedding lookups") + + # Handle tied weights + if qwen_config.tie_word_embeddings: + if model.lm_head.weight.is_meta: + model.lm_head.weight = model.model.embed_tokens.weight + else: + model.tie_weights() + + # Re-initialize meta tensor buffers + for name, buffer in list(model.named_buffers()): + if buffer.is_meta: + parts = name.rsplit(".", 1) + if len(parts) == 2: + parent = model.get_submodule(parts[0]) + buffer_name = parts[1] + else: + parent = model + buffer_name = name + + if buffer_name == "inv_freq": + base = qwen_config.rope_theta + inv_freq = 1.0 / (base ** (torch.arange(0, head_dim, 2, dtype=torch.float32) / head_dim)) + parent.register_buffer(buffer_name, inv_freq.to(dtype=compute_dtype), persistent=False) + + return model + + def _load_sdnq_transformer(self, transformer_path: Path) -> AnyModel: + """Load SDNQ-quantized transformer from folder.""" + from diffusers import ZImageTransformer2DModel + + from invokeai.backend.quantization.sdnq.loaders import sdnq_sd_loader + from invokeai.backend.util.logging import InvokeAILogger + + logger = InvokeAILogger.get_logger(self.__class__.__name__) + logger.info(f"Loading SDNQ-quantized transformer from {transformer_path}") + + target_device = TorchDevice.choose_torch_device() + compute_dtype = TorchDevice.choose_bfloat16_safe_dtype(target_device) + + # Load the SDNQ state dict + sd = sdnq_sd_loader(transformer_path, compute_dtype=compute_dtype) + + # Check if conversion is needed (original format vs diffusers format) + needs_conversion = any(k.startswith("x_embedder.") for k in sd.keys() if isinstance(k, str)) + if needs_conversion: + sd = _convert_z_image_gguf_to_diffusers(sd) + + # Create an empty model with the default Z-Image config + with accelerate.init_empty_weights(): + model = ZImageTransformer2DModel( + all_patch_size=(2,), + all_f_patch_size=(1,), + in_channels=16, + dim=3840, + n_layers=30, + n_refiner_layers=2, + n_heads=30, + n_kv_heads=30, + norm_eps=1e-05, + qk_norm=True, + cap_feat_dim=2560, + rope_theta=256.0, + t_scale=1000.0, + axes_dims=[32, 48, 48], + axes_lens=[1024, 512, 512], + ) + + model.load_state_dict(sd, assign=True) + return model + @ModelLoaderRegistry.register(base=BaseModelType.ZImage, type=ModelType.Main, format=ModelFormat.Checkpoint) class ZImageCheckpointModel(ModelLoader): @@ -375,6 +562,171 @@ def _load_from_singlefile( return model +@ModelLoaderRegistry.register(base=BaseModelType.ZImage, type=ModelType.Main, format=ModelFormat.SDNQQuantized) +class ZImageSDNQCheckpointModel(ModelLoader): + """Class to load SDNQ-quantized Z-Image transformer models. + + Handles both single-file SDNQ checkpoints (``Main_SDNQ_ZImage_Config``) and full + diffusers-pipeline folders (``Main_SDNQ_Diffusers_ZImage_Config``), where the + quantized weights live under ``transformer/`` alongside a ``config.json`` that + describes the architecture. + """ + + def _load_model( + self, + config: AnyModelConfig, + submodel_type: Optional[SubModelType] = None, + ) -> AnyModel: + if not isinstance(config, (Main_SDNQ_ZImage_Config, Main_SDNQ_Diffusers_ZImage_Config)): + raise ValueError( + "Only Main_SDNQ_ZImage_Config or Main_SDNQ_Diffusers_ZImage_Config models are supported here." + ) + + # Single-file SDNQ checkpoints only carry the transformer. + if isinstance(config, Main_SDNQ_ZImage_Config): + if submodel_type == SubModelType.Transformer: + return self._load_from_singlefile(config) + raise ValueError( + f"Single-file SDNQ Z-Image checkpoints only provide the Transformer submodel. " + f"Received: {submodel_type.value if submodel_type else 'None'}" + ) + + # Full ZImagePipeline folder — dispatch each submodel out of its own subfolder so the + # model can be used as a 'Qwen3 & VAE source model' for other Z-Image runs. + match submodel_type: + case SubModelType.Transformer: + return self._load_from_diffusers_folder(config) + case SubModelType.TextEncoder: + return self._load_text_encoder(config) + case SubModelType.Tokenizer: + return self._load_tokenizer(config) + case SubModelType.VAE: + return self._load_vae(config) + + raise ValueError( + f"Unsupported submodel type for SDNQ ZImagePipeline: {submodel_type.value if submodel_type else 'None'}" + ) + + def _load_text_encoder(self, config: Main_SDNQ_Diffusers_ZImage_Config) -> AnyModel: + from transformers import AutoConfig, Qwen3ForCausalLM + + te_dir = Path(config.path) / "text_encoder" + target_device = TorchDevice.choose_torch_device() + compute_dtype = TorchDevice.choose_bfloat16_safe_dtype(target_device) + + te_config = AutoConfig.from_pretrained(te_dir, local_files_only=True) + with accelerate.init_empty_weights(): + model = Qwen3ForCausalLM(te_config) + + sd = sdnq_sd_loader(te_dir, compute_dtype=compute_dtype) + # Qwen3ForCausalLM may share lm_head.weight with model.embed_tokens.weight; missing keys + # for that tie are expected and handled by re-sharing post-load. + missing, unexpected = model.load_state_dict(sd, assign=True, strict=False) + if unexpected: + raise ValueError(f"Unexpected keys loading SDNQ Qwen3 text encoder: {unexpected}") + if missing and missing != ["lm_head.weight"]: + raise ValueError(f"Unexpected missing keys loading SDNQ Qwen3 text encoder: {missing}") + if missing == ["lm_head.weight"]: + model.lm_head.weight = model.model.embed_tokens.weight + return model + + def _load_tokenizer(self, config: Main_SDNQ_Diffusers_ZImage_Config) -> AnyModel: + tok_dir = Path(config.path) / "tokenizer" + return AutoTokenizer.from_pretrained(tok_dir, local_files_only=True) + + def _load_vae(self, config: Main_SDNQ_Diffusers_ZImage_Config) -> AnyModel: + from diffusers import AutoencoderKL + + vae_dir = Path(config.path) / "vae" + return AutoencoderKL.from_pretrained(vae_dir, local_files_only=True) + + def _load_from_singlefile( + self, + config: Main_SDNQ_ZImage_Config, + ) -> AnyModel: + from diffusers import ZImageTransformer2DModel + + model_path = Path(config.path) + + target_device = TorchDevice.choose_torch_device() + compute_dtype = TorchDevice.choose_bfloat16_safe_dtype(target_device) + + sd = sdnq_sd_loader(model_path, compute_dtype=compute_dtype) + + # Some Z-Image SDNQ models may have keys prefixed with "diffusion_model." or + # "model.diffusion_model." (ComfyUI-style format). Check if we need to strip this prefix. + prefix_to_strip = None + for prefix in ["model.diffusion_model.", "diffusion_model."]: + if any(k.startswith(prefix) for k in sd.keys() if isinstance(k, str)): + prefix_to_strip = prefix + break + + if prefix_to_strip: + stripped_sd = {} + for key, value in sd.items(): + if isinstance(key, str) and key.startswith(prefix_to_strip): + stripped_sd[key[len(prefix_to_strip) :]] = value + else: + stripped_sd[key] = value + sd = stripped_sd + + # Check if conversion is needed (original format vs diffusers format) + needs_conversion = any(k.startswith("x_embedder.") for k in sd.keys() if isinstance(k, str)) + if needs_conversion: + sd = _convert_z_image_gguf_to_diffusers(sd) + + # Create an empty model with the default Z-Image config + with accelerate.init_empty_weights(): + model = ZImageTransformer2DModel( + all_patch_size=(2,), + all_f_patch_size=(1,), + in_channels=16, + dim=3840, + n_layers=30, + n_refiner_layers=2, + n_heads=30, + n_kv_heads=30, + norm_eps=1e-05, + qk_norm=True, + cap_feat_dim=2560, + rope_theta=256.0, + t_scale=1000.0, + axes_dims=[32, 48, 48], + axes_lens=[1024, 512, 512], + ) + + model.load_state_dict(sd, assign=True) + return model + + def _load_from_diffusers_folder( + self, + config: Main_SDNQ_Diffusers_ZImage_Config, + ) -> AnyModel: + from diffusers import ZImageTransformer2DModel + + # When ZImagePipeline is registered with submodels, the transformer submodel's path points + # into transformer/ directly. The pipeline-level Main config has its own path at the root. + # Either way, locate the transformer/config.json + safetensors. + model_path = Path(config.path) + transformer_path = model_path / "transformer" if (model_path / "transformer").is_dir() else model_path + + target_device = TorchDevice.choose_torch_device() + compute_dtype = TorchDevice.choose_bfloat16_safe_dtype(target_device) + + # Build the empty model from the on-disk architecture description so we honor non-default + # axes_lens / dim / etc. that the single-file path hardcodes. + with accelerate.init_empty_weights(): + model = ZImageTransformer2DModel.from_config( + ZImageTransformer2DModel.load_config(transformer_path, local_files_only=True) + ) + + sd = sdnq_sd_loader(transformer_path, compute_dtype=compute_dtype) + # Diffusers-format Z-Image keys already match ZImageTransformer2DModel.state_dict(), + # so no BFL→diffusers conversion is needed here. + model.load_state_dict(sd, assign=True, strict=False) + return model + + @ModelLoaderRegistry.register(base=BaseModelType.Any, type=ModelType.Qwen3Encoder, format=ModelFormat.Qwen3Encoder) class Qwen3EncoderLoader(ModelLoader): """Class to load standalone Qwen3 Encoder models for Z-Image (directory format).""" @@ -1080,3 +1432,176 @@ def _convert_llamacpp_to_pytorch(self, sd: dict[str, Any]) -> dict[str, Any]: new_sd[key] = value return new_sd + + +@ModelLoaderRegistry.register(base=BaseModelType.Any, type=ModelType.Qwen3Encoder, format=ModelFormat.SDNQQuantized) +class Qwen3EncoderSDNQLoader(ModelLoader): + """Class to load SDNQ-quantized Qwen3 Encoder models for Z-Image.""" + + # Default HuggingFace model to load tokenizer from when using SDNQ Qwen3 encoder + DEFAULT_TOKENIZER_SOURCE = "Qwen/Qwen3-4B" + + def _load_model( + self, + config: AnyModelConfig, + submodel_type: Optional[SubModelType] = None, + ) -> AnyModel: + if not isinstance(config, (Qwen3Encoder_SDNQ_Config, Qwen3Encoder_SDNQ_Folder_Config)): + raise ValueError( + "Only Qwen3Encoder_SDNQ_Config or Qwen3Encoder_SDNQ_Folder_Config models are supported here." + ) + + match submodel_type: + case SubModelType.TextEncoder: + return self._load_from_sdnq(config) + case SubModelType.Tokenizer: + return self._load_tokenizer_with_offline_fallback() + + submodel_str = submodel_type.value if submodel_type else "None" + raise ValueError(f"Only TextEncoder and Tokenizer submodels are supported. Received: {submodel_str}") + + def _load_tokenizer_with_offline_fallback(self) -> AnyModel: + """Load tokenizer with local_files_only fallback for offline support.""" + try: + return AutoTokenizer.from_pretrained(self.DEFAULT_TOKENIZER_SOURCE, local_files_only=True) + except OSError: + return AutoTokenizer.from_pretrained(self.DEFAULT_TOKENIZER_SOURCE) + + def _load_from_sdnq( + self, + config: AnyModelConfig, + ) -> AnyModel: + from transformers import Qwen3Config, Qwen3ForCausalLM + + from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor + from invokeai.backend.util.logging import InvokeAILogger + + logger = InvokeAILogger.get_logger(self.__class__.__name__) + + if not isinstance(config, (Qwen3Encoder_SDNQ_Config, Qwen3Encoder_SDNQ_Folder_Config)): + raise TypeError( + f"Expected Qwen3Encoder_SDNQ_Config or Qwen3Encoder_SDNQ_Folder_Config, got {type(config).__name__}." + ) + model_path = Path(config.path) + + # Determine safe dtype based on target device capabilities + target_device = TorchDevice.choose_torch_device() + compute_dtype = TorchDevice.choose_bfloat16_safe_dtype(target_device) + + # Load the SDNQ state dict - this returns SDNQTensor wrappers (on CPU) + sd = sdnq_sd_loader(model_path, compute_dtype=compute_dtype) + + # Determine Qwen model configuration from state dict + layer_count = 0 + for key in sd.keys(): + if isinstance(key, str) and key.startswith("model.layers."): + parts = key.split(".") + if len(parts) > 2: + try: + layer_idx = int(parts[2]) + layer_count = max(layer_count, layer_idx + 1) + except ValueError: + pass + + # Get hidden size from embed_tokens weight shape + embed_weight = sd.get("model.embed_tokens.weight") + if embed_weight is None: + raise ValueError("Could not find model.embed_tokens.weight in state dict") + + embed_shape = embed_weight.shape if hasattr(embed_weight, "shape") else embed_weight.tensor_shape + if len(embed_shape) != 2: + raise ValueError(f"Expected 2D embed_tokens weight tensor, got shape {embed_shape}.") + hidden_size = embed_shape[1] + vocab_size = embed_shape[0] + + # Detect attention configuration from layer 0 weights + q_proj_weight = sd.get("model.layers.0.self_attn.q_proj.weight") + k_proj_weight = sd.get("model.layers.0.self_attn.k_proj.weight") + gate_proj_weight = sd.get("model.layers.0.mlp.gate_proj.weight") + + if q_proj_weight is None or k_proj_weight is None or gate_proj_weight is None: + raise ValueError("Could not find attention/mlp weights in state dict to determine configuration") + + q_shape = q_proj_weight.shape if hasattr(q_proj_weight, "shape") else q_proj_weight.tensor_shape + k_shape = k_proj_weight.shape if hasattr(k_proj_weight, "shape") else k_proj_weight.tensor_shape + gate_shape = gate_proj_weight.shape if hasattr(gate_proj_weight, "shape") else gate_proj_weight.tensor_shape + + head_dim = 128 # Standard head dimension for Qwen3 models + num_attention_heads = q_shape[0] // head_dim + num_kv_heads = k_shape[0] // head_dim + intermediate_size = gate_shape[0] + + logger.info( + f"Qwen3 SDNQ Encoder config detected: layers={layer_count}, hidden={hidden_size}, " + f"heads={num_attention_heads}, kv_heads={num_kv_heads}, intermediate={intermediate_size}, " + f"head_dim={head_dim}" + ) + + # Create Qwen3 config + qwen_config = Qwen3Config( + vocab_size=vocab_size, + hidden_size=hidden_size, + intermediate_size=intermediate_size, + num_hidden_layers=layer_count, + num_attention_heads=num_attention_heads, + num_key_value_heads=num_kv_heads, + head_dim=head_dim, + max_position_embeddings=40960, + rms_norm_eps=1e-6, + tie_word_embeddings=True, + rope_theta=1000000.0, + use_sliding_window=False, + attention_bias=False, + attention_dropout=0.0, + torch_dtype=compute_dtype, + ) + + # Use Qwen3ForCausalLM with empty weights, then load SDNQ tensors + with accelerate.init_empty_weights(): + model = Qwen3ForCausalLM(qwen_config) + + # Load the SDNQ weights with assign=True + model.load_state_dict(sd, strict=False, assign=True) + + # Dequantize embed_tokens weight - embedding lookups require indexed access + embed_tokens_weight = model.model.embed_tokens.weight + if isinstance(embed_tokens_weight, SDNQTensor): + dequantized = embed_tokens_weight.get_dequantized_tensor() + model.model.embed_tokens.weight = torch.nn.Parameter(dequantized, requires_grad=False) + logger.info("Dequantized embed_tokens weight for embedding lookups") + + # Handle tied weights + if qwen_config.tie_word_embeddings: + if model.lm_head.weight.is_meta: + model.lm_head.weight = model.model.embed_tokens.weight + logger.info("Tied lm_head.weight to embed_tokens.weight") + else: + model.tie_weights() + + # Re-initialize any remaining meta tensor buffers + for name, buffer in list(model.named_buffers()): + if buffer.is_meta: + parts = name.rsplit(".", 1) + if len(parts) == 2: + parent = model.get_submodule(parts[0]) + buffer_name = parts[1] + else: + parent = model + buffer_name = name + + if buffer_name == "inv_freq": + base = qwen_config.rope_theta + inv_freq = 1.0 / (base ** (torch.arange(0, head_dim, 2, dtype=torch.float32) / head_dim)) + parent.register_buffer(buffer_name, inv_freq.to(dtype=compute_dtype), persistent=False) + else: + logger.warning(f"Re-initializing unknown meta buffer: {name}") + + # Final check: ensure no meta tensors remain in parameters + meta_params = [(name, p) for name, p in model.named_parameters() if p.is_meta] + if meta_params: + meta_names = [name for name, _ in meta_params] + raise RuntimeError( + f"Failed to load all parameters from SDNQ. The following remain as meta tensors: {meta_names}." + ) + + return model diff --git a/invokeai/backend/model_manager/model_on_disk.py b/invokeai/backend/model_manager/model_on_disk.py index 284c4998589..1b71c947228 100644 --- a/invokeai/backend/model_manager/model_on_disk.py +++ b/invokeai/backend/model_manager/model_on_disk.py @@ -10,6 +10,7 @@ from invokeai.backend.model_hash.model_hash import HASHING_ALGORITHMS, ModelHash from invokeai.backend.model_manager.taxonomy import ModelRepoVariant from invokeai.backend.quantization.gguf.loaders import gguf_sd_loader +from invokeai.backend.quantization.sdnq.loaders import sdnq_sd_loader from invokeai.backend.util.logging import InvokeAILogger from invokeai.backend.util.silence_warnings import SilenceWarnings @@ -18,6 +19,21 @@ logger = InvokeAILogger.get_logger() +def _is_sdnq_safetensors(path: Path) -> bool: + """Check if a safetensors file contains SDNQ-quantized weights by checking for weight+scale pairs.""" + try: + with safe_open(path, framework="pt", device="cpu") as f: + keys = set(f.keys()) + for key in keys: + if key.endswith(".weight"): + base = key[:-7] + if f"{base}.scale" in keys: + return True + except Exception: + pass + return False + + class ModelOnDisk: """A utility class representing a model stored on disk.""" @@ -113,7 +129,10 @@ def load_state_dict(self, path: Optional[Path] = None) -> StateDict: elif path.suffix.endswith(".gguf"): checkpoint = gguf_sd_loader(path, compute_dtype=torch.float32) elif path.suffix.endswith(".safetensors"): - checkpoint = safetensors.torch.load_file(path) + if _is_sdnq_safetensors(path): + checkpoint = sdnq_sd_loader(path, compute_dtype=torch.float32) + else: + checkpoint = safetensors.torch.load_file(path) else: raise ValueError(f"Unrecognized model extension: {path.suffix}") diff --git a/invokeai/backend/model_manager/starter_models.py b/invokeai/backend/model_manager/starter_models.py index 2ab3b2767ee..403861b97b0 100644 --- a/invokeai/backend/model_manager/starter_models.py +++ b/invokeai/backend/model_manager/starter_models.py @@ -161,6 +161,15 @@ class StarterModelBundle(BaseModel): type=ModelType.Main, dependencies=[t5_base_encoder, flux_vae, clip_l_encoder], ) +flux_schnell_sdnq = StarterModel( + name="FLUX.1 schnell (SDNQ uint4 + SVD)", + base=BaseModelType.Flux, + source="Disty0/FLUX.1-schnell-SDNQ-uint4-svd-r32", + description="FLUX.1 schnell quantized via SDNQ to uint4 + SVD rank 32. Full self-contained " + "Flux pipeline (transformer + T5 + CLIP + VAE). ~15GB", + type=ModelType.Main, + format=ModelFormat.SDNQQuantized, +) flux_kontext = StarterModel( name="FLUX.1 Kontext dev", base=BaseModelType.Flux, @@ -981,6 +990,26 @@ class StarterModelBundle(BaseModel): dependencies=[flux2_vae, flux2_klein_qwen3_8b_encoder], ) +flux2_klein_4b_sdnq = StarterModel( + name="FLUX.2 Klein 4B (SDNQ dynamic 4-bit)", + base=BaseModelType.Flux2, + source="Disty0/FLUX.2-klein-4B-SDNQ-4bit-dynamic", + description="FLUX.2 Klein 4B quantized via SDNQ to dynamic uint4/int5 mixed precision. " + "Full self-contained Flux2KleinPipeline (transformer + Qwen3 4B + AutoencoderKLFlux2). ~5GB", + type=ModelType.Main, + format=ModelFormat.SDNQQuantized, +) + +flux2_klein_9b_sdnq = StarterModel( + name="FLUX.2 Klein 9B (SDNQ dynamic 4-bit + SVD)", + base=BaseModelType.Flux2, + source="Disty0/FLUX.2-klein-9B-SDNQ-4bit-dynamic-svd-r32", + description="FLUX.2 Klein 9B quantized via SDNQ to dynamic uint4/int5 + SVD rank 32. " + "Full self-contained Flux2KleinPipeline. ~13GB", + type=ModelType.Main, + format=ModelFormat.SDNQQuantized, +) + flux2_klein_4b_gguf_q4 = StarterModel( name="FLUX.2 Klein 4B (GGUF Q4)", base=BaseModelType.Flux2, @@ -1068,6 +1097,16 @@ class StarterModelBundle(BaseModel): dependencies=[z_image_qwen3_encoder_quantized, flux_vae], ) +z_image_turbo_sdnq = StarterModel( + name="Z-Image Turbo (SDNQ uint4 + SVD)", + base=BaseModelType.ZImage, + source="Disty0/Z-Image-Turbo-SDNQ-uint4-svd-r32", + description="Z-Image Turbo quantized via SDNQ to uint4 + SVD rank 32. Full self-contained " + "ZImagePipeline (transformer + Qwen3 + VAE). ~5GB", + type=ModelType.Main, + format=ModelFormat.SDNQQuantized, +) + z_image_controlnet_union = StarterModel( name="Z-Image ControlNet Union", base=BaseModelType.ZImage, @@ -1582,6 +1621,7 @@ def _gemini_3_resolution_presets( flux_dev_quantized, flux_schnell, flux_dev, + flux_schnell_sdnq, sd35_medium, sd35_large, cyberrealistic_sd1, @@ -1657,6 +1697,8 @@ def _gemini_3_resolution_presets( flux2_klein_4b_fp8, flux2_klein_9b, flux2_klein_9b_fp8, + flux2_klein_4b_sdnq, + flux2_klein_9b_sdnq, flux2_klein_4b_gguf_q4, flux2_klein_4b_gguf_q8, flux2_klein_9b_gguf_q4, @@ -1686,6 +1728,7 @@ def _gemini_3_resolution_presets( z_image_turbo, z_image_turbo_quantized, z_image_turbo_q8, + z_image_turbo_sdnq, z_image_qwen3_encoder, z_image_qwen3_encoder_quantized, z_image_controlnet_union, diff --git a/invokeai/backend/model_manager/taxonomy.py b/invokeai/backend/model_manager/taxonomy.py index a2e4e58bdc4..9c711ae22ea 100644 --- a/invokeai/backend/model_manager/taxonomy.py +++ b/invokeai/backend/model_manager/taxonomy.py @@ -197,6 +197,7 @@ class ModelFormat(str, Enum): BnbQuantizednf4b = "bnb_quantized_nf4b" GGUFQuantized = "gguf_quantized" ExternalApi = "external_api" + SDNQQuantized = "sdnq_quantized" Unknown = "unknown" diff --git a/invokeai/backend/quantization/sdnq/__init__.py b/invokeai/backend/quantization/sdnq/__init__.py new file mode 100644 index 00000000000..e200218722f --- /dev/null +++ b/invokeai/backend/quantization/sdnq/__init__.py @@ -0,0 +1,17 @@ +"""SDNQ (SD.Next Quantization) support for InvokeAI. + +This module provides support for loading SDNQ quantized models with on-the-fly +CPU dequantization, similar to GGUF support. +""" + +from invokeai.backend.quantization.sdnq.loaders import has_sdnq_keys, has_sdnq_tensors, sdnq_sd_loader +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor +from invokeai.backend.quantization.sdnq.utils import SDNQQuantizationType + +__all__ = [ + "SDNQTensor", + "sdnq_sd_loader", + "has_sdnq_tensors", + "has_sdnq_keys", + "SDNQQuantizationType", +] diff --git a/invokeai/backend/quantization/sdnq/loaders.py b/invokeai/backend/quantization/sdnq/loaders.py new file mode 100644 index 00000000000..4ae4d1891b8 --- /dev/null +++ b/invokeai/backend/quantization/sdnq/loaders.py @@ -0,0 +1,363 @@ +"""SDNQ State Dict Loader - loads SDNQ quantized safetensors files.""" + +import gc +import json +import logging +from pathlib import Path +from typing import Any, Union + +import torch +from safetensors.torch import load_file + +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor +from invokeai.backend.quantization.sdnq.utils import SDNQQuantizationType + +logger = logging.getLogger(__name__) + + +def _parse_quantization_config(config_path: Path) -> dict[str, Any]: + """Parse quantization_config.json for SDNQ parameters.""" + if not config_path.exists(): + return {} + + with open(config_path, "r", encoding="utf-8") as f: + return json.load(f) + + +_DTYPE_NAME_TO_QUANT_TYPE = { + "uint4": SDNQQuantizationType.UINT4_ASYM, + "int4": SDNQQuantizationType.UINT4_ASYM, # signed naming, same packed storage + "uint5": SDNQQuantizationType.INT5_ASYM, + "int5": SDNQQuantizationType.INT5_ASYM, # SDNQ dynamic-mixed uses this label + "int8": SDNQQuantizationType.INT8_SYM, + "uint8": SDNQQuantizationType.UINT8_SYM, +} + + +def _infer_quantization_type( + weight_dtype: torch.dtype, + has_zero_point: bool, + config: dict[str, Any], + weight: torch.Tensor | None = None, + scale: torch.Tensor | None = None, + tensor_name: str | None = None, + per_tensor_dtype: str | None = None, +) -> SDNQQuantizationType: + """Infer quantization type from weight dtype and config. + + Dynamic-mixed-precision SDNQ models use ``modules_dtype_dict`` to record per-tensor dtypes + (e.g. some layers uint4, others int5). ``per_tensor_dtype`` is the resolved label for this + tensor; when provided it takes precedence over the global ``weights_dtype``. + """ + # Per-tensor override (dynamic mixed precision) + if per_tensor_dtype: + mapped = _DTYPE_NAME_TO_QUANT_TYPE.get(per_tensor_dtype) + if mapped is not None: + # Asymmetric vs symmetric: int8/uint8 follow has_zero_point; everything else is asym. + if mapped == SDNQQuantizationType.INT8_SYM and has_zero_point: + return SDNQQuantizationType.INT8_ASYM + if mapped == SDNQQuantizationType.UINT8_SYM and has_zero_point: + return SDNQQuantizationType.UINT8_ASYM + return mapped + + # Check config for weights_dtype (SDNQ style; static-quantization fallback) + weights_dtype = config.get("weights_dtype", "") + if weights_dtype == "uint4": + return SDNQQuantizationType.UINT4_ASYM + if weights_dtype == "uint5" or weights_dtype == "int5": + return SDNQQuantizationType.INT5_ASYM + + # Check config for quant_type + if "quant_type" in config: + return SDNQQuantizationType(config["quant_type"]) + + # Try to detect uint4 from shapes: + # uint4 is packed as 2 values per uint8, so if scale suggests a shape that's + # 2x the weight's last dimension, it's likely uint4 + if weight is not None and scale is not None and weight_dtype == torch.uint8: + if len(weight.shape) == 2 and len(scale.shape) >= 2: + # For per-group quantization: scale shape [out, num_groups, 1] or [out, num_groups] + # weight shape [out, in/2] for uint4 + # Check if scale's out_features matches weight's out_features + # and if num_groups * group_size would be 2x weight's in_features + out_features = weight.shape[0] + packed_in = weight.shape[1] + scale_out = scale.shape[0] + num_groups = scale.shape[1] + + if scale_out == out_features and num_groups > 0: + # If we can infer that unpacked shape would be 2x packed, it's uint4 + # For typical group sizes (32, 64, 128), num_groups * group_size = in_features + # If packed_in * 2 can be divided by num_groups, it's likely uint4 + unpacked_in = packed_in * 2 + if unpacked_in % num_groups == 0: + inferred_group_size = unpacked_in // num_groups + if inferred_group_size in (32, 64, 128, 256): + return SDNQQuantizationType.UINT4_ASYM + + # Infer from dtype + if weight_dtype == torch.int8: + return SDNQQuantizationType.INT8_ASYM if has_zero_point else SDNQQuantizationType.INT8_SYM + elif weight_dtype == torch.uint8: + return SDNQQuantizationType.UINT8_ASYM if has_zero_point else SDNQQuantizationType.UINT8_SYM + elif weight_dtype in (torch.float8_e4m3fn, torch.float8_e4m3fnuz): + return SDNQQuantizationType.FP8_E4M3 + elif weight_dtype in (torch.float8_e5m2, torch.float8_e5m2fnuz): + return SDNQQuantizationType.FP8_E5M2 + + # Default to symmetric int8 + return SDNQQuantizationType.INT8_SYM + + +def _get_original_shape( + weight: torch.Tensor, + config: dict[str, Any], + tensor_name: str, + quant_type: SDNQQuantizationType, + scale: torch.Tensor | None = None, + group_size: int = 128, +) -> torch.Size: + """Determine the original tensor shape before quantization.""" + # Check if shape is stored in config + if "shapes" in config and tensor_name in config["shapes"]: + return torch.Size(config["shapes"][tensor_name]) + + # For uint4 packed weights with per-group quantization. The packed size is the + # authoritative source for in_features: scale's num_groups × the passed-in group_size can + # be wrong if group_size was not actually known (e.g. Klein 4B uses group_size=64 for some + # to_out layers while others use 128). + if quant_type == SDNQQuantizationType.UINT4_ASYM: + if scale is not None and scale.shape and scale.shape[0] > 0: + out_features = scale.shape[0] + # uint4 packs 2 values per byte; total unpacked = numel * 2 regardless of 1D/2D layout. + total_elements = weight.numel() * 2 + if total_elements % out_features == 0: + in_features = total_elements // out_features + return torch.Size([out_features, in_features]) + + # Fallback for 2D packed weights without usable scale info. + if len(weight.shape) == 2: + return torch.Size([weight.shape[0], weight.shape[1] * 2]) + + # Final fallback for 1D. + if len(weight.shape) == 1: + return torch.Size([weight.numel() * 2]) + + # For uint5 packed weights with per-group quantization. SDNQ stores uint5 as 2D with last + # dim = 5 (8 unpacked values per 5 bytes). Same authoritative-packed-size logic. + if quant_type == SDNQQuantizationType.INT5_ASYM: + total_elements = weight.numel() * 8 // 5 + if scale is not None and scale.shape and scale.shape[0] > 0: + out_features = scale.shape[0] + if total_elements % out_features == 0: + return torch.Size([out_features, total_elements // out_features]) + # Fallback when there's no scale: trust the 2D packed layout. + if len(weight.shape) == 2 and weight.shape[-1] == 5: + return torch.Size([total_elements]) + + # Quantized tensors usually keep the same shape + return weight.shape + + +def sdnq_sd_loader( + model_path: Path, + compute_dtype: torch.dtype = torch.bfloat16, +) -> dict[str, Union[SDNQTensor, torch.Tensor]]: + """Load SDNQ quantized safetensors files. + + SDNQ stores quantized weights with associated scale, zero_point (optional), + and SVD correction matrices (optional). This loader creates SDNQTensor + wrappers that provide on-the-fly dequantization. + + Args: + model_path: Path to safetensors file or directory containing model files. + compute_dtype: Dtype for dequantized computation (default: bfloat16). + + Returns: + State dict with SDNQTensor wrappers for quantized weights and + regular tensors for non-quantized weights. + """ + # Determine which safetensors file(s) hold the weights. For larger models (FLUX.2 Klein 9B, + # FLUX.2 dev, ...) the transformer is sharded across multiple ``*-NNNNN-of-MMMMM.safetensors`` + # files; we merge all of them into one state_dict before grouping. + if model_path.is_dir(): + safetensors_files = sorted(model_path.glob("*.safetensors")) + if not safetensors_files: + raise ValueError(f"No safetensors files found in {model_path}") + config_path = model_path / "quantization_config.json" + else: + safetensors_files = [model_path] + config_path = model_path.parent / "quantization_config.json" + + # Load quantization config if available + quant_config = _parse_quantization_config(config_path) + + # Get group_size from config (default: 128 for SDNQ) + # Note: group_size=0 in config means per-tensor quantization or it needs to be inferred + config_group_size = quant_config.get("group_size", 128) + + # Build a reverse map for dynamic-mixed-precision models. SDNQ stores + # ``modules_dtype_dict`` as ``{dtype_name: [list of layer keys]}``; we flip it to + # ``{layer_key: dtype_name}`` for O(1) lookup during the per-tensor type inference. + per_tensor_dtype_map: dict[str, str] = {} + modules_dtype_dict = quant_config.get("modules_dtype_dict") or {} + if isinstance(modules_dtype_dict, dict): + for dtype_name, layer_keys in modules_dtype_dict.items(): + if isinstance(layer_keys, list): + for layer_key in layer_keys: + per_tensor_dtype_map[layer_key] = dtype_name + + # Load and merge all safetensors shards. + raw_sd: dict[str, torch.Tensor] = {} + for shard in safetensors_files: + shard_sd = load_file(shard) + # Detect accidental key collisions between shards — would indicate a corrupted bundle. + overlap = set(shard_sd).intersection(raw_sd) + if overlap: + raise ValueError(f"Duplicate keys across SDNQ shards (model={model_path}): {sorted(overlap)[:3]}") + raw_sd.update(shard_sd) + + # Group related tensors (weight, scale, zero_point, svd_up, svd_down) + sd: dict[str, Union[SDNQTensor, torch.Tensor]] = {} + processed_keys: set[str] = set() + + for key in raw_sd.keys(): + if key in processed_keys: + continue + + # Check if this is a base weight tensor + if key.endswith(".weight"): + base_key = key[:-7] # Remove ".weight" + + weight = raw_sd[key] + scale_key = f"{base_key}.scale" + zero_point_key = f"{base_key}.zero_point" + svd_up_key = f"{base_key}.svd_up" + svd_down_key = f"{base_key}.svd_down" + + # Check if this is a quantized tensor (has scale) + if scale_key in raw_sd: + scale = raw_sd[scale_key] + zero_point = raw_sd.get(zero_point_key) + svd_up = raw_sd.get(svd_up_key) + svd_down = raw_sd.get(svd_down_key) + + # Determine quantization type. For dynamic-mixed-precision models, look up the + # per-tensor dtype in the reverse map we built from ``modules_dtype_dict``. + quant_type = _infer_quantization_type( + weight.dtype, + zero_point is not None, + quant_config, + weight=weight, + scale=scale, + tensor_name=key, + per_tensor_dtype=per_tensor_dtype_map.get(key), + ) + + # Determine group_size for this tensor + # If config has group_size=0, infer from scale tensor shape + if config_group_size > 0: + tensor_group_size = config_group_size + elif len(scale.shape) >= 2 and scale.shape[1] > 0: + # Scale shape is [out_features, num_groups, ...] or [out_features, num_groups] + # We'll compute group_size after determining original_shape + tensor_group_size = None # Will be computed below + else: + tensor_group_size = 128 # Default fallback + + # Determine original shape (need quant_type and scale to handle uint4 packing) + original_shape = _get_original_shape( + weight, quant_config, key, quant_type, scale=scale, group_size=tensor_group_size or 128 + ) + + # Compute group_size from original_shape and scale if not set + if tensor_group_size is None and len(original_shape) == 2 and len(scale.shape) >= 2: + out_features, in_features = original_shape + num_groups = scale.shape[1] + if num_groups > 0: + tensor_group_size = in_features // num_groups + else: + tensor_group_size = 128 # Fallback + + # Log quantization info for debugging + logger.debug( + f"SDNQ: {key} - type={quant_type.value}, weight_shape={weight.shape}, " + f"weight_dtype={weight.dtype}, scale_shape={scale.shape}, " + f"original_shape={original_shape}, group_size={tensor_group_size}" + ) + + # Create SDNQTensor + sd[key] = SDNQTensor( + data=weight, + quantization_type=quant_type, + tensor_shape=original_shape, + compute_dtype=compute_dtype, + scale=scale, + zero_point=zero_point, + svd_up=svd_up, + svd_down=svd_down, + group_size=tensor_group_size + if quant_type in (SDNQQuantizationType.UINT4_ASYM, SDNQQuantizationType.INT5_ASYM) + else None, + ) + + # Mark related keys as processed + processed_keys.add(key) + processed_keys.add(scale_key) + if zero_point is not None: + processed_keys.add(zero_point_key) + if svd_up is not None: + processed_keys.add(svd_up_key) + if svd_down is not None: + processed_keys.add(svd_down_key) + else: + # Regular tensor, not quantized + sd[key] = weight + processed_keys.add(key) + + elif key.endswith((".scale", ".zero_point", ".svd_up", ".svd_down")): + # Skip - these are handled with their parent weight tensor + continue + + else: + # Other tensors (biases, norms, etc.) - copy as-is + sd[key] = raw_sd[key] + processed_keys.add(key) + + # Log summary + sdnq_count = sum(1 for v in sd.values() if isinstance(v, SDNQTensor)) + regular_count = len(sd) - sdnq_count + source_desc = ( + str(safetensors_files[0]) if len(safetensors_files) == 1 else f"{len(safetensors_files)} shards in {model_path}" + ) + print(f"[SDNQ] Loaded {sdnq_count} quantized tensors, {regular_count} regular tensors from {source_desc}") + logger.info(f"SDNQ loader: {sdnq_count} quantized tensors, {regular_count} regular tensors from {source_desc}") + + gc.collect() + return sd + + +def has_sdnq_tensors(state_dict: dict[str, Any]) -> bool: + """Check if state dict contains SDNQTensor instances.""" + return any(isinstance(v, SDNQTensor) for v in state_dict.values()) + + +def has_sdnq_keys(state_dict: dict[str, Any]) -> bool: + """Check if state dict has SDNQ-style keys (weight + scale pairs). + + SDNQ quantized models store weights with associated scale tensors. + This function detects this pattern to identify SDNQ models. + + Args: + state_dict: State dict to check. + + Returns: + True if state dict has SDNQ-style key patterns. + """ + keys = {k for k in state_dict.keys() if isinstance(k, str)} + for key in keys: + if key.endswith(".weight"): + base = key[:-7] + if f"{base}.scale" in keys: + return True + return False diff --git a/invokeai/backend/quantization/sdnq/sdnq_tensor.py b/invokeai/backend/quantization/sdnq/sdnq_tensor.py new file mode 100644 index 00000000000..9f345376d5f --- /dev/null +++ b/invokeai/backend/quantization/sdnq/sdnq_tensor.py @@ -0,0 +1,379 @@ +"""SDNQTensor - A torch.Tensor subclass for SDNQ quantized weights with on-the-fly dequantization.""" + +import logging +from typing import Optional, overload + +import torch + +from invokeai.backend.quantization.sdnq.utils import ( + SDNQQuantizationType, + apply_svd_correction, + dequantize_asymmetric, + dequantize_int5_per_group, + dequantize_symmetric, + dequantize_uint4_per_group, +) + +logger = logging.getLogger(__name__) + + +def dequantize_and_run(func, args, kwargs): + """Helper function for running math ops on SDNQTensor inputs. + + Dequantizes the inputs and runs the function. + Also casts other floating point tensors to match the compute_dtype of SDNQTensors. + """ + compute_dtype = None + target_device = None + + for a in args: + if hasattr(a, "compute_dtype"): + compute_dtype = a.compute_dtype + if isinstance(a, torch.Tensor) and target_device is None: + target_device = a.device + if compute_dtype is not None and target_device is not None: + break + + if compute_dtype is None or target_device is None: + for v in kwargs.values(): + if hasattr(v, "compute_dtype") and compute_dtype is None: + compute_dtype = v.compute_dtype + if isinstance(v, torch.Tensor) and target_device is None: + target_device = v.device + if compute_dtype is not None and target_device is not None: + break + + def process_tensor(t): + if hasattr(t, "get_dequantized_tensor"): + result = t.get_dequantized_tensor() + if target_device is not None and result.device != target_device: + result = result.to(target_device) + return result + elif isinstance(t, torch.Tensor) and compute_dtype is not None and t.is_floating_point(): + return t.to(compute_dtype) + return t + + dequantized_args = [process_tensor(a) for a in args] + dequantized_kwargs = {k: process_tensor(v) for k, v in kwargs.items()} + return func(*dequantized_args, **dequantized_kwargs) + + +def apply_to_quantized_tensor(func, args, kwargs): + """Apply function to quantized tensor and re-wrap result in SDNQTensor. + + Assumes that the first argument is an SDNQTensor. + """ + sdnq_tensor = args[0] + assert isinstance(sdnq_tensor, SDNQTensor) + assert all(not isinstance(a, SDNQTensor) for a in args[1:]) + assert all(not isinstance(v, SDNQTensor) for v in kwargs.values()) + + new_data = func(sdnq_tensor.quantized_data, *args[1:], **kwargs) + + if new_data.dtype != sdnq_tensor.quantized_data.dtype: + raise ValueError("Operation changed the dtype of SDNQTensor unexpectedly.") + + return SDNQTensor( + data=new_data, + quantization_type=sdnq_tensor._quantization_type, + tensor_shape=sdnq_tensor.tensor_shape, + compute_dtype=sdnq_tensor.compute_dtype, + scale=sdnq_tensor._scale, + zero_point=sdnq_tensor._zero_point, + svd_up=sdnq_tensor._svd_up, + svd_down=sdnq_tensor._svd_down, + group_size=sdnq_tensor._group_size, + ) + + +SDNQ_TENSOR_OP_TABLE = { + # Ops to run on the quantized tensor (keep quantized). + torch.ops.aten.detach.default: apply_to_quantized_tensor, # pyright: ignore + torch.ops.aten._to_copy.default: apply_to_quantized_tensor, # pyright: ignore + torch.ops.aten.clone.default: apply_to_quantized_tensor, # pyright: ignore + # Ops to run on dequantized tensors. + torch.ops.aten.t.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.linear.default: dequantize_and_run, # pyright: ignore - needed for F.linear + torch.ops.aten.addmm.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.mm.default: dequantize_and_run, # pyright: ignore - matrix multiply + torch.ops.aten.bmm.default: dequantize_and_run, # pyright: ignore - batch matrix multiply + torch.ops.aten.baddbmm.default: dequantize_and_run, # pyright: ignore - batch add batch mm + torch.ops.aten.matmul.default: dequantize_and_run, # pyright: ignore - general matmul + # Element-wise ops + torch.ops.aten.mul.Tensor: dequantize_and_run, # pyright: ignore + torch.ops.aten.mul.Scalar: dequantize_and_run, # pyright: ignore + torch.ops.aten.add.Tensor: dequantize_and_run, # pyright: ignore + torch.ops.aten.add.Scalar: dequantize_and_run, # pyright: ignore + torch.ops.aten.sub.Tensor: dequantize_and_run, # pyright: ignore + torch.ops.aten.sub.Scalar: dequantize_and_run, # pyright: ignore + torch.ops.aten.div.Tensor: dequantize_and_run, # pyright: ignore + torch.ops.aten.div.Scalar: dequantize_and_run, # pyright: ignore + torch.ops.aten.neg.default: dequantize_and_run, # pyright: ignore + # Shape manipulation ops + torch.ops.aten.slice.Tensor: dequantize_and_run, # pyright: ignore + torch.ops.aten.view.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.reshape.default: dequantize_and_run, # pyright: ignore + torch.ops.aten._unsafe_view.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.expand.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.permute.default: dequantize_and_run, # pyright: ignore - attention reshaping + torch.ops.aten.transpose.int: dequantize_and_run, # pyright: ignore - attention + torch.ops.aten.contiguous.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.squeeze.dim: dequantize_and_run, # pyright: ignore + torch.ops.aten.squeeze.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.unsqueeze.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.select.int: dequantize_and_run, # pyright: ignore + torch.ops.aten.split.Tensor: dequantize_and_run, # pyright: ignore + torch.ops.aten.split_with_sizes.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.chunk.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.cat.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.stack.default: dequantize_and_run, # pyright: ignore + # Attention ops + torch.ops.aten.scaled_dot_product_attention.default: dequantize_and_run, # pyright: ignore - SDPA + torch.ops.aten._scaled_dot_product_flash_attention.default: dequantize_and_run, # pyright: ignore + torch.ops.aten._scaled_dot_product_efficient_attention.default: dequantize_and_run, # pyright: ignore + # Normalization and activation ops + torch.ops.aten.layer_norm.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.group_norm.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.native_layer_norm.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.native_group_norm.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.silu.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.gelu.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.relu.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.softmax.int: dequantize_and_run, # pyright: ignore + torch.ops.aten._softmax.default: dequantize_and_run, # pyright: ignore + # Reduction ops + torch.ops.aten.mean.dim: dequantize_and_run, # pyright: ignore + torch.ops.aten.sum.dim_IntList: dequantize_and_run, # pyright: ignore + torch.ops.aten.var.correction: dequantize_and_run, # pyright: ignore - for RMSNorm + torch.ops.aten.std.correction: dequantize_and_run, # pyright: ignore + torch.ops.aten.pow.Tensor_Scalar: dequantize_and_run, # pyright: ignore - for RMSNorm + torch.ops.aten.rsqrt.default: dequantize_and_run, # pyright: ignore - for RMSNorm + torch.ops.aten.sqrt.default: dequantize_and_run, # pyright: ignore + # Misc ops + torch.ops.aten.allclose.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.index_put_.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.embedding.default: dequantize_and_run, # pyright: ignore + torch.ops.aten.copy_.default: dequantize_and_run, # pyright: ignore + # Conv ops - needed for VAE + torch.ops.aten.convolution.default: dequantize_and_run, # pyright: ignore - Conv2d uses this + torch.ops.aten.conv2d.default: dequantize_and_run, # pyright: ignore +} + + +class SDNQTensor(torch.Tensor): + """A torch.Tensor subclass holding SDNQ quantized weights. + + Provides on-the-fly dequantization when used in operations. + Supports symmetric/asymmetric quantization and optional SVD correction. + """ + + @staticmethod + def __new__( + cls, + data: torch.Tensor, + quantization_type: SDNQQuantizationType, + tensor_shape: torch.Size, + compute_dtype: torch.dtype, + scale: torch.Tensor, + zero_point: Optional[torch.Tensor] = None, + svd_up: Optional[torch.Tensor] = None, + svd_down: Optional[torch.Tensor] = None, + group_size: Optional[int] = None, + ): + # Use tensor_shape (the dequantized shape) for the wrapper, not data.shape (packed shape) + # This ensures PyTorch's load_state_dict sees the correct shape for assignment + # Compute strides for row-major (C-contiguous) layout + strides = [] + stride = 1 + for dim in reversed(tensor_shape): + strides.append(stride) + stride *= dim + strides = tuple(reversed(strides)) + + return torch.Tensor._make_wrapper_subclass( # pyright: ignore + cls, + tensor_shape, + dtype=data.dtype, + layout=data.layout, + device=data.device, + strides=strides, + storage_offset=0, + ) + + def __init__( + self, + data: torch.Tensor, + quantization_type: SDNQQuantizationType, + tensor_shape: torch.Size, + compute_dtype: torch.dtype, + scale: torch.Tensor, + zero_point: Optional[torch.Tensor] = None, + svd_up: Optional[torch.Tensor] = None, + svd_down: Optional[torch.Tensor] = None, + group_size: Optional[int] = None, + ): + self.quantized_data = data + self._quantization_type = quantization_type + self.tensor_shape = tensor_shape + self.compute_dtype = compute_dtype + self._scale = scale + self._zero_point = zero_point + self._svd_up = svd_up + self._svd_down = svd_down + self._group_size = group_size + + def __repr__(self, *, tensor_contents=None): + return ( + f"SDNQTensor(type={self._quantization_type.value}, " + f"dequantized_shape=({self.tensor_shape}), " + f"has_svd={self._svd_up is not None})" + ) + + @overload + def size(self, dim: None = None) -> torch.Size: ... + + @overload + def size(self, dim: int) -> int: ... + + def size(self, dim: int | None = None): + """Return the size of the tensor after dequantization.""" + if dim is not None: + return self.tensor_shape[dim] + return self.tensor_shape + + @property + def shape(self) -> torch.Size: # pyright: ignore[reportIncompatibleVariableOverride] + """The shape of the tensor after dequantization.""" + return self.size() + + @property + def quantized_shape(self) -> torch.Size: + """The shape of the quantized tensor.""" + return self.quantized_data.shape + + @property + def is_asymmetric(self) -> bool: + """Whether this tensor uses asymmetric quantization.""" + return self._zero_point is not None + + @property + def has_svd(self) -> bool: + """Whether this tensor has SVD correction components.""" + return self._svd_up is not None and self._svd_down is not None + + def requires_grad_(self, mode: bool = True) -> torch.Tensor: + """SDNQTensor is only for inference, not training. This is a no-op.""" + return self + + # Track which tensors we've logged (to avoid spam) + _logged_tensors: set = set() + + def get_dequantized_tensor(self) -> torch.Tensor: + """Return the dequantized tensor. + + Returns: + Dequantized tensor with compute_dtype. + """ + # Debug logging for first few tensors + tensor_id = id(self) + should_log = tensor_id not in SDNQTensor._logged_tensors and len(SDNQTensor._logged_tensors) < 5 + if should_log: + SDNQTensor._logged_tensors.add(tensor_id) + print( + f"[SDNQ] dequantize: type={self._quantization_type.value}, " + f"weight_shape={self.quantized_data.shape}, weight_dtype={self.quantized_data.dtype}, " + f"scale_shape={self._scale.shape}, zp={self._zero_point is not None}, " + f"svd={self.has_svd}, group_size={self._group_size}" + ) + logger.info( + f"SDNQ dequantize: type={self._quantization_type.value}, " + f"weight_shape={self.quantized_data.shape}, weight_dtype={self.quantized_data.dtype}, " + f"scale_shape={self._scale.shape}, scale_range=[{self._scale.min():.6f}, {self._scale.max():.6f}], " + f"zp={self._zero_point is not None}, svd={self.has_svd}, group_size={self._group_size}" + ) + if self._zero_point is not None: + logger.info( + f" zero_point_shape={self._zero_point.shape}, " + f"zp_range=[{self._zero_point.min():.6f}, {self._zero_point.max():.6f}]" + ) + + # Perform dequantization based on quantization type + if self._quantization_type == SDNQQuantizationType.UINT4_ASYM: + # uint4 with per-group quantization + assert self._zero_point is not None + assert self._group_size is not None + dequantized = dequantize_uint4_per_group( + self.quantized_data, + self._scale, + self._zero_point, + self.tensor_shape, + self._group_size, + dtype=self.compute_dtype, + ) + elif self._quantization_type == SDNQQuantizationType.INT5_ASYM: + # Signed 5-bit with per-group quantization (8 values per 5 bytes). zero_point may + # be absent for scale-only 5-bit tensors produced by SDNQ's dynamic-mixed pass. + assert self._group_size is not None + dequantized = dequantize_int5_per_group( + self.quantized_data, + self._scale, + self._zero_point, + self.tensor_shape, + self._group_size, + dtype=self.compute_dtype, + ) + elif self.is_asymmetric: + assert self._zero_point is not None + dequantized = dequantize_asymmetric( + self.quantized_data, + self._scale, + self._zero_point, + dtype=self.compute_dtype, + ) + else: + dequantized = dequantize_symmetric( + self.quantized_data, + self._scale, + dtype=self.compute_dtype, + ) + + # Apply SVD correction if present + if self.has_svd: + dequantized = apply_svd_correction( + dequantized, + self._svd_up, + self._svd_down, + dtype=self.compute_dtype, + ) + + # Reshape to original tensor shape if needed + if dequantized.shape != self.tensor_shape: + dequantized = dequantized.view(self.tensor_shape) + + result = dequantized.to(self.compute_dtype) + + # Log output range for debugging + if should_log: + logger.info(f" -> output_shape={result.shape}, output_range=[{result.min():.6f}, {result.max():.6f}]") + + return result + + # Track unknown ops to avoid spamming warnings + _warned_ops: set = set() + + @classmethod + def __torch_dispatch__(cls, func, types, args, kwargs): + if func in SDNQ_TENSOR_OP_TABLE: + return SDNQ_TENSOR_OP_TABLE[func](func, args, kwargs) + + # Fallback: dequantize and run for unknown operations + # This ensures computation works, but may indicate missing ops + if func not in cls._warned_ops: + cls._warned_ops.add(func) + import logging + + logging.getLogger(__name__).warning( + f"SDNQTensor: unknown op {func}, dequantizing (add to op table for efficiency)" + ) + + return dequantize_and_run(func, args, kwargs) diff --git a/invokeai/backend/quantization/sdnq/utils.py b/invokeai/backend/quantization/sdnq/utils.py new file mode 100644 index 00000000000..a9a8a6aff91 --- /dev/null +++ b/invokeai/backend/quantization/sdnq/utils.py @@ -0,0 +1,322 @@ +"""SDNQ (SD.Next Quantization) utility functions and enums.""" + +from enum import Enum +from typing import Optional + +import torch + + +class SDNQQuantizationType(str, Enum): + """SDNQ Quantization types from SD.Next.""" + + INT8_SYM = "int8_sym" # Symmetric Int8 quantization + INT8_ASYM = "int8_asym" # Asymmetric Int8 quantization + UINT8_SYM = "uint8_sym" # Symmetric UInt8 quantization + UINT8_ASYM = "uint8_asym" # Asymmetric UInt8 quantization + UINT4_ASYM = "uint4_asym" # Asymmetric UInt4 quantization (packed in uint8) + INT5_ASYM = "int5_asym" # Asymmetric signed 5-bit quantization (range -16..15, packed 8/5 bytes) + FP8_E4M3 = "fp8_e4m3" # FP8 E4M3 format + FP8_E5M2 = "fp8_e5m2" # FP8 E5M2 format + + +def unpack_uint4(packed: torch.Tensor, original_shape: torch.Size) -> torch.Tensor: + """Unpack uint4 values from packed uint8 tensor. + + SDNQ stores uint4 values packed as 2 values per uint8 byte: + - Lower 4 bits: first value (packed & 0x0F) + - Upper 4 bits: second value (packed >> 4) + + Args: + packed: Packed uint8 tensor with shape [..., N/2] or flattened 1D. + original_shape: Original tensor shape before packing. + + Returns: + Unpacked tensor with shape original_shape containing uint4 values (0-15). + """ + # If packed is 1D but original_shape is 2D, reshape to packed 2D first + # This ensures correct unpacking order + if packed.dim() == 1 and len(original_shape) == 2: + out_features, in_features = original_shape + packed_in_features = in_features // 2 + packed = packed.view(out_features, packed_in_features) + + # Extract lower and upper 4 bits + lower = torch.bitwise_and(packed, 15) + upper = torch.bitwise_right_shift(packed, 4) + + # Standard packing order: lower nibble first, upper nibble second + # byte = (upper << 4) | lower means lower is at even indices, upper at odd + unpacked = torch.stack((lower, upper), dim=-1).view(original_shape) + return unpacked + + +def unpack_uint5(packed: torch.Tensor, original_shape: torch.Size) -> torch.Tensor: + """Unpack 5-bit values from packed uint8 tensor. + + SDNQ packs 8 5-bit values into 5 bytes (40 bits). The packed tensor has its + last dim equal to 5, where each (..., 5) row encodes 8 unpacked values. + Bit layout matches Disty0/sdnq's ``unpack_uint5`` so the produced order is the + same as the producer's pack order. + + Args: + packed: Packed uint8 tensor with shape (..., 5). + original_shape: Original tensor shape before packing (must total to packed.numel() * 8 // 5). + + Returns: + Unpacked tensor with shape ``original_shape`` containing values 0-31. + """ + if packed.dim() < 2 or packed.shape[-1] != 5: + raise ValueError(f"unpack_uint5 expects packed tensor with last dim = 5; got shape {tuple(packed.shape)}") + result_bitwise_right_shift = torch.bitwise_right_shift(packed[..., :3], 5) + result = torch.cat( + ( + torch.bitwise_and(packed[..., :5], 31), + torch.bitwise_or( + result_bitwise_right_shift[..., :2], + torch.bitwise_and(torch.bitwise_right_shift(packed[..., 3:5], 2), 24), + ), + torch.bitwise_or( + result_bitwise_right_shift[..., 2:3], + torch.bitwise_or( + torch.bitwise_and(torch.bitwise_right_shift(packed[..., 3:4], 3), 16), + torch.bitwise_and(torch.bitwise_right_shift(packed[..., 4:5], 4), 8), + ), + ), + ), + dim=-1, + ).view(original_shape) + return result + + +def dequantize_symmetric( + weight: torch.Tensor, + scale: torch.Tensor, + dtype: torch.dtype = torch.float32, +) -> torch.Tensor: + """Symmetric dequantization: result = weight * scale. + + Args: + weight: Quantized weight tensor (int8, uint8, or fp8). + scale: Scale factor tensor. + dtype: Target dtype for the result. + + Returns: + Dequantized tensor. + """ + # Ensure scale is on the same device as weight + if scale.device != weight.device: + scale = scale.to(weight.device) + + # Handle scale broadcasting for different shapes + # Scale might be: scalar, [1], [out_features], [out_features, 1], etc. + scale = scale.to(dtype) + weight = weight.to(dtype) + + # Reshape scale for broadcasting if needed + if scale.dim() == 1 and weight.dim() == 2: + # Per-channel scale for Linear: [out_features] -> [out_features, 1] + scale = scale.unsqueeze(-1) + elif scale.dim() == 1 and weight.dim() == 4: + # Per-channel scale for Conv2d: [out_channels] -> [out_channels, 1, 1, 1] + scale = scale.view(-1, 1, 1, 1) + + return weight * scale + + +def dequantize_asymmetric( + weight: torch.Tensor, + scale: torch.Tensor, + zero_point: torch.Tensor, + dtype: torch.dtype = torch.float32, +) -> torch.Tensor: + """Asymmetric dequantization: result = weight * scale + zero_point. + + Note: SDNQ uses a different convention where zero_point is a pre-computed bias. + Standard formula: result = (weight - zp) * scale + SDNQ formula: result = weight * scale + zero_point (where zero_point = -zp * scale) + + Args: + weight: Quantized weight tensor. + scale: Scale factor tensor. + zero_point: Zero point/bias tensor for asymmetric quantization. + dtype: Target dtype for the result. + + Returns: + Dequantized tensor. + """ + # Ensure scale and zero_point are on the same device as weight + if scale.device != weight.device: + scale = scale.to(weight.device) + if zero_point.device != weight.device: + zero_point = zero_point.to(weight.device) + + # Convert to compute dtype + scale = scale.to(dtype) + zero_point = zero_point.to(dtype) + weight = weight.to(dtype) + + # Reshape scale and zero_point for broadcasting if needed + if scale.dim() == 1 and weight.dim() == 2: + # Per-channel for Linear: [out_features] -> [out_features, 1] + scale = scale.unsqueeze(-1) + elif scale.dim() == 1 and weight.dim() == 4: + # Per-channel for Conv2d: [out_channels] -> [out_channels, 1, 1, 1] + scale = scale.view(-1, 1, 1, 1) + + if zero_point.dim() == 1 and weight.dim() == 2: + zero_point = zero_point.unsqueeze(-1) + elif zero_point.dim() == 1 and weight.dim() == 4: + zero_point = zero_point.view(-1, 1, 1, 1) + + # SDNQ formula: x = q * scale + zero_point (zero_point is actually a bias) + return weight * scale + zero_point + + +# Track whether we've done diagnostic logging +_uint4_diagnostic_done = False + + +def dequantize_uint4_per_group( + packed_weight: torch.Tensor, + scale: torch.Tensor, + zero_point: torch.Tensor, + original_shape: torch.Size, + group_size: int, + dtype: torch.dtype = torch.float32, +) -> torch.Tensor: + """Dequantize uint4 weights with per-group scaling. + + SDNQ uint4 quantization uses: + - Packed uint8 storage (2 uint4 values per byte) + - Per-group scale factors + - Per-group zero points for asymmetric quantization + + The scale tensor has shape [out_features, num_groups, 1] where + num_groups = in_features / group_size. + + Args: + packed_weight: Packed uint8 tensor with shape [out_features, in_features/2]. + scale: Per-group scale tensor with shape [out_features, num_groups, 1]. + zero_point: Per-group zero point tensor with shape [out_features, num_groups, 1]. + original_shape: Original weight shape [out_features, in_features]. + group_size: Number of elements per quantization group. + dtype: Target dtype for the result. + + Returns: + Dequantized tensor with shape original_shape. + """ + global _uint4_diagnostic_done + + # Ensure scale and zero_point are on the same device as packed_weight + device = packed_weight.device + if scale.device != device: + scale = scale.to(device) + if zero_point.device != device: + zero_point = zero_point.to(device) + + # Unpack uint4 values + unpacked = unpack_uint4(packed_weight, original_shape) + + out_features, in_features = original_shape + num_groups = in_features // group_size + + # Reshape for per-group operations: [out_features, num_groups, group_size] + weight_grouped = unpacked.view(out_features, num_groups, group_size).to(dtype) + + # SDNQ uses pre-computed bias: x = q * scale + zero_point + # where zero_point = -original_zp * scale (already a floating point bias) + scale_f = scale.to(dtype) + zp_f = zero_point.to(dtype) + dequantized = weight_grouped * scale_f + zp_f + + # Diagnostic logging (once) + if not _uint4_diagnostic_done: + _uint4_diagnostic_done = True + print("[SDNQ uint4] Diagnostic:") + print(f" packed_weight: shape={packed_weight.shape}, dtype={packed_weight.dtype}") + print(f" unpacked: min={unpacked.min().item()}, max={unpacked.max().item()}, unique={len(unpacked.unique())}") + print(f" scale: shape={scale.shape}, dtype={scale.dtype}, range=[{scale.min():.6f}, {scale.max():.6f}]") + print( + f" zero_point: shape={zero_point.shape}, dtype={zero_point.dtype}, range=[{zero_point.min():.6f}, {zero_point.max():.6f}]" + ) + print(f" group_size={group_size}, num_groups={num_groups}") + # Sample values + zp_sample = zero_point.flatten()[:10].tolist() + print(f" zero_point sample: {zp_sample}") + sample_dequant = dequantized.flatten()[:10].tolist() + print(f" dequantized sample: {sample_dequant}") + print(f" dequantized range: [{dequantized.min():.6f}, {dequantized.max():.6f}]") + + # Reshape back to original shape + return dequantized.view(original_shape) + + +def dequantize_int5_per_group( + packed_weight: torch.Tensor, + scale: torch.Tensor, + zero_point: Optional[torch.Tensor], + original_shape: torch.Size, + group_size: int, + dtype: torch.dtype = torch.float32, +) -> torch.Tensor: + """Dequantize signed 5-bit weights with per-group scaling. + + SDNQ packs 8 int5 values into 5 bytes. The raw unpacked bytes give 0..31; for signed + int5 storage we shift by ``min = -16`` to get the actual range -16..15 (matches Disty0 + upstream's ``unpack_int`` sign-extension step). + + ``zero_point`` is optional: dynamic-mixed-precision SDNQ can emit scale-only 5-bit + tensors when the per-group loss budget allows it. + """ + device = packed_weight.device + if scale.device != device: + scale = scale.to(device) + if zero_point is not None and zero_point.device != device: + zero_point = zero_point.to(device) + + unpacked = unpack_uint5(packed_weight, original_shape) + # Sign-extend: SDNQ stores signed int5 (-16..15) in unsigned 5-bit (0..31) by adding 16 + # at pack time. We subtract it back here. We cast to int8 first so the subtraction stays + # within an exact integer domain before dtype promotion. + signed = unpacked.to(torch.int8) - 16 + + out_features, in_features = original_shape + num_groups = in_features // group_size + weight_grouped = signed.view(out_features, num_groups, group_size).to(dtype) + scale_f = scale.to(dtype) + dequantized = weight_grouped * scale_f + if zero_point is not None: + dequantized = dequantized + zero_point.to(dtype) + return dequantized.view(original_shape) + + +def apply_svd_correction( + dequantized: torch.Tensor, + svd_up: Optional[torch.Tensor], + svd_down: Optional[torch.Tensor], + dtype: torch.dtype = torch.float32, +) -> torch.Tensor: + """Apply SVD correction: result = dequantized + svd_up @ svd_down. + + SVD (Singular Value Decomposition) correction adds a low-rank approximation + to improve the accuracy of quantized weights. + + Args: + dequantized: Already dequantized tensor. + svd_up: SVD up matrix (U * S component). + svd_down: SVD down matrix (V^T component). + dtype: Target dtype for the result. + + Returns: + Tensor with SVD correction applied. + """ + if svd_up is not None and svd_down is not None: + # Ensure SVD matrices are on the same device as dequantized tensor + device = dequantized.device + if svd_up.device != device: + svd_up = svd_up.to(device) + if svd_down.device != device: + svd_down = svd_down.to(device) + svd_correction = svd_up.to(dtype) @ svd_down.to(dtype) + return dequantized + svd_correction + return dequantized diff --git a/invokeai/frontend/web/openapi.json b/invokeai/frontend/web/openapi.json index e13946511e2..d5d6b582f46 100644 --- a/invokeai/frontend/web/openapi.json +++ b/invokeai/frontend/web/openapi.json @@ -843,6 +843,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -957,6 +966,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -966,6 +978,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -1161,6 +1176,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -1275,6 +1299,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -1284,6 +1311,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -1479,6 +1509,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -1593,6 +1632,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -1602,6 +1644,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -1847,6 +1892,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -1961,6 +2015,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -1970,6 +2027,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -2239,6 +2299,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -2353,6 +2422,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -2362,6 +2434,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -3451,6 +3526,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -3565,6 +3649,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -3574,6 +3661,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -11439,6 +11529,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -11553,6 +11652,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -11562,6 +11664,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -24853,14 +24958,13 @@ } ], "default": null, - "description": "Diffusers Flux2 Klein model to extract VAE and/or Qwen3 encoder from. Use this if you don't have separate VAE/Qwen3 models. Ignored if both VAE and Qwen3 Encoder are provided separately.", + "description": "Diffusers or SDNQ-pipeline Flux2 Klein model to extract VAE and/or Qwen3 encoder from. Use this if you don't have separate VAE/Qwen3 models. Ignored if both VAE and Qwen3 Encoder are provided separately.", "field_kind": "input", "input": "direct", "orig_default": null, "orig_required": false, - "title": "Qwen3 Source (Diffusers)", + "title": "Qwen3 Source", "ui_model_base": ["flux2"], - "ui_model_format": ["diffusers"], "ui_model_type": ["main"] }, "max_seq_len": { @@ -47170,20 +47274,523 @@ "title": "Config Path", "description": "Path to the config for this model, if any." }, - "base": { - "type": "string", - "const": "flux", - "title": "Base", - "default": "flux" - }, + "base": { + "type": "string", + "const": "flux", + "title": "Base", + "default": "flux" + }, + "format": { + "type": "string", + "const": "bnb_quantized_nf4b", + "title": "Format", + "default": "bnb_quantized_nf4b" + }, + "variant": { + "$ref": "#/components/schemas/FluxVariantType" + } + }, + "type": "object", + "required": [ + "key", + "hash", + "path", + "file_size", + "name", + "description", + "source", + "source_type", + "source_api_response", + "source_url", + "cover_image", + "type", + "trigger_phrases", + "default_settings", + "config_path", + "base", + "format", + "variant" + ], + "title": "Main_BnBNF4_FLUX_Config", + "description": "Model config for main checkpoint models." + }, + "Main_Checkpoint_Anima_Config": { + "properties": { + "key": { + "type": "string", + "title": "Key", + "description": "A unique key for this model." + }, + "hash": { + "type": "string", + "title": "Hash", + "description": "The hash of the model file(s)." + }, + "path": { + "type": "string", + "title": "Path", + "description": "Path to the model on the filesystem. Relative paths are relative to the Invoke root directory." + }, + "file_size": { + "type": "integer", + "title": "File Size", + "description": "The size of the model in bytes." + }, + "name": { + "type": "string", + "title": "Name", + "description": "Name of the model." + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "Model description" + }, + "source": { + "type": "string", + "title": "Source", + "description": "The original source of the model (path, URL or repo_id)." + }, + "source_type": { + "$ref": "#/components/schemas/ModelSourceType", + "description": "The type of source" + }, + "source_api_response": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Api Response", + "description": "The original API response from the source, as stringified JSON." + }, + "source_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Url", + "description": "Optional URL for the model (e.g. download page or model page)." + }, + "cover_image": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Cover Image", + "description": "Url for image to preview model" + }, + "type": { + "type": "string", + "const": "main", + "title": "Type", + "default": "main" + }, + "trigger_phrases": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array", + "uniqueItems": true + }, + { + "type": "null" + } + ], + "title": "Trigger Phrases", + "description": "Set of trigger phrases for this model" + }, + "default_settings": { + "anyOf": [ + { + "$ref": "#/components/schemas/MainModelDefaultSettings" + }, + { + "type": "null" + } + ], + "description": "Default settings for this model" + }, + "config_path": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Config Path", + "description": "Path to the config for this model, if any." + }, + "base": { + "type": "string", + "const": "anima", + "title": "Base", + "default": "anima" + }, + "format": { + "type": "string", + "const": "checkpoint", + "title": "Format", + "default": "checkpoint" + } + }, + "type": "object", + "required": [ + "key", + "hash", + "path", + "file_size", + "name", + "description", + "source", + "source_type", + "source_api_response", + "source_url", + "cover_image", + "type", + "trigger_phrases", + "default_settings", + "config_path", + "base", + "format" + ], + "title": "Main_Checkpoint_Anima_Config", + "description": "Model config for Anima single-file checkpoint models (safetensors).\n\nAnima is built on NVIDIA Cosmos Predict2 DiT with a custom LLM Adapter\nthat bridges Qwen3 0.6B text encoder outputs to the DiT." + }, + "Main_Checkpoint_FLUX_Config": { + "properties": { + "key": { + "type": "string", + "title": "Key", + "description": "A unique key for this model." + }, + "hash": { + "type": "string", + "title": "Hash", + "description": "The hash of the model file(s)." + }, + "path": { + "type": "string", + "title": "Path", + "description": "Path to the model on the filesystem. Relative paths are relative to the Invoke root directory." + }, + "file_size": { + "type": "integer", + "title": "File Size", + "description": "The size of the model in bytes." + }, + "name": { + "type": "string", + "title": "Name", + "description": "Name of the model." + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "Model description" + }, + "source": { + "type": "string", + "title": "Source", + "description": "The original source of the model (path, URL or repo_id)." + }, + "source_type": { + "$ref": "#/components/schemas/ModelSourceType", + "description": "The type of source" + }, + "source_api_response": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Api Response", + "description": "The original API response from the source, as stringified JSON." + }, + "source_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Url", + "description": "Optional URL for the model (e.g. download page or model page)." + }, + "cover_image": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Cover Image", + "description": "Url for image to preview model" + }, + "type": { + "type": "string", + "const": "main", + "title": "Type", + "default": "main" + }, + "trigger_phrases": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array", + "uniqueItems": true + }, + { + "type": "null" + } + ], + "title": "Trigger Phrases", + "description": "Set of trigger phrases for this model" + }, + "default_settings": { + "anyOf": [ + { + "$ref": "#/components/schemas/MainModelDefaultSettings" + }, + { + "type": "null" + } + ], + "description": "Default settings for this model" + }, + "config_path": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Config Path", + "description": "Path to the config for this model, if any." + }, + "format": { + "type": "string", + "const": "checkpoint", + "title": "Format", + "default": "checkpoint" + }, + "base": { + "type": "string", + "const": "flux", + "title": "Base", + "default": "flux" + }, + "variant": { + "$ref": "#/components/schemas/FluxVariantType" + } + }, + "type": "object", + "required": [ + "key", + "hash", + "path", + "file_size", + "name", + "description", + "source", + "source_type", + "source_api_response", + "source_url", + "cover_image", + "type", + "trigger_phrases", + "default_settings", + "config_path", + "format", + "base", + "variant" + ], + "title": "Main_Checkpoint_FLUX_Config", + "description": "Model config for main checkpoint models." + }, + "Main_Checkpoint_Flux2_Config": { + "properties": { + "key": { + "type": "string", + "title": "Key", + "description": "A unique key for this model." + }, + "hash": { + "type": "string", + "title": "Hash", + "description": "The hash of the model file(s)." + }, + "path": { + "type": "string", + "title": "Path", + "description": "Path to the model on the filesystem. Relative paths are relative to the Invoke root directory." + }, + "file_size": { + "type": "integer", + "title": "File Size", + "description": "The size of the model in bytes." + }, + "name": { + "type": "string", + "title": "Name", + "description": "Name of the model." + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "Model description" + }, + "source": { + "type": "string", + "title": "Source", + "description": "The original source of the model (path, URL or repo_id)." + }, + "source_type": { + "$ref": "#/components/schemas/ModelSourceType", + "description": "The type of source" + }, + "source_api_response": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Api Response", + "description": "The original API response from the source, as stringified JSON." + }, + "source_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Url", + "description": "Optional URL for the model (e.g. download page or model page)." + }, + "cover_image": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Cover Image", + "description": "Url for image to preview model" + }, + "type": { + "type": "string", + "const": "main", + "title": "Type", + "default": "main" + }, + "trigger_phrases": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array", + "uniqueItems": true + }, + { + "type": "null" + } + ], + "title": "Trigger Phrases", + "description": "Set of trigger phrases for this model" + }, + "default_settings": { + "anyOf": [ + { + "$ref": "#/components/schemas/MainModelDefaultSettings" + }, + { + "type": "null" + } + ], + "description": "Default settings for this model" + }, + "config_path": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Config Path", + "description": "Path to the config for this model, if any." + }, "format": { "type": "string", - "const": "bnb_quantized_nf4b", + "const": "checkpoint", "title": "Format", - "default": "bnb_quantized_nf4b" + "default": "checkpoint" + }, + "base": { + "type": "string", + "const": "flux2", + "title": "Base", + "default": "flux2" }, "variant": { - "$ref": "#/components/schemas/FluxVariantType" + "$ref": "#/components/schemas/Flux2VariantType" } }, "type": "object", @@ -47203,14 +47810,14 @@ "trigger_phrases", "default_settings", "config_path", - "base", "format", + "base", "variant" ], - "title": "Main_BnBNF4_FLUX_Config", - "description": "Model config for main checkpoint models." + "title": "Main_Checkpoint_Flux2_Config", + "description": "Model config for FLUX.2 checkpoint models (e.g. Klein)." }, - "Main_Checkpoint_Anima_Config": { + "Main_Checkpoint_SD1_Config": { "properties": { "key": { "type": "string", @@ -47339,17 +47946,23 @@ "title": "Config Path", "description": "Path to the config for this model, if any." }, - "base": { - "type": "string", - "const": "anima", - "title": "Base", - "default": "anima" - }, "format": { "type": "string", "const": "checkpoint", "title": "Format", "default": "checkpoint" + }, + "prediction_type": { + "$ref": "#/components/schemas/SchedulerPredictionType" + }, + "variant": { + "$ref": "#/components/schemas/ModelVariantType" + }, + "base": { + "type": "string", + "const": "sd-1", + "title": "Base", + "default": "sd-1" } }, "type": "object", @@ -47369,13 +47982,14 @@ "trigger_phrases", "default_settings", "config_path", - "base", - "format" + "format", + "prediction_type", + "variant", + "base" ], - "title": "Main_Checkpoint_Anima_Config", - "description": "Model config for Anima single-file checkpoint models (safetensors).\n\nAnima is built on NVIDIA Cosmos Predict2 DiT with a custom LLM Adapter\nthat bridges Qwen3 0.6B text encoder outputs to the DiT." + "title": "Main_Checkpoint_SD1_Config" }, - "Main_Checkpoint_FLUX_Config": { + "Main_Checkpoint_SD2_Config": { "properties": { "key": { "type": "string", @@ -47510,14 +48124,17 @@ "title": "Format", "default": "checkpoint" }, + "prediction_type": { + "$ref": "#/components/schemas/SchedulerPredictionType" + }, + "variant": { + "$ref": "#/components/schemas/ModelVariantType" + }, "base": { "type": "string", - "const": "flux", + "const": "sd-2", "title": "Base", - "default": "flux" - }, - "variant": { - "$ref": "#/components/schemas/FluxVariantType" + "default": "sd-2" } }, "type": "object", @@ -47538,13 +48155,13 @@ "default_settings", "config_path", "format", - "base", - "variant" + "prediction_type", + "variant", + "base" ], - "title": "Main_Checkpoint_FLUX_Config", - "description": "Model config for main checkpoint models." + "title": "Main_Checkpoint_SD2_Config" }, - "Main_Checkpoint_Flux2_Config": { + "Main_Checkpoint_SDXLRefiner_Config": { "properties": { "key": { "type": "string", @@ -47679,14 +48296,17 @@ "title": "Format", "default": "checkpoint" }, + "prediction_type": { + "$ref": "#/components/schemas/SchedulerPredictionType" + }, + "variant": { + "$ref": "#/components/schemas/ModelVariantType" + }, "base": { "type": "string", - "const": "flux2", + "const": "sdxl-refiner", "title": "Base", - "default": "flux2" - }, - "variant": { - "$ref": "#/components/schemas/Flux2VariantType" + "default": "sdxl-refiner" } }, "type": "object", @@ -47707,13 +48327,13 @@ "default_settings", "config_path", "format", - "base", - "variant" + "prediction_type", + "variant", + "base" ], - "title": "Main_Checkpoint_Flux2_Config", - "description": "Model config for FLUX.2 checkpoint models (e.g. Klein)." + "title": "Main_Checkpoint_SDXLRefiner_Config" }, - "Main_Checkpoint_SD1_Config": { + "Main_Checkpoint_SDXL_Config": { "properties": { "key": { "type": "string", @@ -47856,9 +48476,9 @@ }, "base": { "type": "string", - "const": "sd-1", + "const": "sdxl", "title": "Base", - "default": "sd-1" + "default": "sdxl" } }, "type": "object", @@ -47883,9 +48503,9 @@ "variant", "base" ], - "title": "Main_Checkpoint_SD1_Config" + "title": "Main_Checkpoint_SDXL_Config" }, - "Main_Checkpoint_SD2_Config": { + "Main_Checkpoint_ZImage_Config": { "properties": { "key": { "type": "string", @@ -48014,23 +48634,20 @@ "title": "Config Path", "description": "Path to the config for this model, if any." }, + "base": { + "type": "string", + "const": "z-image", + "title": "Base", + "default": "z-image" + }, "format": { "type": "string", "const": "checkpoint", "title": "Format", "default": "checkpoint" }, - "prediction_type": { - "$ref": "#/components/schemas/SchedulerPredictionType" - }, "variant": { - "$ref": "#/components/schemas/ModelVariantType" - }, - "base": { - "type": "string", - "const": "sd-2", - "title": "Base", - "default": "sd-2" + "$ref": "#/components/schemas/ZImageVariantType" } }, "type": "object", @@ -48050,14 +48667,14 @@ "trigger_phrases", "default_settings", "config_path", + "base", "format", - "prediction_type", - "variant", - "base" + "variant" ], - "title": "Main_Checkpoint_SD2_Config" + "title": "Main_Checkpoint_ZImage_Config", + "description": "Model config for Z-Image single-file checkpoint models (safetensors, etc)." }, - "Main_Checkpoint_SDXLRefiner_Config": { + "Main_Diffusers_CogView4_Config": { "properties": { "key": { "type": "string", @@ -48174,35 +48791,21 @@ ], "description": "Default settings for this model" }, - "config_path": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Config Path", - "description": "Path to the config for this model, if any." - }, "format": { "type": "string", - "const": "checkpoint", + "const": "diffusers", "title": "Format", - "default": "checkpoint" - }, - "prediction_type": { - "$ref": "#/components/schemas/SchedulerPredictionType" + "default": "diffusers" }, - "variant": { - "$ref": "#/components/schemas/ModelVariantType" + "repo_variant": { + "$ref": "#/components/schemas/ModelRepoVariant", + "default": "" }, "base": { "type": "string", - "const": "sdxl-refiner", + "const": "cogview4", "title": "Base", - "default": "sdxl-refiner" + "default": "cogview4" } }, "type": "object", @@ -48221,15 +48824,13 @@ "type", "trigger_phrases", "default_settings", - "config_path", "format", - "prediction_type", - "variant", + "repo_variant", "base" ], - "title": "Main_Checkpoint_SDXLRefiner_Config" + "title": "Main_Diffusers_CogView4_Config" }, - "Main_Checkpoint_SDXL_Config": { + "Main_Diffusers_FLUX_Config": { "properties": { "key": { "type": "string", @@ -48346,35 +48947,24 @@ ], "description": "Default settings for this model" }, - "config_path": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Config Path", - "description": "Path to the config for this model, if any." - }, "format": { "type": "string", - "const": "checkpoint", + "const": "diffusers", "title": "Format", - "default": "checkpoint" - }, - "prediction_type": { - "$ref": "#/components/schemas/SchedulerPredictionType" + "default": "diffusers" }, - "variant": { - "$ref": "#/components/schemas/ModelVariantType" + "repo_variant": { + "$ref": "#/components/schemas/ModelRepoVariant", + "default": "" }, "base": { "type": "string", - "const": "sdxl", + "const": "flux", "title": "Base", - "default": "sdxl" + "default": "flux" + }, + "variant": { + "$ref": "#/components/schemas/FluxVariantType" } }, "type": "object", @@ -48393,15 +48983,15 @@ "type", "trigger_phrases", "default_settings", - "config_path", "format", - "prediction_type", - "variant", - "base" + "repo_variant", + "base", + "variant" ], - "title": "Main_Checkpoint_SDXL_Config" + "title": "Main_Diffusers_FLUX_Config", + "description": "Model config for FLUX.1 models in diffusers format." }, - "Main_Checkpoint_ZImage_Config": { + "Main_Diffusers_Flux2_Config": { "properties": { "key": { "type": "string", @@ -48518,32 +49108,24 @@ ], "description": "Default settings for this model" }, - "config_path": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Config Path", - "description": "Path to the config for this model, if any." + "format": { + "type": "string", + "const": "diffusers", + "title": "Format", + "default": "diffusers" + }, + "repo_variant": { + "$ref": "#/components/schemas/ModelRepoVariant", + "default": "" }, "base": { "type": "string", - "const": "z-image", + "const": "flux2", "title": "Base", - "default": "z-image" - }, - "format": { - "type": "string", - "const": "checkpoint", - "title": "Format", - "default": "checkpoint" + "default": "flux2" }, "variant": { - "$ref": "#/components/schemas/ZImageVariantType" + "$ref": "#/components/schemas/Flux2VariantType" } }, "type": "object", @@ -48562,15 +49144,15 @@ "type", "trigger_phrases", "default_settings", - "config_path", - "base", "format", + "repo_variant", + "base", "variant" ], - "title": "Main_Checkpoint_ZImage_Config", - "description": "Model config for Z-Image single-file checkpoint models (safetensors, etc)." + "title": "Main_Diffusers_Flux2_Config", + "description": "Model config for FLUX.2 models in diffusers format (e.g. FLUX.2 Klein)." }, - "Main_Diffusers_CogView4_Config": { + "Main_Diffusers_QwenImage_Config": { "properties": { "key": { "type": "string", @@ -48699,9 +49281,19 @@ }, "base": { "type": "string", - "const": "cogview4", + "const": "qwen-image", "title": "Base", - "default": "cogview4" + "default": "qwen-image" + }, + "variant": { + "anyOf": [ + { + "$ref": "#/components/schemas/QwenImageVariantType" + }, + { + "type": "null" + } + ] } }, "type": "object", @@ -48722,11 +49314,13 @@ "default_settings", "format", "repo_variant", - "base" + "base", + "variant" ], - "title": "Main_Diffusers_CogView4_Config" + "title": "Main_Diffusers_QwenImage_Config", + "description": "Model config for Qwen Image diffusers models (both txt2img and edit)." }, - "Main_Diffusers_FLUX_Config": { + "Main_Diffusers_SD1_Config": { "properties": { "key": { "type": "string", @@ -48853,14 +49447,17 @@ "$ref": "#/components/schemas/ModelRepoVariant", "default": "" }, + "prediction_type": { + "$ref": "#/components/schemas/SchedulerPredictionType" + }, + "variant": { + "$ref": "#/components/schemas/ModelVariantType" + }, "base": { "type": "string", - "const": "flux", + "const": "sd-1", "title": "Base", - "default": "flux" - }, - "variant": { - "$ref": "#/components/schemas/FluxVariantType" + "default": "sd-1" } }, "type": "object", @@ -48881,13 +49478,13 @@ "default_settings", "format", "repo_variant", - "base", - "variant" + "prediction_type", + "variant", + "base" ], - "title": "Main_Diffusers_FLUX_Config", - "description": "Model config for FLUX.1 models in diffusers format." + "title": "Main_Diffusers_SD1_Config" }, - "Main_Diffusers_Flux2_Config": { + "Main_Diffusers_SD2_Config": { "properties": { "key": { "type": "string", @@ -49014,14 +49611,17 @@ "$ref": "#/components/schemas/ModelRepoVariant", "default": "" }, + "prediction_type": { + "$ref": "#/components/schemas/SchedulerPredictionType" + }, + "variant": { + "$ref": "#/components/schemas/ModelVariantType" + }, "base": { "type": "string", - "const": "flux2", + "const": "sd-2", "title": "Base", - "default": "flux2" - }, - "variant": { - "$ref": "#/components/schemas/Flux2VariantType" + "default": "sd-2" } }, "type": "object", @@ -49042,13 +49642,13 @@ "default_settings", "format", "repo_variant", - "base", - "variant" + "prediction_type", + "variant", + "base" ], - "title": "Main_Diffusers_Flux2_Config", - "description": "Model config for FLUX.2 models in diffusers format (e.g. FLUX.2 Klein)." + "title": "Main_Diffusers_SD2_Config" }, - "Main_Diffusers_QwenImage_Config": { + "Main_Diffusers_SD3_Config": { "properties": { "key": { "type": "string", @@ -49177,19 +49777,27 @@ }, "base": { "type": "string", - "const": "qwen-image", + "const": "sd-3", "title": "Base", - "default": "qwen-image" + "default": "sd-3" }, - "variant": { + "submodels": { "anyOf": [ { - "$ref": "#/components/schemas/QwenImageVariantType" + "additionalProperties": { + "$ref": "#/components/schemas/SubmodelDefinition" + }, + "propertyNames": { + "$ref": "#/components/schemas/SubModelType" + }, + "type": "object" }, { "type": "null" } - ] + ], + "title": "Submodels", + "description": "Loadable submodels in this model" } }, "type": "object", @@ -49211,12 +49819,11 @@ "format", "repo_variant", "base", - "variant" + "submodels" ], - "title": "Main_Diffusers_QwenImage_Config", - "description": "Model config for Qwen Image diffusers models (both txt2img and edit)." + "title": "Main_Diffusers_SD3_Config" }, - "Main_Diffusers_SD1_Config": { + "Main_Diffusers_SDXLRefiner_Config": { "properties": { "key": { "type": "string", @@ -49351,9 +49958,9 @@ }, "base": { "type": "string", - "const": "sd-1", + "const": "sdxl-refiner", "title": "Base", - "default": "sd-1" + "default": "sdxl-refiner" } }, "type": "object", @@ -49378,9 +49985,173 @@ "variant", "base" ], - "title": "Main_Diffusers_SD1_Config" + "title": "Main_Diffusers_SDXLRefiner_Config" }, - "Main_Diffusers_SD2_Config": { + "Main_Diffusers_SDXL_Config": { + "properties": { + "key": { + "type": "string", + "title": "Key", + "description": "A unique key for this model." + }, + "hash": { + "type": "string", + "title": "Hash", + "description": "The hash of the model file(s)." + }, + "path": { + "type": "string", + "title": "Path", + "description": "Path to the model on the filesystem. Relative paths are relative to the Invoke root directory." + }, + "file_size": { + "type": "integer", + "title": "File Size", + "description": "The size of the model in bytes." + }, + "name": { + "type": "string", + "title": "Name", + "description": "Name of the model." + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "Model description" + }, + "source": { + "type": "string", + "title": "Source", + "description": "The original source of the model (path, URL or repo_id)." + }, + "source_type": { + "$ref": "#/components/schemas/ModelSourceType", + "description": "The type of source" + }, + "source_api_response": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Api Response", + "description": "The original API response from the source, as stringified JSON." + }, + "source_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Url", + "description": "Optional URL for the model (e.g. download page or model page)." + }, + "cover_image": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Cover Image", + "description": "Url for image to preview model" + }, + "type": { + "type": "string", + "const": "main", + "title": "Type", + "default": "main" + }, + "trigger_phrases": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array", + "uniqueItems": true + }, + { + "type": "null" + } + ], + "title": "Trigger Phrases", + "description": "Set of trigger phrases for this model" + }, + "default_settings": { + "anyOf": [ + { + "$ref": "#/components/schemas/MainModelDefaultSettings" + }, + { + "type": "null" + } + ], + "description": "Default settings for this model" + }, + "format": { + "type": "string", + "const": "diffusers", + "title": "Format", + "default": "diffusers" + }, + "repo_variant": { + "$ref": "#/components/schemas/ModelRepoVariant", + "default": "" + }, + "prediction_type": { + "$ref": "#/components/schemas/SchedulerPredictionType" + }, + "variant": { + "$ref": "#/components/schemas/ModelVariantType" + }, + "base": { + "type": "string", + "const": "sdxl", + "title": "Base", + "default": "sdxl" + } + }, + "type": "object", + "required": [ + "key", + "hash", + "path", + "file_size", + "name", + "description", + "source", + "source_type", + "source_api_response", + "source_url", + "cover_image", + "type", + "trigger_phrases", + "default_settings", + "format", + "repo_variant", + "prediction_type", + "variant", + "base" + ], + "title": "Main_Diffusers_SDXL_Config" + }, + "Main_Diffusers_ZImage_Config": { "properties": { "key": { "type": "string", @@ -49507,17 +50278,14 @@ "$ref": "#/components/schemas/ModelRepoVariant", "default": "" }, - "prediction_type": { - "$ref": "#/components/schemas/SchedulerPredictionType" - }, - "variant": { - "$ref": "#/components/schemas/ModelVariantType" - }, "base": { "type": "string", - "const": "sd-2", + "const": "z-image", "title": "Base", - "default": "sd-2" + "default": "z-image" + }, + "variant": { + "$ref": "#/components/schemas/ZImageVariantType" } }, "type": "object", @@ -49538,13 +50306,13 @@ "default_settings", "format", "repo_variant", - "prediction_type", - "variant", - "base" + "base", + "variant" ], - "title": "Main_Diffusers_SD2_Config" + "title": "Main_Diffusers_ZImage_Config", + "description": "Model config for Z-Image diffusers models (Z-Image-Turbo, Z-Image-Base)." }, - "Main_Diffusers_SD3_Config": { + "Main_GGUF_FLUX_Config": { "properties": { "key": { "type": "string", @@ -49661,39 +50429,32 @@ ], "description": "Default settings for this model" }, - "format": { - "type": "string", - "const": "diffusers", - "title": "Format", - "default": "diffusers" - }, - "repo_variant": { - "$ref": "#/components/schemas/ModelRepoVariant", - "default": "" - }, - "base": { - "type": "string", - "const": "sd-3", - "title": "Base", - "default": "sd-3" - }, - "submodels": { + "config_path": { "anyOf": [ { - "additionalProperties": { - "$ref": "#/components/schemas/SubmodelDefinition" - }, - "propertyNames": { - "$ref": "#/components/schemas/SubModelType" - }, - "type": "object" + "type": "string" }, { "type": "null" } ], - "title": "Submodels", - "description": "Loadable submodels in this model" + "title": "Config Path", + "description": "Path to the config for this model, if any." + }, + "base": { + "type": "string", + "const": "flux", + "title": "Base", + "default": "flux" + }, + "format": { + "type": "string", + "const": "gguf_quantized", + "title": "Format", + "default": "gguf_quantized" + }, + "variant": { + "$ref": "#/components/schemas/FluxVariantType" } }, "type": "object", @@ -49712,14 +50473,15 @@ "type", "trigger_phrases", "default_settings", - "format", - "repo_variant", + "config_path", "base", - "submodels" + "format", + "variant" ], - "title": "Main_Diffusers_SD3_Config" + "title": "Main_GGUF_FLUX_Config", + "description": "Model config for main checkpoint models." }, - "Main_Diffusers_SDXLRefiner_Config": { + "Main_GGUF_Flux2_Config": { "properties": { "key": { "type": "string", @@ -49836,27 +50598,32 @@ ], "description": "Default settings for this model" }, + "config_path": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Config Path", + "description": "Path to the config for this model, if any." + }, + "base": { + "type": "string", + "const": "flux2", + "title": "Base", + "default": "flux2" + }, "format": { "type": "string", - "const": "diffusers", + "const": "gguf_quantized", "title": "Format", - "default": "diffusers" - }, - "repo_variant": { - "$ref": "#/components/schemas/ModelRepoVariant", - "default": "" - }, - "prediction_type": { - "$ref": "#/components/schemas/SchedulerPredictionType" + "default": "gguf_quantized" }, "variant": { - "$ref": "#/components/schemas/ModelVariantType" - }, - "base": { - "type": "string", - "const": "sdxl-refiner", - "title": "Base", - "default": "sdxl-refiner" + "$ref": "#/components/schemas/Flux2VariantType" } }, "type": "object", @@ -49875,15 +50642,15 @@ "type", "trigger_phrases", "default_settings", + "config_path", + "base", "format", - "repo_variant", - "prediction_type", - "variant", - "base" + "variant" ], - "title": "Main_Diffusers_SDXLRefiner_Config" + "title": "Main_GGUF_Flux2_Config", + "description": "Model config for GGUF-quantized FLUX.2 checkpoint models (e.g. Klein)." }, - "Main_Diffusers_SDXL_Config": { + "Main_GGUF_QwenImage_Config": { "properties": { "key": { "type": "string", @@ -50000,27 +50767,39 @@ ], "description": "Default settings for this model" }, + "config_path": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Config Path", + "description": "Path to the config for this model, if any." + }, + "base": { + "type": "string", + "const": "qwen-image", + "title": "Base", + "default": "qwen-image" + }, "format": { "type": "string", - "const": "diffusers", + "const": "gguf_quantized", "title": "Format", - "default": "diffusers" - }, - "repo_variant": { - "$ref": "#/components/schemas/ModelRepoVariant", - "default": "" - }, - "prediction_type": { - "$ref": "#/components/schemas/SchedulerPredictionType" + "default": "gguf_quantized" }, "variant": { - "$ref": "#/components/schemas/ModelVariantType" - }, - "base": { - "type": "string", - "const": "sdxl", - "title": "Base", - "default": "sdxl" + "anyOf": [ + { + "$ref": "#/components/schemas/QwenImageVariantType" + }, + { + "type": "null" + } + ] } }, "type": "object", @@ -50039,15 +50818,15 @@ "type", "trigger_phrases", "default_settings", + "config_path", + "base", "format", - "repo_variant", - "prediction_type", - "variant", - "base" + "variant" ], - "title": "Main_Diffusers_SDXL_Config" + "title": "Main_GGUF_QwenImage_Config", + "description": "Model config for GGUF-quantized Qwen Image transformer models." }, - "Main_Diffusers_ZImage_Config": { + "Main_GGUF_ZImage_Config": { "properties": { "key": { "type": "string", @@ -50164,15 +50943,17 @@ ], "description": "Default settings for this model" }, - "format": { - "type": "string", - "const": "diffusers", - "title": "Format", - "default": "diffusers" - }, - "repo_variant": { - "$ref": "#/components/schemas/ModelRepoVariant", - "default": "" + "config_path": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Config Path", + "description": "Path to the config for this model, if any." }, "base": { "type": "string", @@ -50180,6 +50961,12 @@ "title": "Base", "default": "z-image" }, + "format": { + "type": "string", + "const": "gguf_quantized", + "title": "Format", + "default": "gguf_quantized" + }, "variant": { "$ref": "#/components/schemas/ZImageVariantType" } @@ -50200,15 +50987,15 @@ "type", "trigger_phrases", "default_settings", - "format", - "repo_variant", + "config_path", "base", + "format", "variant" ], - "title": "Main_Diffusers_ZImage_Config", - "description": "Model config for Z-Image diffusers models (Z-Image-Turbo, Z-Image-Base)." + "title": "Main_GGUF_ZImage_Config", + "description": "Model config for GGUF-quantized Z-Image transformer models." }, - "Main_GGUF_FLUX_Config": { + "Main_SDNQ_Diffusers_FLUX_Config": { "properties": { "key": { "type": "string", @@ -50325,18 +51112,6 @@ ], "description": "Default settings for this model" }, - "config_path": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Config Path", - "description": "Path to the config for this model, if any." - }, "base": { "type": "string", "const": "flux", @@ -50345,12 +51120,34 @@ }, "format": { "type": "string", - "const": "gguf_quantized", + "const": "sdnq_quantized", "title": "Format", - "default": "gguf_quantized" + "default": "sdnq_quantized" }, "variant": { "$ref": "#/components/schemas/FluxVariantType" + }, + "repo_variant": { + "$ref": "#/components/schemas/ModelRepoVariant", + "default": "" + }, + "submodels": { + "anyOf": [ + { + "additionalProperties": { + "$ref": "#/components/schemas/SubmodelDefinition" + }, + "propertyNames": { + "$ref": "#/components/schemas/SubModelType" + }, + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Submodels", + "description": "Loadable submodels in this model" } }, "type": "object", @@ -50369,15 +51166,16 @@ "type", "trigger_phrases", "default_settings", - "config_path", "base", "format", - "variant" + "variant", + "repo_variant", + "submodels" ], - "title": "Main_GGUF_FLUX_Config", - "description": "Model config for main checkpoint models." + "title": "Main_SDNQ_Diffusers_FLUX_Config", + "description": "Model config for SDNQ-quantized FLUX models in diffusers format (folder with transformer, text_encoder, etc.)." }, - "Main_GGUF_Flux2_Config": { + "Main_SDNQ_Diffusers_Flux2_Config": { "properties": { "key": { "type": "string", @@ -50494,18 +51292,6 @@ ], "description": "Default settings for this model" }, - "config_path": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Config Path", - "description": "Path to the config for this model, if any." - }, "base": { "type": "string", "const": "flux2", @@ -50514,188 +51300,34 @@ }, "format": { "type": "string", - "const": "gguf_quantized", + "const": "sdnq_quantized", "title": "Format", - "default": "gguf_quantized" + "default": "sdnq_quantized" }, "variant": { "$ref": "#/components/schemas/Flux2VariantType" - } - }, - "type": "object", - "required": [ - "key", - "hash", - "path", - "file_size", - "name", - "description", - "source", - "source_type", - "source_api_response", - "source_url", - "cover_image", - "type", - "trigger_phrases", - "default_settings", - "config_path", - "base", - "format", - "variant" - ], - "title": "Main_GGUF_Flux2_Config", - "description": "Model config for GGUF-quantized FLUX.2 checkpoint models (e.g. Klein)." - }, - "Main_GGUF_QwenImage_Config": { - "properties": { - "key": { - "type": "string", - "title": "Key", - "description": "A unique key for this model." - }, - "hash": { - "type": "string", - "title": "Hash", - "description": "The hash of the model file(s)." - }, - "path": { - "type": "string", - "title": "Path", - "description": "Path to the model on the filesystem. Relative paths are relative to the Invoke root directory." }, - "file_size": { - "type": "integer", - "title": "File Size", - "description": "The size of the model in bytes." - }, - "name": { - "type": "string", - "title": "Name", - "description": "Name of the model." - }, - "description": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Description", - "description": "Model description" - }, - "source": { - "type": "string", - "title": "Source", - "description": "The original source of the model (path, URL or repo_id)." - }, - "source_type": { - "$ref": "#/components/schemas/ModelSourceType", - "description": "The type of source" - }, - "source_api_response": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Source Api Response", - "description": "The original API response from the source, as stringified JSON." - }, - "source_url": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Source Url", - "description": "Optional URL for the model (e.g. download page or model page)." - }, - "cover_image": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Cover Image", - "description": "Url for image to preview model" - }, - "type": { - "type": "string", - "const": "main", - "title": "Type", - "default": "main" + "repo_variant": { + "$ref": "#/components/schemas/ModelRepoVariant", + "default": "" }, - "trigger_phrases": { + "submodels": { "anyOf": [ { - "items": { - "type": "string" + "additionalProperties": { + "$ref": "#/components/schemas/SubmodelDefinition" }, - "type": "array", - "uniqueItems": true - }, - { - "type": "null" - } - ], - "title": "Trigger Phrases", - "description": "Set of trigger phrases for this model" - }, - "default_settings": { - "anyOf": [ - { - "$ref": "#/components/schemas/MainModelDefaultSettings" - }, - { - "type": "null" - } - ], - "description": "Default settings for this model" - }, - "config_path": { - "anyOf": [ - { - "type": "string" + "propertyNames": { + "$ref": "#/components/schemas/SubModelType" + }, + "type": "object" }, { "type": "null" } ], - "title": "Config Path", - "description": "Path to the config for this model, if any." - }, - "base": { - "type": "string", - "const": "qwen-image", - "title": "Base", - "default": "qwen-image" - }, - "format": { - "type": "string", - "const": "gguf_quantized", - "title": "Format", - "default": "gguf_quantized" - }, - "variant": { - "anyOf": [ - { - "$ref": "#/components/schemas/QwenImageVariantType" - }, - { - "type": "null" - } - ] + "title": "Submodels", + "description": "Loadable submodels in this model" } }, "type": "object", @@ -50714,15 +51346,16 @@ "type", "trigger_phrases", "default_settings", - "config_path", "base", "format", - "variant" + "variant", + "repo_variant", + "submodels" ], - "title": "Main_GGUF_QwenImage_Config", - "description": "Model config for GGUF-quantized Qwen Image transformer models." + "title": "Main_SDNQ_Diffusers_Flux2_Config", + "description": "Model config for SDNQ-quantized FLUX.2 models in diffusers format\n(Flux2KleinPipeline / Flux2Pipeline folder with transformer/, text_encoder/, vae/, ...)." }, - "Main_GGUF_ZImage_Config": { + "Main_SDNQ_Diffusers_ZImage_Config": { "properties": { "key": { "type": "string", @@ -50839,18 +51472,6 @@ ], "description": "Default settings for this model" }, - "config_path": { - "anyOf": [ - { - "type": "string" - }, - { - "type": "null" - } - ], - "title": "Config Path", - "description": "Path to the config for this model, if any." - }, "base": { "type": "string", "const": "z-image", @@ -50859,12 +51480,34 @@ }, "format": { "type": "string", - "const": "gguf_quantized", + "const": "sdnq_quantized", "title": "Format", - "default": "gguf_quantized" + "default": "sdnq_quantized" }, "variant": { "$ref": "#/components/schemas/ZImageVariantType" + }, + "repo_variant": { + "$ref": "#/components/schemas/ModelRepoVariant", + "default": "" + }, + "submodels": { + "anyOf": [ + { + "additionalProperties": { + "$ref": "#/components/schemas/SubmodelDefinition" + }, + "propertyNames": { + "$ref": "#/components/schemas/SubModelType" + }, + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Submodels", + "description": "Loadable submodels in this model" } }, "type": "object", @@ -50883,13 +51526,14 @@ "type", "trigger_phrases", "default_settings", - "config_path", "base", "format", - "variant" + "variant", + "repo_variant", + "submodels" ], - "title": "Main_GGUF_ZImage_Config", - "description": "Model config for GGUF-quantized Z-Image transformer models." + "title": "Main_SDNQ_Diffusers_ZImage_Config", + "description": "Model config for SDNQ-quantized Z-Image models in diffusers format (full ZImagePipeline folder)." }, "MaskCombineInvocation": { "category": "mask", @@ -54406,6 +55050,7 @@ "bnb_quantized_nf4b", "gguf_quantized", "external_api", + "sdnq_quantized", "unknown" ], "title": "ModelFormat", @@ -54714,6 +55359,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -54828,6 +55482,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -54837,6 +55494,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -55283,6 +55943,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -55397,6 +56066,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -55406,6 +56078,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -55737,6 +56412,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -55851,6 +56535,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -55860,6 +56547,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -56041,6 +56731,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -56155,6 +56854,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -56164,6 +56866,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -56794,6 +57499,15 @@ { "$ref": "#/components/schemas/Main_GGUF_ZImage_Config" }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_FLUX_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_Flux2_Config" + }, + { + "$ref": "#/components/schemas/Main_SDNQ_Diffusers_ZImage_Config" + }, { "$ref": "#/components/schemas/VAE_Checkpoint_SD1_Config" }, @@ -56908,6 +57622,9 @@ { "$ref": "#/components/schemas/T5Encoder_BnBLLMint8_Config" }, + { + "$ref": "#/components/schemas/T5Encoder_SDNQ_Config" + }, { "$ref": "#/components/schemas/Qwen3Encoder_Qwen3Encoder_Config" }, @@ -56917,6 +57634,9 @@ { "$ref": "#/components/schemas/Qwen3Encoder_GGUF_Config" }, + { + "$ref": "#/components/schemas/Qwen3Encoder_SDNQ_Folder_Config" + }, { "$ref": "#/components/schemas/QwenVLEncoder_Diffusers_Config" }, @@ -59561,6 +60281,147 @@ "title": "Qwen3Encoder_Qwen3Encoder_Config", "description": "Configuration for Qwen3 Encoder models in a diffusers-like format.\n\nThe model weights are expected to be in a folder called text_encoder inside the model directory,\ncompatible with Qwen2VLForConditionalGeneration or similar architectures used by Z-Image." }, + "Qwen3Encoder_SDNQ_Folder_Config": { + "properties": { + "key": { + "type": "string", + "title": "Key", + "description": "A unique key for this model." + }, + "hash": { + "type": "string", + "title": "Hash", + "description": "The hash of the model file(s)." + }, + "path": { + "type": "string", + "title": "Path", + "description": "Path to the model on the filesystem. Relative paths are relative to the Invoke root directory." + }, + "file_size": { + "type": "integer", + "title": "File Size", + "description": "The size of the model in bytes." + }, + "name": { + "type": "string", + "title": "Name", + "description": "Name of the model." + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "Model description" + }, + "source": { + "type": "string", + "title": "Source", + "description": "The original source of the model (path, URL or repo_id)." + }, + "source_type": { + "$ref": "#/components/schemas/ModelSourceType", + "description": "The type of source" + }, + "source_api_response": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Api Response", + "description": "The original API response from the source, as stringified JSON." + }, + "source_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Url", + "description": "Optional URL for the model (e.g. download page or model page)." + }, + "cover_image": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Cover Image", + "description": "Url for image to preview model" + }, + "base": { + "type": "string", + "const": "any", + "title": "Base", + "default": "any" + }, + "type": { + "type": "string", + "const": "qwen3_encoder", + "title": "Type", + "default": "qwen3_encoder" + }, + "format": { + "type": "string", + "const": "sdnq_quantized", + "title": "Format", + "default": "sdnq_quantized" + }, + "cpu_only": { + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "title": "Cpu Only", + "description": "Whether this model should run on CPU only" + }, + "variant": { + "$ref": "#/components/schemas/Qwen3VariantType", + "description": "Qwen3 model size variant (4B or 8B)" + } + }, + "type": "object", + "required": [ + "key", + "hash", + "path", + "file_size", + "name", + "description", + "source", + "source_type", + "source_api_response", + "source_url", + "cover_image", + "base", + "type", + "format", + "cpu_only", + "variant" + ], + "title": "Qwen3Encoder_SDNQ_Folder_Config", + "description": "Configuration for folder-based SDNQ-quantized Qwen3 Encoder models.\n\nUsed for SDNQ bundles where the text_encoder is a folder containing\nquantization_config.json and safetensors files with SDNQ keys." + }, "Qwen3VariantType": { "type": "string", "enum": ["qwen3_4b", "qwen3_8b", "qwen3_06b"], @@ -67752,6 +68613,142 @@ "title": "T5Encoder_BnBLLMint8_Config", "description": "Configuration for T5 Encoder models quantized by bitsandbytes' LLM.int8." }, + "T5Encoder_SDNQ_Config": { + "properties": { + "key": { + "type": "string", + "title": "Key", + "description": "A unique key for this model." + }, + "hash": { + "type": "string", + "title": "Hash", + "description": "The hash of the model file(s)." + }, + "path": { + "type": "string", + "title": "Path", + "description": "Path to the model on the filesystem. Relative paths are relative to the Invoke root directory." + }, + "file_size": { + "type": "integer", + "title": "File Size", + "description": "The size of the model in bytes." + }, + "name": { + "type": "string", + "title": "Name", + "description": "Name of the model." + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "Model description" + }, + "source": { + "type": "string", + "title": "Source", + "description": "The original source of the model (path, URL or repo_id)." + }, + "source_type": { + "$ref": "#/components/schemas/ModelSourceType", + "description": "The type of source" + }, + "source_api_response": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Api Response", + "description": "The original API response from the source, as stringified JSON." + }, + "source_url": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Url", + "description": "Optional URL for the model (e.g. download page or model page)." + }, + "cover_image": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Cover Image", + "description": "Url for image to preview model" + }, + "base": { + "type": "string", + "const": "any", + "title": "Base", + "default": "any" + }, + "type": { + "type": "string", + "const": "t5_encoder", + "title": "Type", + "default": "t5_encoder" + }, + "format": { + "type": "string", + "const": "sdnq_quantized", + "title": "Format", + "default": "sdnq_quantized" + }, + "cpu_only": { + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "title": "Cpu Only", + "description": "Whether this model should run on CPU only" + } + }, + "type": "object", + "required": [ + "key", + "hash", + "path", + "file_size", + "name", + "description", + "source", + "source_type", + "source_api_response", + "source_url", + "cover_image", + "base", + "type", + "format", + "cpu_only" + ], + "title": "T5Encoder_SDNQ_Config", + "description": "Configuration for SDNQ-quantized T5 Encoder models.\n\nMatches two layouts:\n\n1. **Standalone T5 bundle**: ``mod.path`` is the pipeline-style root, with\n ``text_encoder_2/`` (and usually ``tokenizer_2/``) as subfolders.\n2. **Inline submodel**: ``mod.path`` *is* the ``text_encoder_2`` folder itself \u2014\n this is how a parent FluxPipeline / similar config registers its T5 submodel\n (``submodels[TextEncoder2].path_or_prefix`` points straight at the folder).\n\nIn both cases, the SDNQ-quantized state lives next to a ``config.json`` declaring\n``T5EncoderModel`` and is signalled either by ``quantization_config.json`` with\n``quant_method == \"sdnq\"`` or by SDNQ-style ``weight`` + ``scale`` key pairs." + }, "T5Encoder_T5Encoder_Config": { "properties": { "key": { diff --git a/invokeai/frontend/web/src/features/modelManagerV2/models.ts b/invokeai/frontend/web/src/features/modelManagerV2/models.ts index cf295c9af6a..45dcf975fc4 100644 --- a/invokeai/frontend/web/src/features/modelManagerV2/models.ts +++ b/invokeai/frontend/web/src/features/modelManagerV2/models.ts @@ -274,6 +274,7 @@ export const MODEL_FORMAT_TO_LONG_NAME: Record = { bnb_quantized_int8b: 'BNB Quantized (int8b)', bnb_quantized_nf4b: 'BNB Quantized (nf4b)', gguf_quantized: 'GGUF Quantized', + sdnq_quantized: 'SDNQ Quantized', unknown: 'Unknown', }; diff --git a/invokeai/frontend/web/src/features/modelManagerV2/subpanels/ModelManagerPanel/ModelFormatBadge.tsx b/invokeai/frontend/web/src/features/modelManagerV2/subpanels/ModelManagerPanel/ModelFormatBadge.tsx index 71d2efe0e45..ecc1eee29f4 100644 --- a/invokeai/frontend/web/src/features/modelManagerV2/subpanels/ModelManagerPanel/ModelFormatBadge.tsx +++ b/invokeai/frontend/web/src/features/modelManagerV2/subpanels/ModelManagerPanel/ModelFormatBadge.tsx @@ -19,6 +19,7 @@ const FORMAT_NAME_MAP: Record = { bnb_quantized_int8b: 'bnb_quantized_int8b', bnb_quantized_nf4b: 'quantized', gguf_quantized: 'gguf', + sdnq_quantized: 'sdnq', omi: 'omi', external_api: 'external_api', unknown: 'unknown', @@ -40,6 +41,7 @@ const FORMAT_COLOR_MAP: Record = { bnb_quantized_int8b: 'base', bnb_quantized_nf4b: 'base', gguf_quantized: 'base', + sdnq_quantized: 'base', unknown: 'red', olive: 'base', onnx: 'base', diff --git a/invokeai/frontend/web/src/features/nodes/types/common.ts b/invokeai/frontend/web/src/features/nodes/types/common.ts index fb2a1ce946a..95d7fd38f32 100644 --- a/invokeai/frontend/web/src/features/nodes/types/common.ts +++ b/invokeai/frontend/web/src/features/nodes/types/common.ts @@ -190,6 +190,7 @@ export const zModelFormat = z.enum([ 'bnb_quantized_int8b', 'bnb_quantized_nf4b', 'gguf_quantized', + 'sdnq_quantized', 'external_api', 'unknown', ]); diff --git a/invokeai/frontend/web/src/features/queue/store/readiness.ts b/invokeai/frontend/web/src/features/queue/store/readiness.ts index 230fa3348d6..df62e961b8d 100644 --- a/invokeai/frontend/web/src/features/queue/store/readiness.ts +++ b/invokeai/frontend/web/src/features/queue/store/readiness.ts @@ -289,15 +289,23 @@ export const getReasonsWhyCannotEnqueueGenerateTab = (arg: { } } - if (model?.base === 'flux2' && model.format !== 'diffusers') { - // Non-diffusers FLUX.2 Klein models require standalone VAE and Qwen3 Encoder - // unless a diffusers flux2 model is available to extract them from. - // VAE is shared across variants, but Qwen3 encoder requires a variant-matching diffusers model. - if (!params.kleinVaeModel && !hasFlux2DiffusersVaeSource) { - reasons.push({ content: i18n.t('parameters.invoke.noFlux2KleinVaeModelSelected') }); - } - if (!params.kleinQwen3EncoderModel && !hasFlux2DiffusersQwen3Source) { - reasons.push({ content: i18n.t('parameters.invoke.noFlux2KleinQwen3EncoderModelSelected') }); + if (model?.base === 'flux2') { + // A FLUX.2 Klein model is a self-sufficient source when its config exposes the diffusers-style + // submodels (transformer/vae/text_encoder/tokenizer). That's the case for both plain Diffusers + // pipelines and SDNQ-quantized ZImagePipeline / Flux2KleinPipeline folders. Single-file or + // GGUF Klein models don't have submodels and need a standalone VAE + Qwen3 (or a diffusers- + // shaped source model installed elsewhere). + const mainIsPipeline = + model.format === 'diffusers' || + ((model as { format?: unknown }).format === 'sdnq_quantized' && + Boolean((model as { submodels?: unknown }).submodels)); + if (!mainIsPipeline) { + if (!params.kleinVaeModel && !hasFlux2DiffusersVaeSource) { + reasons.push({ content: i18n.t('parameters.invoke.noFlux2KleinVaeModelSelected') }); + } + if (!params.kleinQwen3EncoderModel && !hasFlux2DiffusersQwen3Source) { + reasons.push({ content: i18n.t('parameters.invoke.noFlux2KleinQwen3EncoderModelSelected') }); + } } } diff --git a/invokeai/frontend/web/src/services/api/schema.ts b/invokeai/frontend/web/src/services/api/schema.ts index 7ca0f26fe9f..c1f58784b55 100644 --- a/invokeai/frontend/web/src/services/api/schema.ts +++ b/invokeai/frontend/web/src/services/api/schema.ts @@ -3563,7 +3563,7 @@ export type components = { */ type: "anima_text_encoder"; }; - AnyModelConfig: components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; + AnyModelConfig: components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; /** * AppVersion * @description App Version Response @@ -10464,8 +10464,8 @@ export type components = { */ qwen3_encoder_model?: components["schemas"]["ModelIdentifierField"] | null; /** - * Qwen3 Source (Diffusers) - * @description Diffusers Flux2 Klein model to extract VAE and/or Qwen3 encoder from. Use this if you don't have separate VAE/Qwen3 models. Ignored if both VAE and Qwen3 Encoder are provided separately. + * Qwen3 Source + * @description Diffusers or SDNQ-pipeline Flux2 Klein model to extract VAE and/or Qwen3 encoder from. Use this if you don't have separate VAE/Qwen3 models. Ignored if both VAE and Qwen3 Encoder are provided separately. * @default null */ qwen3_source_model?: components["schemas"]["ModelIdentifierField"] | null; @@ -21564,6 +21564,286 @@ export type components = { format: "gguf_quantized"; variant: components["schemas"]["ZImageVariantType"]; }; + /** + * Main_SDNQ_Diffusers_FLUX_Config + * @description Model config for SDNQ-quantized FLUX models in diffusers format (folder with transformer, text_encoder, etc.). + */ + Main_SDNQ_Diffusers_FLUX_Config: { + /** + * Key + * @description A unique key for this model. + */ + key: string; + /** + * Hash + * @description The hash of the model file(s). + */ + hash: string; + /** + * Path + * @description Path to the model on the filesystem. Relative paths are relative to the Invoke root directory. + */ + path: string; + /** + * File Size + * @description The size of the model in bytes. + */ + file_size: number; + /** + * Name + * @description Name of the model. + */ + name: string; + /** + * Description + * @description Model description + */ + description: string | null; + /** + * Source + * @description The original source of the model (path, URL or repo_id). + */ + source: string; + /** @description The type of source */ + source_type: components["schemas"]["ModelSourceType"]; + /** + * Source Api Response + * @description The original API response from the source, as stringified JSON. + */ + source_api_response: string | null; + /** + * Source Url + * @description Optional URL for the model (e.g. download page or model page). + */ + source_url: string | null; + /** + * Cover Image + * @description Url for image to preview model + */ + cover_image: string | null; + /** + * Type + * @default main + * @constant + */ + type: "main"; + /** + * Trigger Phrases + * @description Set of trigger phrases for this model + */ + trigger_phrases: string[] | null; + /** @description Default settings for this model */ + default_settings: components["schemas"]["MainModelDefaultSettings"] | null; + /** + * Base + * @default flux + * @constant + */ + base: "flux"; + /** + * Format + * @default sdnq_quantized + * @constant + */ + format: "sdnq_quantized"; + variant: components["schemas"]["FluxVariantType"]; + /** @default */ + repo_variant: components["schemas"]["ModelRepoVariant"]; + /** + * Submodels + * @description Loadable submodels in this model + */ + submodels: { + [key: string]: components["schemas"]["SubmodelDefinition"]; + } | null; + }; + /** + * Main_SDNQ_Diffusers_Flux2_Config + * @description Model config for SDNQ-quantized FLUX.2 models in diffusers format + * (Flux2KleinPipeline / Flux2Pipeline folder with transformer/, text_encoder/, vae/, ...). + */ + Main_SDNQ_Diffusers_Flux2_Config: { + /** + * Key + * @description A unique key for this model. + */ + key: string; + /** + * Hash + * @description The hash of the model file(s). + */ + hash: string; + /** + * Path + * @description Path to the model on the filesystem. Relative paths are relative to the Invoke root directory. + */ + path: string; + /** + * File Size + * @description The size of the model in bytes. + */ + file_size: number; + /** + * Name + * @description Name of the model. + */ + name: string; + /** + * Description + * @description Model description + */ + description: string | null; + /** + * Source + * @description The original source of the model (path, URL or repo_id). + */ + source: string; + /** @description The type of source */ + source_type: components["schemas"]["ModelSourceType"]; + /** + * Source Api Response + * @description The original API response from the source, as stringified JSON. + */ + source_api_response: string | null; + /** + * Source Url + * @description Optional URL for the model (e.g. download page or model page). + */ + source_url: string | null; + /** + * Cover Image + * @description Url for image to preview model + */ + cover_image: string | null; + /** + * Type + * @default main + * @constant + */ + type: "main"; + /** + * Trigger Phrases + * @description Set of trigger phrases for this model + */ + trigger_phrases: string[] | null; + /** @description Default settings for this model */ + default_settings: components["schemas"]["MainModelDefaultSettings"] | null; + /** + * Base + * @default flux2 + * @constant + */ + base: "flux2"; + /** + * Format + * @default sdnq_quantized + * @constant + */ + format: "sdnq_quantized"; + variant: components["schemas"]["Flux2VariantType"]; + /** @default */ + repo_variant: components["schemas"]["ModelRepoVariant"]; + /** + * Submodels + * @description Loadable submodels in this model + */ + submodels: { + [key: string]: components["schemas"]["SubmodelDefinition"]; + } | null; + }; + /** + * Main_SDNQ_Diffusers_ZImage_Config + * @description Model config for SDNQ-quantized Z-Image models in diffusers format (full ZImagePipeline folder). + */ + Main_SDNQ_Diffusers_ZImage_Config: { + /** + * Key + * @description A unique key for this model. + */ + key: string; + /** + * Hash + * @description The hash of the model file(s). + */ + hash: string; + /** + * Path + * @description Path to the model on the filesystem. Relative paths are relative to the Invoke root directory. + */ + path: string; + /** + * File Size + * @description The size of the model in bytes. + */ + file_size: number; + /** + * Name + * @description Name of the model. + */ + name: string; + /** + * Description + * @description Model description + */ + description: string | null; + /** + * Source + * @description The original source of the model (path, URL or repo_id). + */ + source: string; + /** @description The type of source */ + source_type: components["schemas"]["ModelSourceType"]; + /** + * Source Api Response + * @description The original API response from the source, as stringified JSON. + */ + source_api_response: string | null; + /** + * Source Url + * @description Optional URL for the model (e.g. download page or model page). + */ + source_url: string | null; + /** + * Cover Image + * @description Url for image to preview model + */ + cover_image: string | null; + /** + * Type + * @default main + * @constant + */ + type: "main"; + /** + * Trigger Phrases + * @description Set of trigger phrases for this model + */ + trigger_phrases: string[] | null; + /** @description Default settings for this model */ + default_settings: components["schemas"]["MainModelDefaultSettings"] | null; + /** + * Base + * @default z-image + * @constant + */ + base: "z-image"; + /** + * Format + * @default sdnq_quantized + * @constant + */ + format: "sdnq_quantized"; + variant: components["schemas"]["ZImageVariantType"]; + /** @default */ + repo_variant: components["schemas"]["ModelRepoVariant"]; + /** + * Submodels + * @description Loadable submodels in this model + */ + submodels: { + [key: string]: components["schemas"]["SubmodelDefinition"]; + } | null; + }; /** * Combine Masks * @description Combine two masks together by multiplying them using `PIL.ImageChops.multiply()`. @@ -23255,7 +23535,7 @@ export type components = { * @description Storage format of model. * @enum {string} */ - ModelFormat: "omi" | "diffusers" | "checkpoint" | "lycoris" | "onnx" | "olive" | "embedding_file" | "embedding_folder" | "invokeai" | "t5_encoder" | "qwen3_encoder" | "qwen_vl_encoder" | "bnb_quantized_int8b" | "bnb_quantized_nf4b" | "gguf_quantized" | "external_api" | "unknown"; + ModelFormat: "omi" | "diffusers" | "checkpoint" | "lycoris" | "onnx" | "olive" | "embedding_file" | "embedding_folder" | "invokeai" | "t5_encoder" | "qwen3_encoder" | "qwen_vl_encoder" | "bnb_quantized_int8b" | "bnb_quantized_nf4b" | "gguf_quantized" | "external_api" | "sdnq_quantized" | "unknown"; /** ModelIdentifierField */ ModelIdentifierField: { /** @@ -23392,7 +23672,7 @@ export type components = { * Config * @description The installed model's config */ - config: components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; + config: components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; }; /** * ModelInstallDownloadProgressEvent @@ -23558,7 +23838,7 @@ export type components = { * Config Out * @description After successful installation, this will hold the configuration object. */ - config_out?: (components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]) | null; + config_out?: (components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]) | null; /** * Inplace * @description Leave model in its current location; otherwise install under models directory @@ -23644,7 +23924,7 @@ export type components = { * Config * @description The model's config */ - config: components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; + config: components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; /** * @description The submodel type, if any * @default null @@ -23665,7 +23945,7 @@ export type components = { * Config * @description The model's config */ - config: components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; + config: components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; /** * @description The submodel type, if any * @default null @@ -23862,7 +24142,7 @@ export type components = { */ ModelsList: { /** Models */ - models: (components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"])[]; + models: (components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"])[]; }; /** * Multiply Integers @@ -25284,6 +25564,92 @@ export type components = { /** @description Qwen3 model size variant (4B or 8B) */ variant: components["schemas"]["Qwen3VariantType"]; }; + /** + * Qwen3Encoder_SDNQ_Folder_Config + * @description Configuration for folder-based SDNQ-quantized Qwen3 Encoder models. + * + * Used for SDNQ bundles where the text_encoder is a folder containing + * quantization_config.json and safetensors files with SDNQ keys. + */ + Qwen3Encoder_SDNQ_Folder_Config: { + /** + * Key + * @description A unique key for this model. + */ + key: string; + /** + * Hash + * @description The hash of the model file(s). + */ + hash: string; + /** + * Path + * @description Path to the model on the filesystem. Relative paths are relative to the Invoke root directory. + */ + path: string; + /** + * File Size + * @description The size of the model in bytes. + */ + file_size: number; + /** + * Name + * @description Name of the model. + */ + name: string; + /** + * Description + * @description Model description + */ + description: string | null; + /** + * Source + * @description The original source of the model (path, URL or repo_id). + */ + source: string; + /** @description The type of source */ + source_type: components["schemas"]["ModelSourceType"]; + /** + * Source Api Response + * @description The original API response from the source, as stringified JSON. + */ + source_api_response: string | null; + /** + * Source Url + * @description Optional URL for the model (e.g. download page or model page). + */ + source_url: string | null; + /** + * Cover Image + * @description Url for image to preview model + */ + cover_image: string | null; + /** + * Base + * @default any + * @constant + */ + base: "any"; + /** + * Type + * @default qwen3_encoder + * @constant + */ + type: "qwen3_encoder"; + /** + * Format + * @default sdnq_quantized + * @constant + */ + format: "sdnq_quantized"; + /** + * Cpu Only + * @description Whether this model should run on CPU only + */ + cpu_only: boolean | null; + /** @description Qwen3 model size variant (4B or 8B) */ + variant: components["schemas"]["Qwen3VariantType"]; + }; /** * Qwen3VariantType * @description Qwen3 text encoder variants based on model size. @@ -29538,6 +29904,99 @@ export type components = { */ cpu_only: boolean | null; }; + /** + * T5Encoder_SDNQ_Config + * @description Configuration for SDNQ-quantized T5 Encoder models. + * + * Matches two layouts: + * + * 1. **Standalone T5 bundle**: ``mod.path`` is the pipeline-style root, with + * ``text_encoder_2/`` (and usually ``tokenizer_2/``) as subfolders. + * 2. **Inline submodel**: ``mod.path`` *is* the ``text_encoder_2`` folder itself — + * this is how a parent FluxPipeline / similar config registers its T5 submodel + * (``submodels[TextEncoder2].path_or_prefix`` points straight at the folder). + * + * In both cases, the SDNQ-quantized state lives next to a ``config.json`` declaring + * ``T5EncoderModel`` and is signalled either by ``quantization_config.json`` with + * ``quant_method == "sdnq"`` or by SDNQ-style ``weight`` + ``scale`` key pairs. + */ + T5Encoder_SDNQ_Config: { + /** + * Key + * @description A unique key for this model. + */ + key: string; + /** + * Hash + * @description The hash of the model file(s). + */ + hash: string; + /** + * Path + * @description Path to the model on the filesystem. Relative paths are relative to the Invoke root directory. + */ + path: string; + /** + * File Size + * @description The size of the model in bytes. + */ + file_size: number; + /** + * Name + * @description Name of the model. + */ + name: string; + /** + * Description + * @description Model description + */ + description: string | null; + /** + * Source + * @description The original source of the model (path, URL or repo_id). + */ + source: string; + /** @description The type of source */ + source_type: components["schemas"]["ModelSourceType"]; + /** + * Source Api Response + * @description The original API response from the source, as stringified JSON. + */ + source_api_response: string | null; + /** + * Source Url + * @description Optional URL for the model (e.g. download page or model page). + */ + source_url: string | null; + /** + * Cover Image + * @description Url for image to preview model + */ + cover_image: string | null; + /** + * Base + * @default any + * @constant + */ + base: "any"; + /** + * Type + * @default t5_encoder + * @constant + */ + type: "t5_encoder"; + /** + * Format + * @default sdnq_quantized + * @constant + */ + format: "sdnq_quantized"; + /** + * Cpu Only + * @description Whether this model should run on CPU only + */ + cpu_only: boolean | null; + }; /** * T5Encoder_T5Encoder_Config * @description Configuration for T5 Encoder models in a bespoke, diffusers-like format. The model weights are expected to be in @@ -33408,7 +33867,7 @@ export interface operations { [name: string]: unknown; }; content: { - "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; + "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; }; }; /** @description Validation Error */ @@ -33440,7 +33899,7 @@ export interface operations { [name: string]: unknown; }; content: { - "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; + "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; }; }; /** @description Validation Error */ @@ -33490,7 +33949,7 @@ export interface operations { * "repo_variant": "fp16", * "upcast_attention": false * } */ - "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; + "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; }; }; /** @description Bad request */ @@ -33595,7 +34054,7 @@ export interface operations { * "repo_variant": "fp16", * "upcast_attention": false * } */ - "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; + "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; }; }; /** @description Bad request */ @@ -33666,7 +34125,7 @@ export interface operations { * "repo_variant": "fp16", * "upcast_attention": false * } */ - "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; + "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; }; }; /** @description Bad request */ @@ -34399,7 +34858,7 @@ export interface operations { * "repo_variant": "fp16", * "upcast_attention": false * } */ - "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; + "application/json": components["schemas"]["Main_Diffusers_SD1_Config"] | components["schemas"]["Main_Diffusers_SD2_Config"] | components["schemas"]["Main_Diffusers_SDXL_Config"] | components["schemas"]["Main_Diffusers_SDXLRefiner_Config"] | components["schemas"]["Main_Diffusers_SD3_Config"] | components["schemas"]["Main_Diffusers_FLUX_Config"] | components["schemas"]["Main_Diffusers_Flux2_Config"] | components["schemas"]["Main_Diffusers_CogView4_Config"] | components["schemas"]["Main_Diffusers_QwenImage_Config"] | components["schemas"]["Main_Diffusers_ZImage_Config"] | components["schemas"]["Main_Checkpoint_SD1_Config"] | components["schemas"]["Main_Checkpoint_SD2_Config"] | components["schemas"]["Main_Checkpoint_SDXL_Config"] | components["schemas"]["Main_Checkpoint_SDXLRefiner_Config"] | components["schemas"]["Main_Checkpoint_Flux2_Config"] | components["schemas"]["Main_Checkpoint_FLUX_Config"] | components["schemas"]["Main_Checkpoint_ZImage_Config"] | components["schemas"]["Main_Checkpoint_Anima_Config"] | components["schemas"]["Main_BnBNF4_FLUX_Config"] | components["schemas"]["Main_GGUF_Flux2_Config"] | components["schemas"]["Main_GGUF_FLUX_Config"] | components["schemas"]["Main_GGUF_QwenImage_Config"] | components["schemas"]["Main_GGUF_ZImage_Config"] | components["schemas"]["Main_SDNQ_Diffusers_FLUX_Config"] | components["schemas"]["Main_SDNQ_Diffusers_Flux2_Config"] | components["schemas"]["Main_SDNQ_Diffusers_ZImage_Config"] | components["schemas"]["VAE_Checkpoint_SD1_Config"] | components["schemas"]["VAE_Checkpoint_SD2_Config"] | components["schemas"]["VAE_Checkpoint_SDXL_Config"] | components["schemas"]["VAE_Checkpoint_FLUX_Config"] | components["schemas"]["VAE_Checkpoint_Flux2_Config"] | components["schemas"]["VAE_Checkpoint_QwenImage_Config"] | components["schemas"]["VAE_Checkpoint_Anima_Config"] | components["schemas"]["VAE_Diffusers_SD1_Config"] | components["schemas"]["VAE_Diffusers_SDXL_Config"] | components["schemas"]["VAE_Diffusers_Flux2_Config"] | components["schemas"]["ControlNet_Checkpoint_SD1_Config"] | components["schemas"]["ControlNet_Checkpoint_SD2_Config"] | components["schemas"]["ControlNet_Checkpoint_SDXL_Config"] | components["schemas"]["ControlNet_Checkpoint_FLUX_Config"] | components["schemas"]["ControlNet_Checkpoint_ZImage_Config"] | components["schemas"]["ControlNet_Diffusers_SD1_Config"] | components["schemas"]["ControlNet_Diffusers_SD2_Config"] | components["schemas"]["ControlNet_Diffusers_SDXL_Config"] | components["schemas"]["ControlNet_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_SD1_Config"] | components["schemas"]["LoRA_LyCORIS_SD2_Config"] | components["schemas"]["LoRA_LyCORIS_SDXL_Config"] | components["schemas"]["LoRA_LyCORIS_Flux2_Config"] | components["schemas"]["LoRA_LyCORIS_FLUX_Config"] | components["schemas"]["LoRA_LyCORIS_ZImage_Config"] | components["schemas"]["LoRA_LyCORIS_QwenImage_Config"] | components["schemas"]["LoRA_LyCORIS_Anima_Config"] | components["schemas"]["LoRA_OMI_SDXL_Config"] | components["schemas"]["LoRA_OMI_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_SD1_Config"] | components["schemas"]["LoRA_Diffusers_SD2_Config"] | components["schemas"]["LoRA_Diffusers_SDXL_Config"] | components["schemas"]["LoRA_Diffusers_Flux2_Config"] | components["schemas"]["LoRA_Diffusers_FLUX_Config"] | components["schemas"]["LoRA_Diffusers_ZImage_Config"] | components["schemas"]["ControlLoRA_LyCORIS_FLUX_Config"] | components["schemas"]["T5Encoder_T5Encoder_Config"] | components["schemas"]["T5Encoder_BnBLLMint8_Config"] | components["schemas"]["T5Encoder_SDNQ_Config"] | components["schemas"]["Qwen3Encoder_Qwen3Encoder_Config"] | components["schemas"]["Qwen3Encoder_Checkpoint_Config"] | components["schemas"]["Qwen3Encoder_GGUF_Config"] | components["schemas"]["Qwen3Encoder_SDNQ_Folder_Config"] | components["schemas"]["QwenVLEncoder_Diffusers_Config"] | components["schemas"]["QwenVLEncoder_Checkpoint_Config"] | components["schemas"]["TI_File_SD1_Config"] | components["schemas"]["TI_File_SD2_Config"] | components["schemas"]["TI_File_SDXL_Config"] | components["schemas"]["TI_Folder_SD1_Config"] | components["schemas"]["TI_Folder_SD2_Config"] | components["schemas"]["TI_Folder_SDXL_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD1_Config"] | components["schemas"]["IPAdapter_InvokeAI_SD2_Config"] | components["schemas"]["IPAdapter_InvokeAI_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD1_Config"] | components["schemas"]["IPAdapter_Checkpoint_SD2_Config"] | components["schemas"]["IPAdapter_Checkpoint_SDXL_Config"] | components["schemas"]["IPAdapter_Checkpoint_FLUX_Config"] | components["schemas"]["T2IAdapter_Diffusers_SD1_Config"] | components["schemas"]["T2IAdapter_Diffusers_SDXL_Config"] | components["schemas"]["Spandrel_Checkpoint_Config"] | components["schemas"]["CLIPEmbed_Diffusers_G_Config"] | components["schemas"]["CLIPEmbed_Diffusers_L_Config"] | components["schemas"]["CLIPVision_Diffusers_Config"] | components["schemas"]["SigLIP_Diffusers_Config"] | components["schemas"]["FLUXRedux_Checkpoint_Config"] | components["schemas"]["LlavaOnevision_Diffusers_Config"] | components["schemas"]["TextLLM_Diffusers_Config"] | components["schemas"]["ExternalApiModelConfig"] | components["schemas"]["Unknown_Config"]; }; }; /** @description Bad request */ diff --git a/invokeai/frontend/web/src/services/api/types.ts b/invokeai/frontend/web/src/services/api/types.ts index 27c6fcbf3c3..d27c16b2733 100644 --- a/invokeai/frontend/web/src/services/api/types.ts +++ b/invokeai/frontend/web/src/services/api/types.ts @@ -475,11 +475,41 @@ export const isFluxFillMainModelModelConfig = (config: AnyModelConfig): config i }; export const isZImageDiffusersMainModelConfig = (config: AnyModelConfig): config is MainModelConfig => { - return config.type === 'main' && config.base === 'z-image' && config.format === 'diffusers'; + if (config.type !== 'main' || config.base !== 'z-image') { + return false; + } + // Read `format` and `submodels` as plain strings/unknown so TS doesn't narrow away the + // `sdnq_quantized` branch. The OpenAPI schema is regenerated separately and currently + // doesn't list the `sdnq_quantized` Z-Image format variant. + const format = (config as { format?: unknown }).format as string | undefined; + if (format === 'diffusers') { + return true; + } + // SDNQ-quantized ZImagePipeline folders carry the same submodels layout (transformer, vae, + // text_encoder, ...) as a plain diffusers ZImagePipeline. Single-file SDNQ Z-Image + // checkpoints have no submodels and must not match here. + if (format !== 'sdnq_quantized') { + return false; + } + const submodels = (config as { submodels?: unknown }).submodels; + return Boolean(submodels); }; export const isFlux2DiffusersMainModelConfig = (config: AnyModelConfig): config is MainModelConfig => { - return config.type === 'main' && config.base === 'flux2' && config.format === 'diffusers'; + if (config.type !== 'main' || config.base !== 'flux2') { + return false; + } + // Same reasoning as isZImageDiffusersMainModelConfig: an SDNQ FLUX.2 pipeline folder ships + // the same submodels (transformer/text_encoder/tokenizer/vae) and qualifies as a source model. + const format = (config as { format?: unknown }).format as string | undefined; + if (format === 'diffusers') { + return true; + } + if (format !== 'sdnq_quantized') { + return false; + } + const submodels = (config as { submodels?: unknown }).submodels; + return Boolean(submodels); }; export const isQwenImageDiffusersMainModelConfig = (config: AnyModelConfig): config is MainModelConfig => { diff --git a/tests/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/test_all_custom_modules.py b/tests/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/test_all_custom_modules.py index fba4ab98b52..a36c2567c0a 100644 --- a/tests/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/test_all_custom_modules.py +++ b/tests/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/test_all_custom_modules.py @@ -19,6 +19,8 @@ from invokeai.backend.patches.layers.lokr_layer import LoKRLayer from invokeai.backend.patches.layers.lora_layer import LoRALayer from invokeai.backend.patches.layers.merged_layer_patch import MergedLayerPatch, Range +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor +from invokeai.backend.quantization.sdnq.utils import SDNQQuantizationType from invokeai.backend.util.original_weights_storage import OriginalWeightsStorage from tests.backend.model_manager.load.model_cache.torch_module_autocast.custom_modules.test_custom_invoke_linear_8_bit_lt import ( build_linear_8bit_lt_layer, @@ -40,6 +42,31 @@ def build_linear_layer_with_ggml_quantized_tensor(orig_layer: torch.nn.Linear | return orig_layer +def build_linear_layer_with_sdnq_quantized_tensor(orig_layer: torch.nn.Linear | None = None): + """Wrap orig_layer's weight in an SDNQTensor (per-tensor symmetric int8). Bias stays unquantized, + which matches how SDNQ-quantized checkpoints are typically produced (only Linear weights are quantized).""" + if orig_layer is None: + orig_layer = torch.nn.Linear(32, 64) + + weight = orig_layer.weight.data + orig_dtype = weight.dtype + abs_max = weight.abs().max() + # Avoid div-by-zero on degenerate all-zero weights. + scale_value = (abs_max / 127.0).clamp(min=torch.finfo(torch.float32).tiny) + scale = scale_value.to(torch.float32).reshape(1) + quantized = (weight.float() / scale).round().clamp(-127, 127).to(torch.int8) + + sdnq_weight = SDNQTensor( + data=quantized, + quantization_type=SDNQQuantizationType.INT8_SYM, + tensor_shape=weight.shape, + compute_dtype=orig_dtype, + scale=scale, + ) + orig_layer.weight = torch.nn.Parameter(sdnq_weight, requires_grad=False) + return orig_layer + + parameterize_all_devices = pytest.mark.parametrize( ("device"), [ @@ -74,6 +101,7 @@ def build_linear_layer_with_ggml_quantized_tensor(orig_layer: torch.nn.Linear | "embedding", "flux_rms_norm", "linear_with_ggml_quantized_tensor", + "linear_with_sdnq_quantized_tensor", "invoke_linear_8_bit_lt", "invoke_linear_nf4", ] @@ -95,6 +123,8 @@ def layer_under_test(request: pytest.FixtureRequest) -> LayerUnderTest: return (RMSNorm(8), torch.randn(1, 8), True) elif layer_type == "linear_with_ggml_quantized_tensor": return (build_linear_layer_with_ggml_quantized_tensor(), torch.randn(1, 32), True) + elif layer_type == "linear_with_sdnq_quantized_tensor": + return (build_linear_layer_with_sdnq_quantized_tensor(), torch.randn(1, 32), True) elif layer_type == "invoke_linear_8_bit_lt": return (build_linear_8bit_lt_layer(), torch.randn(1, 32), False) elif layer_type == "invoke_linear_nf4": @@ -532,6 +562,7 @@ def test_linear_sidecar_patches_with_autocast_from_cpu_to_device(device: str, pa @pytest.fixture( params=[ "linear_ggml_quantized", + "linear_sdnq_quantized", "invoke_linear_8_bit_lt", "invoke_linear_nf4", ] @@ -544,6 +575,10 @@ def quantized_linear_layer_under_test(request: pytest.FixtureRequest): orig_layer = torch.nn.Linear(in_features, out_features) if layer_type == "linear_ggml_quantized": return orig_layer, build_linear_layer_with_ggml_quantized_tensor(orig_layer) + elif layer_type == "linear_sdnq_quantized": + # Re-build so SDNQ gets its own clean orig_layer (the helper modifies in place). + sdnq_layer = build_linear_layer_with_sdnq_quantized_tensor(copy.deepcopy(orig_layer)) + return orig_layer, sdnq_layer elif layer_type == "invoke_linear_8_bit_lt": return orig_layer, build_linear_8bit_lt_layer(orig_layer) elif layer_type == "invoke_linear_nf4": diff --git a/tests/backend/quantization/sdnq/__init__.py b/tests/backend/quantization/sdnq/__init__.py new file mode 100644 index 00000000000..c076f8e4427 --- /dev/null +++ b/tests/backend/quantization/sdnq/__init__.py @@ -0,0 +1 @@ +# Tests for SDNQ (SD.Next Quantization) support diff --git a/tests/backend/quantization/sdnq/test_sdnq_loader.py b/tests/backend/quantization/sdnq/test_sdnq_loader.py new file mode 100644 index 00000000000..0714258b1ec --- /dev/null +++ b/tests/backend/quantization/sdnq/test_sdnq_loader.py @@ -0,0 +1,155 @@ +"""Integration tests for SDNQ state dict loader.""" + +from pathlib import Path + +import pytest +import torch + +from invokeai.backend.quantization.sdnq.loaders import has_sdnq_keys, sdnq_sd_loader +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor + + +class TestSDNQLoader: + """Tests for SDNQ state dict loading.""" + + @pytest.fixture + def mock_sdnq_file(self, tmp_path: Path) -> Path: + """Create a mock SDNQ safetensors file.""" + from safetensors.torch import save_file + + # Create mock quantized weights with scale + tensors = { + "layer1.weight": torch.randint(-128, 127, (256, 256), dtype=torch.int8), + "layer1.scale": torch.tensor([0.01], dtype=torch.float32), + "layer1.bias": torch.randn(256, dtype=torch.float32), + "layer2.weight": torch.randint(-128, 127, (128, 256), dtype=torch.int8), + "layer2.scale": torch.tensor([0.02], dtype=torch.float32), + "layer2.zero_point": torch.tensor([0.5], dtype=torch.float32), + "norm.weight": torch.randn(256, dtype=torch.float32), # Not quantized + } + + file_path = tmp_path / "model.safetensors" + save_file(tensors, str(file_path)) + return file_path + + @pytest.fixture + def mock_sdnq_file_with_svd(self, tmp_path: Path) -> Path: + """Create a mock SDNQ safetensors file with SVD correction.""" + from safetensors.torch import save_file + + tensors = { + "layer.weight": torch.randint(-128, 127, (64, 64), dtype=torch.int8), + "layer.scale": torch.tensor([0.1], dtype=torch.float32), + "layer.svd_up": torch.randn(64, 16, dtype=torch.float32), + "layer.svd_down": torch.randn(16, 64, dtype=torch.float32), + } + + file_path = tmp_path / "model_svd.safetensors" + save_file(tensors, str(file_path)) + return file_path + + @pytest.fixture + def mock_non_sdnq_file(self, tmp_path: Path) -> Path: + """Create a mock non-SDNQ safetensors file (no scale keys).""" + from safetensors.torch import save_file + + tensors = { + "layer.weight": torch.randn(256, 256, dtype=torch.float32), + "layer.bias": torch.randn(256, dtype=torch.float32), + } + + file_path = tmp_path / "model_regular.safetensors" + save_file(tensors, str(file_path)) + return file_path + + def test_load_sdnq_file(self, mock_sdnq_file: Path): + """Test loading an SDNQ quantized file.""" + sd = sdnq_sd_loader(mock_sdnq_file) + + # Check that quantized weights are wrapped in SDNQTensor + assert "layer1.weight" in sd + assert isinstance(sd["layer1.weight"], SDNQTensor) + assert "layer2.weight" in sd + assert isinstance(sd["layer2.weight"], SDNQTensor) + + # Check that non-quantized tensors are regular tensors + assert "layer1.bias" in sd + assert isinstance(sd["layer1.bias"], torch.Tensor) + assert not isinstance(sd["layer1.bias"], SDNQTensor) + + assert "norm.weight" in sd + assert isinstance(sd["norm.weight"], torch.Tensor) + assert not isinstance(sd["norm.weight"], SDNQTensor) + + def test_symmetric_vs_asymmetric(self, mock_sdnq_file: Path): + """Test that symmetric and asymmetric quantization is detected.""" + sd = sdnq_sd_loader(mock_sdnq_file) + + # layer1 is symmetric (no zero_point) + layer1_weight = sd["layer1.weight"] + assert isinstance(layer1_weight, SDNQTensor) + assert not layer1_weight.is_asymmetric + + # layer2 is asymmetric (has zero_point) + layer2_weight = sd["layer2.weight"] + assert isinstance(layer2_weight, SDNQTensor) + assert layer2_weight.is_asymmetric + + def test_svd_loading(self, mock_sdnq_file_with_svd: Path): + """Test that SVD correction matrices are loaded.""" + sd = sdnq_sd_loader(mock_sdnq_file_with_svd) + + layer_weight = sd["layer.weight"] + assert isinstance(layer_weight, SDNQTensor) + assert layer_weight.has_svd + + def test_has_sdnq_keys_positive(self, mock_sdnq_file: Path): + """Test has_sdnq_keys returns True for SDNQ files.""" + from safetensors.torch import load_file + + sd = load_file(str(mock_sdnq_file)) + assert has_sdnq_keys(sd) + + def test_has_sdnq_keys_negative(self, mock_non_sdnq_file: Path): + """Test has_sdnq_keys returns False for non-SDNQ files.""" + from safetensors.torch import load_file + + sd = load_file(str(mock_non_sdnq_file)) + assert not has_sdnq_keys(sd) + + def test_compute_dtype_propagation(self, mock_sdnq_file: Path): + """Test that compute_dtype is set correctly on SDNQTensor.""" + sd = sdnq_sd_loader(mock_sdnq_file, compute_dtype=torch.bfloat16) + + layer1_weight = sd["layer1.weight"] + assert isinstance(layer1_weight, SDNQTensor) + assert layer1_weight.compute_dtype == torch.bfloat16 + + # Verify dequantization produces correct dtype + dequantized = layer1_weight.get_dequantized_tensor() + assert dequantized.dtype == torch.bfloat16 + + def test_dequantization_produces_correct_shape(self, mock_sdnq_file: Path): + """Test that dequantized tensors have correct shape.""" + sd = sdnq_sd_loader(mock_sdnq_file) + + layer1_weight = sd["layer1.weight"] + assert isinstance(layer1_weight, SDNQTensor) + assert layer1_weight.shape == torch.Size([256, 256]) + + dequantized = layer1_weight.get_dequantized_tensor() + assert dequantized.shape == torch.Size([256, 256]) + + def test_scale_keys_not_in_output(self, mock_sdnq_file: Path): + """Test that scale/zero_point keys are not in the output dict.""" + sd = sdnq_sd_loader(mock_sdnq_file) + + # Scale and zero_point should be absorbed into SDNQTensor + assert "layer1.scale" not in sd + assert "layer2.scale" not in sd + assert "layer2.zero_point" not in sd + + def test_empty_directory_raises(self, tmp_path: Path): + """Test that loading from empty directory raises ValueError.""" + with pytest.raises(ValueError, match="No safetensors files found"): + sdnq_sd_loader(tmp_path) diff --git a/tests/backend/quantization/sdnq/test_sdnq_tensor.py b/tests/backend/quantization/sdnq/test_sdnq_tensor.py new file mode 100644 index 00000000000..f9262cb13ef --- /dev/null +++ b/tests/backend/quantization/sdnq/test_sdnq_tensor.py @@ -0,0 +1,197 @@ +"""Unit tests for SDNQTensor class.""" + +import torch + +from invokeai.backend.quantization.sdnq.sdnq_tensor import SDNQTensor +from invokeai.backend.quantization.sdnq.utils import SDNQQuantizationType + + +class TestSDNQTensor: + """Tests for SDNQTensor dequantization and operations.""" + + def test_symmetric_dequantization(self): + """Test symmetric dequantization: result = weight * scale.""" + weight = torch.tensor([[1, 2], [3, 4]], dtype=torch.int8) + scale = torch.tensor([0.1], dtype=torch.float32) + + tensor = SDNQTensor( + data=weight, + quantization_type=SDNQQuantizationType.INT8_SYM, + tensor_shape=torch.Size([2, 2]), + compute_dtype=torch.float32, + scale=scale, + ) + + dequantized = tensor.get_dequantized_tensor() + expected = torch.tensor([[0.1, 0.2], [0.3, 0.4]], dtype=torch.float32) + assert torch.allclose(dequantized, expected, atol=1e-6) + + def test_asymmetric_dequantization(self): + """Test asymmetric dequantization: result = zero_point + weight * scale. + + Matches the SDNQ upstream convention (Disty0/sdnq: torch.addcmul(zero_point, weight, scale)), + where zero_point is a post-scale bias rather than a pre-scale integer offset. + """ + weight = torch.tensor([[10, 20], [30, 40]], dtype=torch.int8) + scale = torch.tensor([0.1], dtype=torch.float32) + zero_point = torch.tensor([5], dtype=torch.float32) + + tensor = SDNQTensor( + data=weight, + quantization_type=SDNQQuantizationType.INT8_ASYM, + tensor_shape=torch.Size([2, 2]), + compute_dtype=torch.float32, + scale=scale, + zero_point=zero_point, + ) + + dequantized = tensor.get_dequantized_tensor() + # zero_point + weight * scale = 5 + [[10, 20], [30, 40]] * 0.1 + expected = torch.tensor([[6.0, 7.0], [8.0, 9.0]], dtype=torch.float32) + assert torch.allclose(dequantized, expected, atol=1e-6) + + def test_svd_correction(self): + """Test SVD correction: result = dequant + svd_up @ svd_down.""" + weight = torch.tensor([[0, 0], [0, 0]], dtype=torch.int8) + scale = torch.tensor([1.0], dtype=torch.float32) + svd_up = torch.tensor([[1.0], [2.0]], dtype=torch.float32) + svd_down = torch.tensor([[1.0, 1.0]], dtype=torch.float32) + + tensor = SDNQTensor( + data=weight, + quantization_type=SDNQQuantizationType.INT8_SYM, + tensor_shape=torch.Size([2, 2]), + compute_dtype=torch.float32, + scale=scale, + svd_up=svd_up, + svd_down=svd_down, + ) + + assert tensor.has_svd + dequantized = tensor.get_dequantized_tensor() + # SVD correction: [[1], [2]] @ [[1, 1]] = [[1, 1], [2, 2]] + expected = torch.tensor([[1.0, 1.0], [2.0, 2.0]], dtype=torch.float32) + assert torch.allclose(dequantized, expected, atol=1e-6) + + def test_shape_properties(self): + """Test tensor shape properties.""" + weight = torch.tensor([[1, 2, 3], [4, 5, 6]], dtype=torch.int8) + scale = torch.tensor([0.1], dtype=torch.float32) + + tensor = SDNQTensor( + data=weight, + quantization_type=SDNQQuantizationType.INT8_SYM, + tensor_shape=torch.Size([2, 3]), + compute_dtype=torch.float32, + scale=scale, + ) + + assert tensor.shape == torch.Size([2, 3]) + assert tensor.size() == torch.Size([2, 3]) + assert tensor.size(0) == 2 + assert tensor.size(1) == 3 + assert tensor.quantized_shape == torch.Size([2, 3]) + + def test_is_asymmetric_property(self): + """Test is_asymmetric property.""" + weight = torch.tensor([[1]], dtype=torch.int8) + scale = torch.tensor([0.1], dtype=torch.float32) + + # Symmetric (no zero_point) + tensor_sym = SDNQTensor( + data=weight, + quantization_type=SDNQQuantizationType.INT8_SYM, + tensor_shape=torch.Size([1, 1]), + compute_dtype=torch.float32, + scale=scale, + ) + assert not tensor_sym.is_asymmetric + + # Asymmetric (has zero_point) + tensor_asym = SDNQTensor( + data=weight, + quantization_type=SDNQQuantizationType.INT8_ASYM, + tensor_shape=torch.Size([1, 1]), + compute_dtype=torch.float32, + scale=scale, + zero_point=torch.tensor([0.0]), + ) + assert tensor_asym.is_asymmetric + + def test_has_svd_property(self): + """Test has_svd property.""" + weight = torch.tensor([[1]], dtype=torch.int8) + scale = torch.tensor([0.1], dtype=torch.float32) + + # No SVD + tensor_no_svd = SDNQTensor( + data=weight, + quantization_type=SDNQQuantizationType.INT8_SYM, + tensor_shape=torch.Size([1, 1]), + compute_dtype=torch.float32, + scale=scale, + ) + assert not tensor_no_svd.has_svd + + # With SVD + tensor_with_svd = SDNQTensor( + data=weight, + quantization_type=SDNQQuantizationType.INT8_SYM, + tensor_shape=torch.Size([1, 1]), + compute_dtype=torch.float32, + scale=scale, + svd_up=torch.tensor([[1.0]]), + svd_down=torch.tensor([[1.0]]), + ) + assert tensor_with_svd.has_svd + + def test_repr(self): + """Test tensor string representation.""" + weight = torch.tensor([[1, 2], [3, 4]], dtype=torch.int8) + scale = torch.tensor([0.1], dtype=torch.float32) + + tensor = SDNQTensor( + data=weight, + quantization_type=SDNQQuantizationType.INT8_SYM, + tensor_shape=torch.Size([2, 2]), + compute_dtype=torch.float32, + scale=scale, + ) + + repr_str = repr(tensor) + assert "SDNQTensor" in repr_str + assert "int8_sym" in repr_str + assert "has_svd=False" in repr_str + + def test_compute_dtype_bfloat16(self): + """Test dequantization with bfloat16 compute dtype.""" + weight = torch.tensor([[1, 2], [3, 4]], dtype=torch.int8) + scale = torch.tensor([0.5], dtype=torch.float32) + + tensor = SDNQTensor( + data=weight, + quantization_type=SDNQQuantizationType.INT8_SYM, + tensor_shape=torch.Size([2, 2]), + compute_dtype=torch.bfloat16, + scale=scale, + ) + + dequantized = tensor.get_dequantized_tensor() + assert dequantized.dtype == torch.bfloat16 + + def test_requires_grad_noop(self): + """Test that requires_grad_ is a no-op for inference-only tensor.""" + weight = torch.tensor([[1]], dtype=torch.int8) + scale = torch.tensor([0.1], dtype=torch.float32) + + tensor = SDNQTensor( + data=weight, + quantization_type=SDNQQuantizationType.INT8_SYM, + tensor_shape=torch.Size([1, 1]), + compute_dtype=torch.float32, + scale=scale, + ) + + # Should return self without error + result = tensor.requires_grad_(True) + assert result is tensor