Repository navigation
[Feature] FLUX.3 Image Klein support #2142
Description
Feature Summary
Support for FLUX.3 Image Klein (Black Forest Labs, released 2026-10-13), the successor of FLUX.2 klein.
Detailed Description
FLUX.3 Image Klein (HF, gated) is a distilled text-to-image and editing model. Most of it looks reusable from the existing FLUX.2 support. From the checkpoint header and config.json:
| Part | FLUX.3 Image Klein | Compared with FLUX.2 klein |
|---|---|---|
| VAE | AutoencoderKLFlux2 |
same |
| Text encoder | Qwen3-VL-4B, hidden states of 8 layers (4, 8, …, 32) concatenated → txt_in [3072, 20480] |
Qwen3-VL is already supported; new layer set |
| Single blocks | 28, hidden 3072, 24 heads, mlp_ratio 3 | same layout, but q_proj/k_proj/v_proj are separate instead of fused |
| Double blocks | 5 content_mode_blocks with three streams with their own weights: image, image_cond (reference image) and txt_mode_blocks, plus per-stream modulations (early_stream_modulations.*, single_stream_modulations.*) |
new: FLUX.2 has two streams and appends reference images to the image tokens |
| Timestep | time_in and ref_time_in: the distilled model is a flow map and also receives the timestep it jumps to (next sigma, 0 after the last step) |
new input; no CFG |
| RoPE | 4 axes [32, 32, 32, 32], theta 10000, image_rope_scale 0.5 |
theta 2000 in FLUX.2 |
| Sampling | 16 steps, scheduler shift 6.93 (base model: 50 steps, guidance 3.5) |
The reference implementation was merged in diffusers in huggingface/diffusers#15046 (transformer_flux3.py, modular_pipelines/flux3/, and scripts/convert_flux3_to_diffusers.py for the weight mapping).
Alternatives you considered
No response
Additional context
I write measured recipes for running image models on an RTX 3050 6 GB (Qwen-Image 2.1 and Qwen-Image 2.1 Turbo with sd.cpp: https://github.com/ruben-salas20/recipes). I'd be happy to test a branch on that card and report speed, memory and output quality against diffusers. The model license (flux-kommunity-license) may affect redistributing GGUF files; I haven't been able to check it yet because the repo is gated.
Activity
gtotaro86-ai commentedon Oct 15, 2026
Some details that may help with the port. Everything below is read from the checkpoint header (model.safetensors of flux-3-image-klein, 359 tensors, 5.549 B parameters, all BF16, no biases) and from BFL's reference code (black-forest-labs/flux-3-image-klein, src/flux3/dit.py, sampling.py, pipeline.py, text_encoder.py, Apache-2.0). I did not run anything yet, so treat the numbers as read, not verified by a parity test.
Tensor names (distilled checkpoint)
emb_in.{image,image_cond}.weight [3072, 128], txt_in.weight [3072, 20480]
time_in.{in_layer,out_layer}.weight [3072,256] / [3072,3072]; ref_time_in.* (same shapes, flow-map checkpoint only)
early_stream_modulations.{image,image_cond,txt}.lin.weight and single_stream_modulations.{image,image_cond,txt}.lin.weight, each [9216, 3072] (= 3 x 3072: shift, scale, gate)
content_mode_blocks.{image,image_cond}.{0..4}., txt_mode_blocks.{0..4}., single_blocks.{0..27}.*, each block: q_proj, k_proj, v_proj [3072,3072], attn_out [3072,3072], mlp_in [18432,3072], mlp_out [3072,9216], norm.query_norm.scale, norm.key_norm.scale [128]
final_layer.image.adaLN_modulation.1.weight [6144,3072], final_layer.image.linear.weight [128,3072]
Structure (from dit.py)
Three token modes: image (128 channels), image_cond (reference image tokens, 128 channels) and text (20480 channels).
The 5 first blocks are per mode: each mode runs through its own weights and attends only to its own tokens of the same sample (no cross-mode attention there). Modulation is the mode's early_stream_modulations.
The 28 single blocks are shared: per sample the tokens are concatenated in the order image, image_cond, text and attend to each other; each token's shift/scale/gate comes from single_stream_modulations of its mode.
Block: x_mod = (1 + scale) * LayerNorm(x) + shift (no affine, eps 1e-6); q, k, v separate projections; per-head RMSNorm (eps 1e-6, scale over head_dim 128) on q and k, then RoPE; parallel attention and SwiGLU MLP: x + gate * (attn_out(attn) + mlp_out(silu(a) * b)) with a, b = mlp_in(x_mod).chunk(2).
24 heads, head_dim 128, RoPE with 4 axes of 32 each, theta 10000. Ids are 4-vectors: text (0, 0, 0, position), image (0, y, x, 0) with axes 1 and 2 multiplied by 0.5 (IMAGE_ROPE_SCALE), references additionally l = -(1 + index) on the last axis.
Time: timestep_embedding(t) = [cos, sin] of 1000 * t * freqs (dim 256, max_period 10000) into MLPEmbedder (Linear, SiLU, Linear). The image tokens carry the current t, the other modes carry 0. For the flow-map checkpoint vec += ref_time_in(timestep_embedding(t_next)), added to every mode of that sample.
Output: only the image tokens, LayerNorm with shift/scale from adaLN_modulation (SiLU, Linear to 2 x 3072), then linear to 128 channels (velocity).
Sampling (sampling.py, pipeline.py)
Schedule: t = linspace(1, 0, steps + 1), sigma = alpha * t / (1 + (alpha - 1) * t) with alpha = 6.93 (default).
Distilled / flow map: 16 steps, no CFG, one sample; at step k the model is called with timesteps = sigma_k and timesteps_ref = sigma_{k+1}; update x += (sigma_{k+1} - sigma_k) * v.
Base model: 50 steps, CFG 3.5 on a two-sample pack [empty prompt, prompt], same Euler update, no ref_time_in.
Latents: AutoencoderKLFlux2 (checkpoint ae.safetensors, same file as in the FLUX.2 repos), 16x spatial downsampling, 128 channels per token.
Text encoder (text_encoder.py)
Qwen3-VL-4B-Instruct, chat template with the prompt as one user message and add_generation_prompt, right-padded to a multiple of 80 tokens (max 4096); the pad positions are passed on to the DiT as tokens.
Hidden states after layers 4, 8, 12, 16, 20, 24, 28, 32 (output_hidden_states indices), concatenated along the feature axis (layer-major) to 8 x 2560 = 20480 channels.
All matrices have dimensions divisible by 256 (3072, 9216, 18432, 20480), so K-quants should apply without special handling; the 128-wide norm scales are better kept in higher precision.
Happy to test a branch on a Strix Halo (gfx1151, Vulkan) and with Q8_0/Q4_K if that helps.
iamwavecut commentedon Oct 15, 2026
main README states day-zero support of Flux3.
ruben-salas20 commentedon Oct 15, 2026
Thanks @leejet for adding it so fast in #2143, and @gtotaro86-ai for the detailed notes. Closing as resolved. I'll test it on an RTX 3050 6 GB and share speed and memory numbers in the recipes repo.