Skip to content

Refactor 2-cfg conditioning + better img-cfg defaults + support for DiT edit models - #929

Merged
leejet merged 7 commits into
leejet:masterfrom
stduhpf:2-cond
Jun 1, 2026
Merged

Refactor 2-cfg conditioning + better img-cfg defaults + support for DiT edit models#929
leejet merged 7 commits into
leejet:masterfrom
stduhpf:2-cond

Conversation

@stduhpf

@stduhpf stduhpf commented Oct 31, 2025

Copy link
Copy Markdown
Contributor

This helps make Inpaint or Pix2Pix models work nicely out of the box with cfg !=1 and unset img-cfg. The current implementation sets img_cfg = txt_cfg by default, but most inpaint/p2p models work best with img_cfg near 1.0, regardless of the txt_cfg value.

Should not affect the behavior of any other (non image conditionned) models hopefully.

While I'm at it, I added support for img-cfg in Flux Kontext and Qwen Image Edit models. It seems to have somewhat of an effect, though these models work just fine without it (especiallly Flux which doesn't handle any kind of cfg very well)

Also optimized sampling when txt_cfg==0. (Prompt is still processed in vain, but at least it's not used during sampling anymore)

@stduhpf
stduhpf marked this pull request as draft October 31, 2025 19:42
@stduhpf
stduhpf marked this pull request as ready for review October 31, 2025 21:29
@stduhpf
stduhpf marked this pull request as draft November 1, 2025 16:05
@stduhpf
stduhpf marked this pull request as ready for review November 1, 2025 16:26
@stduhpf stduhpf changed the title Refactor 2-cfg conditioning + better img-cfg defaults Refactor 2-cfg conditioning + better img-cfg defaults + support for DiT edit models Nov 1, 2025
@leejet

leejet commented Jun 1, 2026

Copy link
Copy Markdown
Owner

I kept the img-cfg default aligned with txt-cfg instead of changing it to 1.0, because changing the default guidance behavior would be too broad for existing users.

@leejet
leejet merged commit f8935d6 into leejet:master Jun 1, 2026
12 of 14 checks passed
@stduhpf

stduhpf commented Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

@leejet The reason I had set the default value for img-cfg to be 1 instead of the same as txt-cfg is because high values for img-cfg generally cause unwated behavior.

Examples:

Inpaint:

sd-cli.exe --model dreamshaper_8Inpainting.safetensors -p "a dog sitting on a bench" --color --steps 16 --cfg-scale 7 -i .\assets\bench.png --mask .\assets\bench_mask.png --strength 1
bench
bench_mask

--img-cfg-scale 1 --img-cfg-scale 7 (=txt-cfg)
output - Copy (260) output

Unet Edit:

sd-cli.exe --model instruct-pix2pix-00-22000-pruned-fp16.safetensors -p "Make it a cat" --color --strength 1 --steps 50 --scheduler karras --cfg-scale 7 -r "dog-on-bench.png"
output - Copy (23)

--img-cfg-scale 1 --img-cfg-scale 7 (=txt-cfg)
output output

@stduhpf

stduhpf commented Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

I feel like some of the changes you made to properly merge master into this branch reverted some meaningful changes of this PR. I will try to re-implement them

@stduhpf stduhpf mentioned this pull request Jun 1, 2026
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants