Skip to content

perf: reduce PNG encoding time for high-entropy images#9346

Open
dexhunter wants to merge 2 commits into
invoke-ai:mainfrom
dexhunter:dex/png-rle-selector
Open

perf: reduce PNG encoding time for high-entropy images#9346
dexhunter wants to merge 2 commits into
invoke-ai:mainfrom
dexhunter:dex/png-rle-selector

Conversation

@dexhunter

@dexhunter dexhunter commented Jul 10, 2026

Copy link
Copy Markdown

Summary

Reduce level-1 PNG encoding time for sufficiently large, high-entropy RGB/RGBA images.

The storage service now samples nine small native-resolution tiles and selects zlib's Z_RLE strategy only when the sample is not highly compressible and the sampled PNG stays within 2% of Pillow's default-strategy size. Other compression levels, smaller images, non-RGB modes, and samples that fail either check retain Pillow's existing behavior.

The selector is necessary because applying Z_RLE globally made a structured gradient 1,562.7% larger, despite substantially improving high-entropy encoding time.

Results

Seven alternating same-core pairs timed the complete storage.save() call:

Variant Median
Current main 328.658 ms
This change 256.179 ms

That is a 22.05% reduction. The strict workload includes high-entropy RGB/RGBA, a structured gradient, line art, L, LA, palette transparency, and 16-bit images. Aggregate output grew by 0.105% only on deliberately random high-entropy fixtures; structured and non-RGB fixtures kept the default strategy.

Across nine repository images and Pillow 10.4, 11.3, and 12.2, aggregate encoding time improved by 14.68-14.99%. Seven high-entropy images were 10.88-22.36% faster and 2.29-10.52% smaller. The two structured fallback images had unchanged bytes; one paid a 0.91-1.10% selector overhead. No repository image became larger.

Absolute timings are hardware-sensitive, so all comparisons alternated baseline and candidate on the same core.

Supplementary autoresearch: 20-step public dashboard.

Related Issues / Discussions

Addresses #6597.

Open PR #9263 modifies the same import and cache regions. It does not implement PNG strategy selection, but this branch will need rebasing if that PR lands first.

QA Instructions

The strict evaluator times candidate storage.save() calls and runs disk decode, cache eviction, and cleanup as untimed correctness gates. The before/after table comes from a separate alternating baseline/candidate harness.

  • tests/app/services/image_files/test_image_files_disk.py: 24 passed on Python 3.11
  • tests/app/services/image_files/test_image_files_disk.py: 24 passed on Python 3.12
  • Exact disk round trips for 1, L, LA, P, RGB, RGBA, and I;16, including alpha, palette transparency, and metadata
  • Ruff check and format
  • Targeted Pyright
  • git diff --check

Merge Plan

Rebase if #9263 lands first and rerun the focused image-storage tests and strict evaluator.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Changes to a redux slice have a corresponding migration (not applicable)
  • Documentation added / updated (if applicable) (not applicable)
  • Updated What's New copy (if doing a release after this PR) (not applicable)

@github-actions github-actions Bot added python PRs that change python files services PRs that change app services python-tests PRs that change python tests labels Jul 10, 2026
@Pfannkuchensack

Copy link
Copy Markdown
Collaborator

Is this something the user can change with a config value ? (did not look at the Code yet)

@JPPhoto JPPhoto assigned JPPhoto and unassigned JPPhoto Jul 10, 2026
@JPPhoto

JPPhoto commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Is this something the user can change with a config value ? (did not look at the Code yet)

It looks like the code only selects RLE after sampling the image and doing a test compression with RLE and the user has compression level at 1:

  if compress_level == 1 and _should_use_png_rle(image):
      save_options["compress_type"] = zlib.Z_RLE

@dexhunter

Copy link
Copy Markdown
Author

The existing pil_compress_level config gates this behavior. With its default value of 1, the selector samples the image and may use Z_RLE only when both size checks pass. Any other compression level keeps Pillow's existing strategy, so changing that value opts out of RLE, although it also changes the compression level.

There is no independent RLE toggle while retaining level 1. I kept the strategy selection internal because it is a lossless encoding implementation detail and avoids adding another configuration setting.

@lstein lstein added the 6.14.x label Jul 12, 2026
@lstein lstein moved this to 6.14.x Theme: USER EXPERIENCE in Invoke - Community Roadmap Jul 12, 2026

@JPPhoto JPPhoto left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nice addition. My only concern is that there are a lot of hardcoded values (sample tile size, number of samples, etc.) that a user might want to tweak; perhaps those could be tunable in invokeai.yaml in a follow-up PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.14.x python PRs that change python files python-tests PRs that change python tests services PRs that change app services

Projects

Status: 6.14.x Theme: USER EXPERIENCE

Development

Successfully merging this pull request may close these issues.

4 participants