perf: reduce PNG encoding time for high-entropy images#9346
Conversation
|
Is this something the user can change with a config value ? (did not look at the Code yet) |
It looks like the code only selects RLE after sampling the image and doing a test compression with RLE and the user has compression level at 1: if compress_level == 1 and _should_use_png_rle(image):
save_options["compress_type"] = zlib.Z_RLE |
|
The existing There is no independent RLE toggle while retaining level 1. I kept the strategy selection internal because it is a lossless encoding implementation detail and avoids adding another configuration setting. |
JPPhoto
left a comment
There was a problem hiding this comment.
A nice addition. My only concern is that there are a lot of hardcoded values (sample tile size, number of samples, etc.) that a user might want to tweak; perhaps those could be tunable in invokeai.yaml in a follow-up PR?
Summary
Reduce level-1 PNG encoding time for sufficiently large, high-entropy RGB/RGBA images.
The storage service now samples nine small native-resolution tiles and selects zlib's
Z_RLEstrategy only when the sample is not highly compressible and the sampled PNG stays within 2% of Pillow's default-strategy size. Other compression levels, smaller images, non-RGB modes, and samples that fail either check retain Pillow's existing behavior.The selector is necessary because applying
Z_RLEglobally made a structured gradient 1,562.7% larger, despite substantially improving high-entropy encoding time.Results
Seven alternating same-core pairs timed the complete
storage.save()call:That is a 22.05% reduction. The strict workload includes high-entropy RGB/RGBA, a structured gradient, line art, L, LA, palette transparency, and 16-bit images. Aggregate output grew by 0.105% only on deliberately random high-entropy fixtures; structured and non-RGB fixtures kept the default strategy.
Across nine repository images and Pillow 10.4, 11.3, and 12.2, aggregate encoding time improved by 14.68-14.99%. Seven high-entropy images were 10.88-22.36% faster and 2.29-10.52% smaller. The two structured fallback images had unchanged bytes; one paid a 0.91-1.10% selector overhead. No repository image became larger.
Absolute timings are hardware-sensitive, so all comparisons alternated baseline and candidate on the same core.
Supplementary autoresearch: 20-step public dashboard.
Related Issues / Discussions
Addresses #6597.
Open PR #9263 modifies the same import and cache regions. It does not implement PNG strategy selection, but this branch will need rebasing if that PR lands first.
QA Instructions
The strict evaluator times candidate
storage.save()calls and runs disk decode, cache eviction, and cleanup as untimed correctness gates. The before/after table comes from a separate alternating baseline/candidate harness.tests/app/services/image_files/test_image_files_disk.py: 24 passed on Python 3.11tests/app/services/image_files/test_image_files_disk.py: 24 passed on Python 3.121,L,LA,P,RGB,RGBA, andI;16, including alpha, palette transparency, and metadatagit diff --checkMerge Plan
Rebase if #9263 lands first and rerun the focused image-storage tests and strict evaluator.
Checklist
What's Newcopy (if doing a release after this PR) (not applicable)