chunkers: stale kernel comments, explicit kernel dispatch, kernel benchmark script - #10308
Merged
ThomasWaldmann merged 3 commits intoSep 2, 2026
Merged
Conversation
- the avx2 kernels' double-buffer notes claimed the store-to-load forwarding stall "disappears"; performance counters on a Zen 4 show 4 failed forwards per block remain (the loads are wider than the stores, so they can only wait for the stores to commit). Say what the double buffer does and does not achieve, without the history. - fastcdc_impl.h said an unknown kernel id falls back to the vector kernel; it is the blockwise kernel (see the dispatch commit). - goldilocks: the branchless-reduction note is only true for gcc -O2 and clang; gcc -O3 turns the final compare-and-select back into a conditional jump (marked TODO). - phte_scan.h: the digest recomputed at a cut in the EVP path is not used by the chunker; say why it is still computed. - chunkers/__init__.py and phte_chunker.pyx: the kernel default is chosen per platform, not "the simplest implementation", and the AES chunkers' kernel property can also say 'vaes'. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rk table Drives `borg benchmark cpu --chunking --json` once per scan kernel combination and prints one throughput table over all chunkers and all kernels this build and CPU can run, which is how the per-platform kernel defaults were and are decided (borgbackup#10160). `borg benchmark cpu` reports a chunker it cannot construct - e.g. because an unusable kernel was requested - as an {"error": ...} row and carries on with exit code 0, so the script treats such a row as the run's failure; that is also how it discovers which kernels exist (an invalid name makes borg list the valid ones). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…se kernel fc_scan() and bz64_scan() ran the platform's vector kernel for any kernel id they did not know - on x86-64 that is the AVX2 kernel, which would SIGILL on a CPU without AVX2. Callers validate the id first, so this was latent; now every id has its case and anything else runs the portable blockwise kernel, and fc_kernel_name() / bz64_kernel_name() say so. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #10308 +/- ##
==========================================
+ Coverage 87.56% 87.70% +0.14%
==========================================
Files 103 103
Lines 18686 18712 +26
Branches 2875 2880 +5
==========================================
+ Hits 16362 16411 +49
+ Misses 1622 1600 -22
+ Partials 702 701 -1 ☔ View full report in Codecov by Harness. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hygiene follow-ups from the chunker kernel audit, one commit each, no change to cut points or (intended) behaviour:
-O2/clang (gcc-O3re-introduces a conditional jump, marked TODO), the EVP path's digest recompute at a cut is unused by the chunker,chunkers/__init__.pysaid kernels default to "the simplest implementation", and the AES chunkers'kerneldocstring lackedvaes.borg benchmark cpu --chunking --jsonreports a chunker it cannot construct as an{"error": ...}row with exit 0 (the previous version of the script found "no selectable kernels" against current master because of that).fc_scan()/bz64_scan()ran the platform's vector kernel for any unknown id (AVX2 on x86-64, which would SIGILL on a CPU without it). Callers validate ids, so it was latent; now every id has its case and unknown ids run the portable blockwise kernel, with*_kernel_name()saying so.Verified: chunker test suite on macOS (Apple clang 17), kernel selection unchanged, the script's smoke run works against this build.
🤖 Generated with Claude Code