chunkers: build the arm64 hardware AES scan path without -march=+crypto - #10306
Merged
ThomasWaldmann merged 1 commit intoSep 2, 2026
Merged
Conversation
The aes-arm64 scan path of the toeplitz-aes, rabin-aes and goldilocks-aes
chunkers was only compiled when __ARM_FEATURE_AES was defined, i.e. when
the whole build targeted the crypto extension. Apple's default target
does, a Linux or BSD python extension build (-march=armv8-a baseline, no
-march flags from setup.py) does not: there the hardware path silently
did not exist, every such build ran the portable OpenSSL EVP path (1.5 to
1.9x slower per scanned byte on an Apple M3, 976 vs 529 MB/s for
toeplitz-aes on the same core under Linux) and BORG_AES_CHUNKER_KERNEL=
aes-arm64 was rejected with "not a kernel of this build" while the valid
values still listed it.
Enable the crypto extension per function with a target attribute (gcc >=
6, clang >= 14), exactly like the x86-64 path does with target("aes,sse2"),
and decide at run time whether the CPU has the instructions: getauxval on
Linux, elf_aux_info on FreeBSD, always on Apple Silicon, and only for
builds that target +crypto as a whole elsewhere. A compiler too old for
the attribute now reports aes-arm64 as "not compiled into this build"
instead of "not a kernel of this build".
Add a regression test: on aarch64 the hardware path must be the default
whenever the CPU has the AES instructions.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #10306 +/- ##
==========================================
+ Coverage 87.56% 87.67% +0.10%
==========================================
Files 103 103
Lines 18686 18712 +26
Branches 2875 2880 +5
==========================================
+ Hits 16362 16405 +43
+ Misses 1622 1606 -16
+ Partials 702 701 -1 ☔ View full report in Codecov by Harness. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
The
aes-arm64scan path of the toeplitz-aes / rabin-aes / goldilocks-aes chunkers was gated on__ARM_FEATURE_AES, which is only defined when the whole build targets the crypto extension. Apple's default target does; a Linux or BSD python extension build does not (gcc/clang-march=armv8-abaseline, and setup.py passes no-march). So on Linux arm64:BORG_AES_CHUNKER_KERNEL=aes-arm64was rejected with "not a kernel of this build" while the message still listed it as valid (this is what the RK3328 report in borg2: benchmarking needed #10160 ran into).The fix
__attribute__((target("+crypto")))(gcc >= 6, clang >= 14), the same way the x86-64 path usestarget("aes,sse2").getauxval(AT_HWCAP)on Linux (as before),elf_aux_infoon FreeBSD, always on Apple Silicon; elsewhere only a build that targets+cryptoas a whole may assume so (it would not run on a lesser CPU anyway), everything else stays on EVP. NetBSD/OpenBSD detection is left as a TODO.aes-arm64as "not compiled into this build" instead of "not a kernel of this build".Verification
__ARM_FEATURE_AESundefined): default kernel is nowaes-arm64,BORG_AES_CHUNKER_KERNEL=aes-arm64is accepted, all chunker tests pass (124 passed), and the hw/evp paths produce identical cut points. With the previous headers in the same build: defaultevp,aes-arm64rejected, the new test fails.PHTE_HW_TARGETempty), chunker tests pass. A kernel-only harness built with-O3 -march=armv8-a+noaes+nosha2(attribute path) runs at the same speed as the default build and is bit-identical.vaesdefault, chunker tests pass.elf_aux_info,AT_HWCAPandHWCAP_AEScome from<sys/auxv.h>(checked against the FreeBSD headers).🤖 Generated with Claude Code