Skip to content

chunkers: build the arm64 hardware AES scan path without -march=+crypto - #10306

Merged
ThomasWaldmann merged 1 commit into
borgbackup:masterfrom
ThomasWaldmann:fix-arm64-hw-aes-chunkers
Sep 2, 2026
Merged

chunkers: build the arm64 hardware AES scan path without -march=+crypto#10306
ThomasWaldmann merged 1 commit into
borgbackup:masterfrom
ThomasWaldmann:fix-arm64-hw-aes-chunkers

Conversation

@ThomasWaldmann

Copy link
Copy Markdown
Member

The bug

The aes-arm64 scan path of the toeplitz-aes / rabin-aes / goldilocks-aes chunkers was gated on __ARM_FEATURE_AES, which is only defined when the whole build targets the crypto extension. Apple's default target does; a Linux or BSD python extension build does not (gcc/clang -march=armv8-a baseline, and setup.py passes no -march). So on Linux arm64:

  • the hardware path was never compiled and every build silently ran the portable OpenSSL EVP path (measured 1.5-1.9x slower per scanned byte on an Apple M3; under Linux on the same core: toeplitz-aes 976 MB/s with the fix vs 529 MB/s before),
  • BORG_AES_CHUNKER_KERNEL=aes-arm64 was rejected with "not a kernel of this build" while the message still listed it as valid (this is what the RK3328 report in borg2: benchmarking needed #10160 ran into).

The fix

  • Enable the crypto extension per function with __attribute__((target("+crypto"))) (gcc >= 6, clang >= 14), the same way the x86-64 path uses target("aes,sse2").
  • Decide at run time whether the CPU has the instructions: getauxval(AT_HWCAP) on Linux (as before), elf_aux_info on FreeBSD, always on Apple Silicon; elsewhere only a build that targets +crypto as a whole may assume so (it would not run on a lesser CPU anyway), everything else stays on EVP. NetBSD/OpenBSD detection is left as a TODO.
  • A compiler too old for the attribute now reports aes-arm64 as "not compiled into this build" instead of "not a kernel of this build".
  • Regression test: on aarch64 the hardware path must be the default whenever the CPU has AES instructions.

Verification

  • Linux arm64 (Debian 13 image, gcc 14.2, default flags, __ARM_FEATURE_AES undefined): default kernel is now aes-arm64, BORG_AES_CHUNKER_KERNEL=aes-arm64 is accepted, all chunker tests pass (124 passed), and the hw/evp paths produce identical cut points. With the previous headers in the same build: default evp, aes-arm64 rejected, the new test fails.
  • macOS (Apple clang 17): default build unchanged (attribute not needed, PHTE_HW_TARGET empty), chunker tests pass. A kernel-only harness built with -O3 -march=armv8-a+noaes+nosha2 (attribute path) runs at the same speed as the default build and is bit-identical.
  • x86-64 (Zen 4, gcc 14): unaffected, vaes default, chunker tests pass.
  • FreeBSD arm64 could not be tested here; elf_aux_info, AT_HWCAP and HWCAP_AES come from <sys/auxv.h> (checked against the FreeBSD headers).

🤖 Generated with Claude Code

The aes-arm64 scan path of the toeplitz-aes, rabin-aes and goldilocks-aes
chunkers was only compiled when __ARM_FEATURE_AES was defined, i.e. when
the whole build targeted the crypto extension. Apple's default target
does, a Linux or BSD python extension build (-march=armv8-a baseline, no
-march flags from setup.py) does not: there the hardware path silently
did not exist, every such build ran the portable OpenSSL EVP path (1.5 to
1.9x slower per scanned byte on an Apple M3, 976 vs 529 MB/s for
toeplitz-aes on the same core under Linux) and BORG_AES_CHUNKER_KERNEL=
aes-arm64 was rejected with "not a kernel of this build" while the valid
values still listed it.

Enable the crypto extension per function with a target attribute (gcc >=
6, clang >= 14), exactly like the x86-64 path does with target("aes,sse2"),
and decide at run time whether the CPU has the instructions: getauxval on
Linux, elf_aux_info on FreeBSD, always on Apple Silicon, and only for
builds that target +crypto as a whole elsewhere. A compiler too old for
the attribute now reports aes-arm64 as "not compiled into this build"
instead of "not a kernel of this build".

Add a regression test: on aarch64 the hardware path must be the default
whenever the CPU has the AES instructions.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.67%. Comparing base (71dc70e) to head (e46872a).
⚠️ Report is 15 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10306      +/-   ##
==========================================
+ Coverage   87.56%   87.67%   +0.10%     
==========================================
  Files         103      103              
  Lines       18686    18712      +26     
  Branches     2875     2880       +5     
==========================================
+ Hits        16362    16405      +43     
+ Misses       1622     1606      -16     
+ Partials      702      701       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@ThomasWaldmann
ThomasWaldmann merged commit c4a35a3 into borgbackup:master Sep 2, 2026
42 of 43 checks passed
@ThomasWaldmann
ThomasWaldmann deleted the fix-arm64-hw-aes-chunkers branch September 2, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant