Skip to content

Slab: reap only the wavefronts handed out since the last reap (port of C #140) - #58

Merged
cjfields merged 1 commit into
mainfrom
perf/slab-busy-reap
Oct 18, 2026
Merged

cjfields merged 1 commit into
mainfrom
perf/slab-busy-reap

Conversation

@cjfields

Copy link
Copy Markdown
Member

Ports Marco-Sola-Lab/WFA2-lib#140 (cda19a0, already in the pinned C commit). I had listed it as "speed only, n/a", but Rust had the same problem, and worse. Relevant to #31 and to dada2-rs, whose per-thread aligner sees mixed read lengths in high memory.

Problem

In high memory (CIGAR), init_alignment calls WavefrontSlab::reap_cigar. It walked every wavefront the slab had ever created, including deallocated ones. A long alignment leaves thousands of them, so every short alignment that followed paid for the whole walk. sample put about 70% of a mixed run in reap_cigar.

Fix

  • The slab records the indices it hands out (busy).
  • When there are no mixed lengths (mixed_lengths, C's dirty), reap_cigar returns only those indices. That's safe because every other live wavefront is already on the free list.
  • Recording costs about 2% on uniform short workloads, so it's done only while the slab has at least 4096 wavefronts, which happens only after long or divergent alignments. A busy set of a quarter of the slab or more still takes the ordered full walk.
  • A debug_assert covers the invariant: a deallocation implies mixed lengths.

Results

M2 Pro, median of 7, alignment time in ms, on C's benchmark data (10 × 10 kb then 200k × 100 bp, 5% error, gap-affine):

Data Mode C before #140 C after #140 Rust main This PR
mixed high 700 421 1231 365 (−70%)
mixed med 774 782 651 653
short only high 362 376 322 318
short only med 609 619 565 556

Interleaved workload (each 10 kb pair followed by 300 × 100 bp), median of 5:

  • dada2-like (-5,4,0,8, ends-free 0.1, banded ±16, corrected): 735 → 209 ms;
  • plain gap-affine: 56 → 45 ms.

Validation

  • Output: byte-identical to main on the interleaved set for every metric, in high/med/ultralow and score-only, plus corrected ends-free (affine and linear, banded) and X-drop.
  • Differential suite:
  • Tests: new unit test test_reap_cigar_busy_after_large_slab. Tests pass in release and debug, so the debug_assert ran; clippy is clean.

PERFORMANCE_LOG and PROGRESS are updated; C #140 is now marked as ported.

🤖 Generated with Claude Code

Port of Marco-Sola-Lab/WFA2-lib#140. In high memory, reap_cigar walked
every wavefront the slab had ever created before each alignment. After a
long alignment that is thousands, so every following short alignment paid
for it (~70% of a mixed run's samples).

The slab now records the indices it hands out (busy) and, with no mixed
lengths (C's dirty), returns only those. Recording is done only while
the slab has >= 4096 wavefronts, so uniform short workloads don't pay for
it, and a busy set of a quarter of the slab or more still takes the
ordered full walk.

C's #140 benchmark (10 x 10 kb then 200k x 100 bp, high): 1231 -> 365 ms
(C after #140: 421 ms). A dada2-like interleaved workload: 735 -> 209 ms.
Output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cjfields
cjfields merged commit bef359a into main Oct 18, 2026
17 checks passed
@cjfields
cjfields deleted the perf/slab-busy-reap branch October 18, 2026 04:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant