Generate pawn moves set-wise and write the captures straight into the move list by aywrite · Pull Request #371 · aywrite/arche

aywrite · 2026-10-04T19:26:34Z

Two exact changes to move generation, one commit each. Every list holds the same moves in the same order, so the search visits the same tree: every per position line of the bench and the games suite is unchanged (5,965,973 and 9,409,127 nodes).

Generate pawn moves set-wise. Shifts of the pawn board give the pawns with a capture and the pawns with an open push, and the generator visits only those, in square order. In the captures door 95% of the pawns it used to visit gave nothing.
Write the captures straight into the caller's move list. The cursor and the count stay in registers. The quiet moves go to one unchecked buffer, and one copy puts them behind the captures. The captures door writes its list once, in place.

Instructions. Callgrind, the tree search against master ec2abac: the games suite 11,388,195,036 to 11,077,306,805 (2.730% fewer), the bench 8,829,440,763 to 8,526,614,230 (3.430% fewer). By commit: 1.409% and 1.339% on the games suite. The second alone, on the old pawn walk, saved 0.19% (measured on adfd966): the two pay mostly together.

Clock. scripts/speed.sh on the head against master ec2abac: +4.1% (95% interval +2.2% to +6.0%, 120 interleaved rounds over shuffled layouts). An interleaved clock of the same code on adfd966 over 100 layouts read the games suite at +3.16% (+2.46% to +3.85%) over 450 rounds at a 16 MB table, and +2.50% (+1.49% to +3.55%) at 256 MB over 120. The clock reads more than the instructions because the pawn loop's branches go with it: 9.4% fewer mispredicted branches over the games suite under callgrind's simulation.

Games. Four Strength runs of 500 games at 10+0.1 on 8moves_v3 against master, run before the rebase (the head b7d96b3 against 587f1f6, the same two changes), the decision fixed before the first game: the mean rate ratio of the four and its 95% interval. The runs (#264 to #267) read 1.0201, 1.0182, 1.0194 and 1.0179, so r = 1.0189 [1.0172, 1.0206], wholly above 1.000. Every game ended normally. CI's speed job read that head at +3.3% (+2.9% to +3.6%) on the bench.

Checked (on the rebased head unless said otherwise).

On adfd966: a build that runs master's generator beside the new one at every call asserted the same list (every field, in order), the same capture count and the same has_legal_move answer over the bench, the games suite and a 448 search UCI workload ending in a 1 MB table.
On adfd966: a million random positions went through every door, among them lists over 64 moves, en passant into and out of check, promotions with captures, and pawns on the back ranks. All 7,063,989 distinct positions in traced searches matched an independent reference generator.
Miri passes all three doors and both growth paths under Stacked Borrows and Tree Borrows.
The release, debug, machine-test and baseline target tests pass, as do the tactical and strategic suites. Master and the head search 1,800 suite positions to depth 7 and 150 play-outs of up to 60 plies identically.
New tests pin the pawns' order in both doors, and a captures list that outgrows the inline list.

Constraints this leaves:

The quiet buffer is written without a bound check. It holds because a side has at most 63 pieces besides its king and no piece more than 27 quiet moves (1,728 moves, 10,368 bytes). A move kind that gives a piece more quiet moves has to resize it. A debug build checks every push.
The full and evasions doors' frame is 10.6 KB, two probed pages where it was one. A 512 move buffer for a side of at most sixteen pieces keeps it under a page; it was exact and not shown faster, so it is left out.
The list's order is the old one, which the ordering's tie breaks read: in each part, the pawns in square order, each pawn's captures before its en passant capture and its single push before its double push.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DtepPHhwporfDvjPSnX4ae

github-actions · 2026-10-04T19:31:33Z

Speed against `ec2abac4`

Measured on INTEL(R) XEON(R) PLATINUM 8573C.
Both sides built with rustc 1.98.1 (48a229cea 2026-09-01).

Both sides built and run on this runner in this job, the way
scripts/speed.sh measures a perf commit. Each round runs
both sides on a layout of its own, the same compiled code with
its code and data shuffled and moved, so the 95% interval carries
where the code landed as well as the run. The verdict holds the
interval against a 1% threshold. The default layout, the one a
release ships, is measured after as a diagnostic. The node
counts are the search's: they move when the search does, and a
speed change leaves them alone.

round     base nps  candidate nps  change
    1      5199073        5374580   +3.4%
    2      5164799        5407526   +4.7%
    3      5119249        5384063   +5.2%
    4      5172747        5416501   +4.7%
    5      5090711        5090455   -0.0%
    6      5046577        5174385   +2.5%
    7      5224885        5377899   +2.9%
    8      5160037        5287919   +2.5%
    9      5048268        5210543   +3.2%
   10      5235913        5387641   +2.9%
   11      5212560        5354672   +2.7%
   12      5024328        5363998   +6.8%
   13      5088874        5409301   +6.3%
   14      5223362        5339151   +2.2%
   15      5179286        5381178   +3.9%
   16      5018137        5276395   +5.1%
   17      5109682        5399046   +5.7%
   18      5264615        5441633   +3.4%
   19      5096704        5133715   +0.7%
   20      5136433        5345758   +4.1%
   21      5198552        5367835   +3.3%
   22      5247210        5409679   +3.1%
   23      5110185        5392867   +5.5%
   24      5122352        5521753   +7.8%
   25      5114921        5307441   +3.8%
   26      5220158        5266730   +0.9%
   27      5290634        5392828   +1.9%
   28      5166843        5377259   +4.1%
   29      5197845        5468443   +5.2%
   30      5213580        5423670   +4.0%
   31      5050935        5305935   +5.0%
   32      5085504        5156870   +1.4%
   33      5229658        5346361   +2.2%
   34      5187644        5445259   +5.0%
   35      5161898        5479850   +6.2%
   36      5196578        5408674   +4.1%
   37      5185678        5439148   +4.9%
   38      5110255        5413768   +5.9%
   39      5178355        5436907   +5.0%
   40      5239896        5437482   +3.8%

             nodes    time  median nps  faster half
base       5965973  1.15 s     5169795      5214911
candidate  5965973  1.11 s     5382620      5425802
change                           +4.1%        +4.0%

paired change +3.9%, 95% interval +3.3% to +4.5%
diagnostic, on the default layout alone +5.4%, 95% interval +4.6% to +6.3%, 13 rounds

faster: the whole interval is above +1.0%

Speed: +3.9% (bench nps, 95% interval +3.3% to +4.5%, 40 interleaved rounds over shuffled layouts vs ec2abac4)

The instructions each side's bench executed, counted under
cachegrind. The count repeats to within a few hundred
instructions, so a small change here is a real one, but it
prices instructions only: cache misses, mispredicted branches
and where the code lands are the speed's to show.

instructions against ec2abac4, one cachegrind run a side, bench

            instructions    nodes  per node
base       9,024,468,202  5965973    1512.7
candidate  8,761,766,177  5965973    1468.6
change            -2.91%             -2.91%

fewer instructions, beyond the ±0.7% that edits not made for speed have moved the count

The generator visited every pawn of the side to move and asked each one its rank, its two captures, its pushes and en passant. In the captures door most of them gave nothing: over the games suite 5,405,615 of the 5,687,761 pawns it walked there emitted no move. The pawns are now walked as sets. Once a call, shifts of the pawn board give the pawns with a capture and the pawns whose single or double push is open (in check, each push masked by the evasion targets on its own square, as before), and the pawn attack table read at the en passant square gives the pawns with an en passant capture. The generator walks the pawns with a capture from the lowest square, giving each its captures and then its en passant capture, and then the pawns with a push, giving each its single push and then its double push. A pawn with nothing to give is never visited. The list is the one the old walk gave, move for move. A pawn's captures and en passant already went to the captures part of the list and its pushes to the other part, so splitting one walk into two keeps each part's order: the pawns in square order, and each pawn's moves in the old order. The shifts drop the steps off the board that the old range test dropped, so a pawn on the first or last rank (which `from_fen` accepts) gives what it gave before. A new test pins the pawns' order in both doors against the lists the one pawn at a time walk gave, over promotions with and without a capture, en passant for either side, single and double pushes, and pawns on the back ranks. Callgrind, the tree search (`search_root` inclusive) on a build with line tables, against this commit's parent: the games suite 11,388,195,036 instructions to 11,227,678,882 (-1.409%), the bench 8,829,440,763 to 8,629,364,315 (-2.266%). Every per position line of both suites is unchanged. Bench: 5965973 Speed: +1.9% (bench nps, 95% interval +0.1% to +4.0%, 60 interleaved rounds over shuffled layouts vs ec2abac) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DtepPHhwporfDvjPSnX4ae

The generator wrote every move twice: into one of two 512 move arrays in its frame, and then into the caller's list when it finished. Each array's count lived beside it in the same struct, so llvm kept the counts in memory, and every push loaded, incremented and stored its count and checked it against 512. The captures now go straight into the caller's list through a cursor, with the count and the capacity in registers. A push that finds the list full grows it on a cold path, so a list of any width is still built. The quiet moves go into one buffer that no position can fill (a side has at most 63 pieces besides its king and no piece more than 27 moves), so their push is not checked, and one copy puts them behind the captures. With the pawns walked as sets, the captures door gives its promoting pushes after every capture, so it writes its list once, in place, with no buffer and no copy. The list is unchanged move for move; only where a move waits before the list is built has changed. The quiet buffer is 10,368 bytes, so the frame of the full and evasions doors grows from about 6.2 KB to 10.6 KB and takes a second probed page. A 512 move buffer for a side of sixteen pieces or fewer, with the full one on a cold path, keeps the frame under a page; it was exact but not shown faster on the clock (+0.37% over the games suite at 16 MB, -0.61% at 256 MB), so the frame stays. The move list's writes into its buffer before `set_len` were one unsafe block in `board.rs`. The generator now has six: taking the cursor from the list, the cursor's write, the quiet buffer's write, the copy behind the captures with its `set_len`, the other `set_len`, and the cold growth of the list. docs/ROADMAP.md counts them, sixteen in the crate. The cursor and the list are held as raw pointers: a list short of a spill keeps its moves inside itself, so a `&mut` to it reborrowed after the cursor was taken would invalidate the cursor. Miri passes a test of all three doors and both growth paths under Stacked Borrows and Tree Borrows, and reports undefined behaviour on the first form, which held the list as a `&mut`. A debug build checks every quiet push against the buffer's room, which is none in the captures door. A new test builds a captures list of more than 64 moves, so the cursor grows the list while it writes. Callgrind, against this commit's parent: the games suite 11,227,678,882 instructions to 11,077,306,805 (-1.339%; -2.730% for this commit and the last together), the bench 8,629,364,315 to 8,526,614,230 (-1.191%; -3.430% together). Writing the captures into the list with the pawns walked one at a time saved 0.19%, so the two changes pay mostly together. Every per position line of both suites is unchanged. Bench: 5965973 Speed: +2.1% (bench nps, 95% interval -0.9% to +4.8%, 60 interleaved rounds over shuffled layouts vs ccc1d04) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DtepPHhwporfDvjPSnX4ae

aywrite force-pushed the claude/movegen-setwise-pawns branch from b7d96b3 to 1ba0a8f Compare October 5, 2026 07:36

claude added 2 commits October 5, 2026 10:41

aywrite force-pushed the claude/movegen-setwise-pawns branch from 1ba0a8f to eba3587 Compare October 5, 2026 10:41

Navigation Menu

Generate pawn moves set-wise and write the captures straight into the move list - #371

Generate pawn moves set-wise and write the captures straight into the move list#371
aywrite wants to merge 2 commits into
masterfrom
claude/movegen-setwise-pawns

aywrite commented Oct 4, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 4, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

aywrite commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Speed against ec2abac4

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aywrite commented Oct 4, 2026 •

edited

Loading

github-actions Bot commented Oct 4, 2026 •

edited

Loading

Speed against `ec2abac4`