Skip to content

bench: remove the random-index FSL take benchmarks - #10010

Merged
connortsui20 merged 1 commit into
developfrom
ct/beautiful-maxwell-7eyyvp
Sep 23, 2026
Merged

connortsui20 merged 1 commit into
developfrom
ct/beautiful-maxwell-7eyyvp

Conversation

@connortsui20

@connortsui20 connortsui20 commented Sep 23, 2026

Copy link
Copy Markdown
Member

Summary

These benchmarks took 10 or 100 indices from 500 lists, so the measured region copied a few
kilobytes at most and nearly all of its cost was allocation. Each reported two fixed values about
27 µs apart and flipped between them on pull requests that changed no Vortex code, including a
documentation-only one (#10007).

Excluding allocator time in #9994 settled dictionary decode but not these. They kept flipping, and
take_fsl_u64_random[64, 10] fell to 0.6 µs, the harness floor, which shows how little of it was
ever Vortex work.

Changes

Remove take_fsl_random, its typed and nullable variants, and the helpers and constants only they
used. The chunked and forced-strategy benchmarks still cover the FSL take paths with deterministic
index patterns.

The #9994 setting stays: dictionary decode is in another shard and still needs it, and this was the
only benchmark it hollowed out.

@codspeed

codspeed Bot commented Sep 23, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 28.99%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 3 benchmarks measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. These results are not comparable, so they count as unchanged.

Preventing compiler optimizations

⚡ 2 improved benchmarks
✅ 2174 untouched benchmarks
⏩ 385 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation take_fsl_f16_force_per_index[256, 10] 56.8 µs 38.7 µs +46.74%
WallTime filtered_owned_i64_avx512[OneNullInEight] 26.1 µs 23 µs +13.39%
⚠️ Simulation fixed_16_advancing_ptr_safe[100] < 1 ns < 1 ns N/A
⚠️ Simulation preverify_advancing_ptr_unchecked[1000] < 1 ns < 1 ns N/A
⚠️ Simulation preverify_advancing_ptr_unchecked[10000] < 1 ns < 1 ns N/A

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/beautiful-maxwell-7eyyvp (7f434f3) with develop (9e8abca)2

Open in CodSpeed

Footnotes

  1. 385 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on develop (515e61e) during the generation of this report, so 9e8abca was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

These took 10 or 100 indices from 500 lists, so the measured region copied
a few kilobytes at most and nearly all of its cost was the allocation that
the take and canonicalization perform. Each one reported two fixed values
about 27 us apart and flipped between them on pull requests that changed
no Vortex code, including a documentation-only one.

Excluding allocator time from the shard in #9994 settled the dictionary
decode and chunked take benchmarks but not these: they kept flipping, and
take_fsl_u64_random[64, 10] fell to 0.6 us, the harness floor, which shows
how little of it was ever Vortex work.

The chunked and forced-strategy benchmarks in the same file still cover
the FSL take paths, with deterministic index patterns.

Signed-off-by: Claude <noreply@anthropic.com>
@connortsui20
connortsui20 force-pushed the ct/beautiful-maxwell-7eyyvp branch from 3a2ddb9 to 7f434f3 Compare September 23, 2026 19:16
@connortsui20 connortsui20 changed the title bench: keep the random-index FSL take benchmarks out of CodSpeed bench: remove the random-index FSL take benchmarks Sep 23, 2026

@robert3005 robert3005 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is me pushing benchmark guided optimisation too far. Thanks

@connortsui20
connortsui20 merged commit 2881302 into develop Sep 23, 2026
128 of 130 checks passed
@connortsui20
connortsui20 deleted the ct/beautiful-maxwell-7eyyvp branch September 23, 2026 21:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants