bench: remove the random-index FSL take benchmarks - #10010
Conversation
Merging this PR will improve performance by 28.99%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Simulation | take_fsl_f16_force_per_index[256, 10] |
56.8 µs | 38.7 µs | +46.74% |
| ⚡ | WallTime | filtered_owned_i64_avx512[OneNullInEight] |
26.1 µs | 23 µs | +13.39% |
| Simulation | fixed_16_advancing_ptr_safe[100] |
< 1 ns | < 1 ns | N/A | |
| Simulation | preverify_advancing_ptr_unchecked[1000] |
< 1 ns | < 1 ns | N/A | |
| Simulation | preverify_advancing_ptr_unchecked[10000] |
< 1 ns | < 1 ns | N/A |
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ct/beautiful-maxwell-7eyyvp (7f434f3) with develop (9e8abca)2
Footnotes
-
385 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
No successful run was found on
develop(515e61e) during the generation of this report, so 9e8abca was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
These took 10 or 100 indices from 500 lists, so the measured region copied a few kilobytes at most and nearly all of its cost was the allocation that the take and canonicalization perform. Each one reported two fixed values about 27 us apart and flipped between them on pull requests that changed no Vortex code, including a documentation-only one. Excluding allocator time from the shard in #9994 settled the dictionary decode and chunked take benchmarks but not these: they kept flipping, and take_fsl_u64_random[64, 10] fell to 0.6 us, the harness floor, which shows how little of it was ever Vortex work. The chunked and forced-strategy benchmarks in the same file still cover the FSL take paths, with deterministic index patterns. Signed-off-by: Claude <noreply@anthropic.com>
3a2ddb9 to
7f434f3
Compare
robert3005
left a comment
There was a problem hiding this comment.
I think this is me pushing benchmark guided optimisation too far. Thanks
Summary
These benchmarks took 10 or 100 indices from 500 lists, so the measured region copied a few
kilobytes at most and nearly all of its cost was allocation. Each reported two fixed values about
27 µs apart and flipped between them on pull requests that changed no Vortex code, including a
documentation-only one (#10007).
Excluding allocator time in #9994 settled dictionary decode but not these. They kept flipping, and
take_fsl_u64_random[64, 10]fell to 0.6 µs, the harness floor, which shows how little of it wasever Vortex work.
Changes
Remove
take_fsl_random, its typed and nullable variants, and the helpers and constants only theyused. The chunked and forced-strategy benchmarks still cover the FSL take paths with deterministic
index patterns.
The #9994 setting stays: dictionary decode is in another shard and still needs it, and this was the
only benchmark it hollowed out.