Skip to content

Canonicalize chunked nested types through the builder [builders-child-stack] - #8967

Open
robert3005 wants to merge 1 commit into
claude/builders-bulk-loop-callers-9ze0t6from
claude/chunked-canonical-via-builder-9ze0t6
Open

Canonicalize chunked nested types through the builder [builders-child-stack]#8967
robert3005 wants to merge 1 commit into
claude/builders-bulk-loop-callers-9ze0t6from
claude/chunked-canonical-via-builder-9ze0t6

Conversation

@robert3005

@robert3005 robert3005 commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Always use builders when canonicalizing chunked arrays

@codspeed-hq

codspeed-hq Bot commented Jul 25, 2026

Copy link
Copy Markdown

Merging this PR will regress 10 benchmarks

⚡ 30 improved benchmarks
❌ 10 regressed benchmarks
✅ 1894 untouched benchmarks
⏩ 51 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation canonicalize[1024, 32] 55.9 µs 82.7 µs -32.35%
Simulation canonicalize[256, 32] 55.9 µs 81.8 µs -31.65%
Simulation canonicalize[1024, 8] 44.1 µs 62.6 µs -29.66%
Simulation canonicalize[16, 32] 58 µs 82.3 µs -29.5%
Simulation canonicalize[256, 8] 44 µs 61.9 µs -28.96%
Simulation canonicalize[1024, 2] 41.9 µs 57 µs -26.54%
Simulation canonicalize[16, 8] 47.3 µs 61.8 µs -23.46%
Simulation canonicalize[256, 2] 42.2 µs 54.5 µs -22.61%
Simulation canonicalize[16, 2] 53.7 µs 65.1 µs -17.49%
Simulation chunked_dict_primitive_into_canonical[u32, (1000, 10, 100)] 1.6 ms 1.8 ms -10.42%
Simulation take[small_m/shuffled/primitive/nonnull/chunks=16384/indices=1] 1,079.2 µs 74.4 µs ×15
Simulation take[value_shape/duplicate99/fsl8/nonnull/chunks=16/indices=10000] 36.6 ms 10.6 ms ×3.5
Simulation extend_from_array_zctl[(1000, 64)] 2,187.2 µs 656.4 µs ×3.3
Simulation take[value_shape/shuffled/fsl8/nonnull/chunks=16/indices=10000] 37.6 ms 11.8 ms ×3.2
Simulation extend_from_array_zctl[(10000, 8)] 3.6 ms 1.3 ms ×2.9
Simulation take_chunked_fsl_random[32, 64] 1,110.7 µs 514 µs ×2.2
Simulation take[small_m/shuffled/primitive/nonnull/chunks=1024/indices=1] 156.2 µs 74.1 µs ×2.1
Simulation take[core/sorted/primitive/nonnull/chunks=16/indices=100000] 29.8 ms 14.3 ms ×2.1
Simulation fsl_large 200.3 µs 106.6 µs +87.89%
Simulation take_chunked_fsl_random[16, 64] 863.5 µs 469.9 µs +83.76%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/chunked-canonical-via-builder-9ze0t6 (5f173bf) with claude/builders-bulk-loop-callers-9ze0t6 (64c3464)

Open in CodSpeed

Footnotes

  1. 51 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@claude claude Bot changed the title refactor(array): canonicalize chunked structs and FSLs through the builder refactor(array): canonicalize chunked structs and FSLs through the builder [builders-child-stack] Jul 25, 2026
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch 2 times, most recently from afed52f to 79f350a Compare July 29, 2026 07:35
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch 2 times, most recently from 1145029 to 8f39b06 Compare July 29, 2026 14:19
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 8f39b06 to 825ab2e Compare July 31, 2026 13:35
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 825ab2e to ce2eb1d Compare July 31, 2026 15:29
Comment thread vortex-array/src/arrays/chunked/vtable/canonical.rs Outdated
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from ce2eb1d to cefbc8e Compare July 31, 2026 16:54
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from cefbc8e to 0d4eb9a Compare July 31, 2026 17:22
@robert3005 robert3005 changed the title refactor(array): canonicalize chunked structs and FSLs through the builder [builders-child-stack] Canonicalize chunked nested types through the builder [builders-child-stack] Jul 31, 2026
@robert3005 robert3005 added the changelog/chore A trivial change label Jul 31, 2026
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 0d4eb9a to 1e8346a Compare July 31, 2026 18:39
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 1e8346a to 0431fa7 Compare July 31, 2026 18:43
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch 2 times, most recently from a9f7d31 to 9c8574c Compare July 31, 2026 22:45
@robert3005
robert3005 changed the base branch from claude/builders-lazy-validity-9ze0t6 to claude/constant-fast-paths-9ze0t6 July 31, 2026 22:47
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 9c8574c to 0b003e4 Compare August 1, 2026 09:43
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch 2 times, most recently from 876aa4e to a66422f Compare August 5, 2026 12:51
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from a66422f to dac04cb Compare August 7, 2026 10:40
@robert3005
robert3005 changed the base branch from claude/constant-fast-paths-9ze0t6 to claude/builders-bulk-loop-callers-9ze0t6 August 7, 2026 11:09
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from dac04cb to 3f93504 Compare August 7, 2026 11:14
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 3f93504 to 697f31a Compare August 7, 2026 13:10
`pack_struct_chunks`, `swizzle_fixed_size_list_chunks` and
`swizzle_list_chunks` existed because `append_to_builder` used to decode a
builder's children: without them, canonicalizing a `ChunkedArray` would
concatenate every chunk's children instead of reusing them. Their doc
comments describe what the builders now do on their own, so all three are
dead weight - the generic builder path produces the same swizzled array,
with each chunk's children kept as chunks of the combined child.

Only `Variant` still needs a hand-written pack, because there is no variant
builder.

This drops the one canonicalization path that allocated its buffers through
the session allocator: `swizzle_list_chunks` allocated its offsets and sizes
with `ctx.allocator()`, whereas `builder_with_capacity_in` still ignores the
allocator it is handed, so chunked primitives, structs and FSLs already went
around it. `list_canonicalize_uses_memory_session_allocator` guarded that
one path and goes with it; restoring the property means teaching the
builders to allocate through a `HostAllocatorRef`, not keeping a bespoke
list swizzle alive.

Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 697f31a to 5f173bf Compare August 7, 2026 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants