Skip to content

Avoid per-node numeric type tuples in compact decoding - #980

Draft
bradhilton wants to merge 1 commit into
mainfrom
hayek/compact-decode-scalar-check-20260926
Draft

bradhilton wants to merge 1 commit into
mainfrom
hayek/compact-decode-scalar-check-20260926

Conversation

@bradhilton

Copy link
Copy Markdown
Collaborator

Compact decoding rebuilds a (bool, int, float) tuple at every JSON node. Replace it with direct int and float checks; bool remains covered by int. Container reconstruction, string references, validation errors and independent mutable results are unchanged.

Validation: 50 focused tests pass (23 new scalar/ownership/error cases plus the existing compact serialization suite), Ruff/format and scoped type checks pass, and uv lock --check --offline passes. An independently reviewed public fixture also passed 22 control groups, including the actual compressed Caladan dataset stream's order, slices, repeated/interleaved iteration and identity.

Public CPU benchmarks on CPython 3.12.13 used 13 randomized paired rounds. With cyclic GC outside the timed sections, compact validation was 14.3% faster and an eight-row compressed stream was 12.3% faster. The initial natural-GC measurements were noisy and showed only about 3% median gains. These are local synthetic measurements, not production latency or GPU-utilization claims. The fixtures use public ChatCompletion objects; no private inputs or GPU workload.

This draft is separate from the tokenizer and checkpoint-copy changes. It adds no cache, decoder API, offload or iteration change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant