Skip to content

Reduce allocations when decoding scalar and numeric results - #18

Merged
quinnj merged 2 commits into
mainfrom
perf/result-decoding
Sep 26, 2026
Merged

quinnj merged 2 commits into
mainfrom
perf/result-decoding

Conversation

@quinnj

@quinnj quinnj commented Sep 26, 2026 •

Copy link
Copy Markdown
Member

Postgres copies every non-null result field into a String before decoding it, and numeric values additionally pass through BigInt. This makes ordinary numeric queries allocate heavily even when their values fit the supported decimal type.

Decode fixed-width integers, floats, and booleans directly from bounded row-buffer views. Reuse DataDecimals' exact native parser for short fixed-point decimals with Parsers 3. The existing fallback preserves long coefficients, trailing-zero scale, exponent limits, special values, overflow policy, and Parsers 2 compatibility. Retained text, lazy JSON, and custom parsers still receive owned strings. Typed results retain custom style lifts and exact fixed-scale conversion.

Warmed, matched measurements on Julia 1.13.0 / PostgreSQL 16.15 (local arm64 server), 10,000-row prepared queries:

Query Median before → after Allocated bytes before → after
Numeric, ordinary results 40.92 → 12.10 ms 36.97 → 4.41 MB
Numeric, typed results 38.76 → 11.53 ms 36.24 → 3.68 MB
Five scalar columns, ordinary results 16.23 → 14.78 ms 6.41 → 4.53 MB
Five scalar columns, typed results 22.90 → 22.48 ms 7.06 → 5.17 MB

Numeric queries are about 3.4× faster with 88–90% fewer allocated bytes. Typed nondecimal query timing is effectively unchanged. The included benchmark/decoding.jl also measures decoder-only workloads and JSON/array controls; it reports timing spread and allocation counts separately from whole-query measurements.

Validation:

  • Full PostgreSQL integration suite: 8,006 checks pass on Julia 1.13.0 with Parsers 3.0.0; 7,980 pass on Julia 1.10.12 with Parsers 2.8.6. Both use two Julia threads and PostgreSQL 16.15.
  • Lowest supported library probe: 2,016 focused checks pass with Parsers 2.7.0, StructUtils 2.3.0, and JSON 1.0.0 on Julia 1.10.12.
  • Independent BigInt/rational oracles verify exact decimal coefficients, scales, overflow, malformed text, scientific notation, whitespace, and seeded boundary cases. Scalar checks use Base parsing. Ownership checks overwrite the original row buffer before inspecting retained text, custom-parser results, and lazy JSON. The same new checks also pass against the unchanged baseline.
  • Registered timestamp and DateTime overrides without custom parsers are checked through bounded fields and typed Any targets, including timezone offsets and microsecond precision: 154 focused decoding checks pass on Julia 1.13 and Julia 1.10 with the lowest supported libraries.
  • Actual JuliaC --trim=safe attempts have the same 20 existing verifier errors before and after: typed StructUtils calls through the dynamic type registry and default interval construction. This PR does not claim to make the driver trimmable. No protocol, buffering, dependency, export, or public API change is included.

Combined validation: the published SASLAuth #6 and #7 sources pass all 690 package checks together on Julia 1.9.4 and 1.13.0. Using that combined authentication source with the published Postgres #18 and #19 sources passes the full local integration suite: 8,375 checks on Julia 1.13.0 and 8,349 on 1.10.11. Production patches apply unchanged; only overlapping test-runner includes needed composition in the temporary fixture. Dependency versions and source hashes were verified. This adds compatibility evidence, not a new timing or native-compilation claim.

Co-authored by Codex

@quinnj
quinnj merged commit 9015740 into main Sep 26, 2026
16 checks passed
quinnj referenced this pull request Sep 28, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant