Skip to content

perf: lazy SpecInfo JSON, SkipMetadataCollection, and hot-path allocation cuts#588

Merged
daveshanley merged 4 commits into
mainfrom
fable-power
Jun 12, 2026
Merged

perf: lazy SpecInfo JSON, SkipMetadataCollection, and hot-path allocation cuts#588
daveshanley merged 4 commits into
mainfrom
fable-power

Conversation

@daveshanley

@daveshanley daveshanley commented Jun 12, 2026

Copy link
Copy Markdown
Member

Build SpecInfo's JSON representation lazily via new GetSpecJSON/GetSpecJSONBytes/GetSpecJSONError accessors (deprecating direct field access), with parse-time structural validation (duplicate key detection, JSON syntax checks) preserved without the full eager conversion.

Add DocumentConfiguration.SkipMetadataCollection to skip collecting diagnostic index metadata (descriptions, summaries, enums, security requirement refs, inline schema Paths), cutting allocations and retained memory on large documents; propagated through v2/v3 document creation.

Reduce hot-path overhead: build oneOf/anyOf/allOf/prefixItems schema proxies inline instead of goroutine-per-item, make SchemaProxy's lock a value mutex, scan extensions directly instead of via FindExtensionNodes, and avoid strings.ToLower allocation for already-lowercase keys in BuildModel.

Includes golden/parity tests for component ID generation, inline ref collection, lazy JSON behavior, duplicate key detection, and metadata skipping.


A profiling-driven pass over the parse and index pipeline. Output — refs, definitions,
paths, hashes — is byte-identical to v0.37, enforced by checked-in golden fixtures.

  • -34% parse time, -50% memory, -38% allocations on large specs (5.8MB Stripe), with
    high-level model builds up to 58% faster. Downstream linting retains 21-26% less heap.
  • Lazy JSON view: SpecInfo.SpecJSON / SpecJSONBytes are now built on first use via
    the new GetSpecJSON(), GetSpecJSONBytes() and GetSpecJSONError() accessors. The
    fields remain but are deprecated and nil until an accessor runs — migrate direct reads.
  • New SkipMetadataCollection config flag: skips descriptions, summaries, enums and
    inline-schema path strings for model-only consumers. Worth ~11% time and 32MB on
    stripe-sized documents. Unsafe for diagnostic/rule tooling — linters must not enable it.
  • Behavior change: nested YAML construct errors (e.g. title: !!int nope) no longer
    fail ExtractSpecInfo — they surface from GetSpecJSONError(). Duplicate-key errors are
    unchanged and still fail eagerly, now reported once per definition.
  • Sturdier index internals: the node line index is a compact slice (no more racy
    partially-built map reads), GetNode waits for completion instead of returning early,
    and Release() no longer races concurrent readers.

If you read SpecJSON or SpecJSONBytes directly, switch to the accessors — your code
still compiles but will see nil. A version-agnostic pattern: assert the accessor interface
at runtime and fall back to the fields, so one build works against old and new libopenapi.

…tion cuts

Build SpecInfo's JSON representation lazily via new GetSpecJSON/GetSpecJSONBytes/GetSpecJSONError accessors (deprecating direct field access), with parse-time structural validation (duplicate key detection, JSON syntax checks) preserved without the full eager conversion.

Add DocumentConfiguration.SkipMetadataCollection to skip collecting diagnostic index metadata (descriptions, summaries, enums, security requirement refs, inline schema Paths), cutting allocations and retained memory on large documents; propagated through v2/v3 document creation.

Reduce hot-path overhead: build oneOf/anyOf/allOf/prefixItems schema proxies inline instead of goroutine-per-item, make SchemaProxy's lock a value mutex, scan extensions directly instead of via FindExtensionNodes, and avoid strings.ToLower allocation for already-lowercase keys in BuildModel.

Includes golden/parity tests for component ID generation, inline ref collection, lazy JSON behavior, duplicate key detection, and metadata skipping.
@blacksmith-sh

This comment has been minimized.

@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.74%. Comparing base (ef151b6) to head (1265c47).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #588      +/-   ##
==========================================
+ Coverage   99.72%   99.74%   +0.01%     
==========================================
  Files         280      280              
  Lines       33894    33977      +83     
==========================================
+ Hits        33802    33891      +89     
+ Misses         55       52       -3     
+ Partials       37       34       -3     
Flag Coverage Δ
unittests 99.74% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Replace the read-lock/build/upgrade dance in SpecIndex.GetNodeMap with a single exclusive lock, removing the double-checked rebuild path so concurrent callers never build the legacy map twice.

Also add a nil-receiver test for SchemaProxy.Schema().
@daveshanley daveshanley merged commit 6c72b83 into main Jun 12, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant