Skip to content

fix(archiver): avoid loading full blocks during checkpoint ingestion - #25279

Closed
spalladino wants to merge 1 commit into
merge-train/spartan-v5from
spl/faster-block-ingestion
Closed

spalladino wants to merge 1 commit into
merge-train/spartan-v5from
spl/faster-block-ingestion

Conversation

@spalladino

@spalladino spalladino commented Aug 21, 2026 •

Copy link
Copy Markdown
Contributor

Stack

Part of A-1817. Bottom to top, each PR targets the branch below it:

Context

The archiver ingests each checkpoint inside a single transactionAsync, so ingestion duration directly delays any read queued behind it (notably getLogsByTags, see #25254). Profiling showed the single biggest cost was not the write itself: getPreviousCheckpointBlock fetched the previous checkpoint's last block as a full L2Block — one sequential native LMDB round trip plus deserialization per tx effect — when its only consumer, validateCheckpointBlocks, reads four header-level fields.

Approach

  • getPreviousCheckpointBlock now reads the lightweight block data (as addProposedBlock already does) and returns just the chaining info (number, checkpointNumber, indexWithinCheckpoint, archive). L2Block satisfies the shape structurally, so the validation body, error types, and messages are unchanged.
  • LogStore.addLogs serialized the same txHash once per log and the same blockHash once per log in the whole block; both are now serialized once per tx / once per block and passed to encodeValue pre-serialized. The stored byte format is unchanged.

Measured A/B (per-checkpoint ingestion addCheckpoints + addLogs, LMDB on NVMe, mean of 10 checkpoints after warmup, two runs each):

txs/block before after delta
36 77.7 / 86.5 ms 42.3 / 41.1 ms ≈ −49%
120 248.1 / 244.0 ms 140.8 / 142.4 ms ≈ −42%

Checkpoint ingestion validated chaining by fetching the previous
checkpoint's last block as a full L2Block inside the write transaction,
deserializing every tx effect for four header fields. Read the
lightweight block data instead, and hoist the per-tx/per-block hash
serialization out of the log indexing loop. Cuts per-checkpoint
ingestion time by 49%/42% at 36/120 txs per block.
@spalladino
spalladino force-pushed the spl/faster-block-ingestion branch from 7ac21f1 to e68fc6b Compare August 26, 2026 19:09
@spalladino

Copy link
Copy Markdown
Contributor Author

Closing: this stack rotted after the repo split. The native read-only transaction support is being redone in aztec-packages (native-packages/kvdb), and the archiver/kv-store TS changes are being redone in aztec-node.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant