perf(engine): decode payload transactions once - #1
Open
gr8h wants to merge 5 commits into
Open
Conversation
Ref paradigmxyz#20431 (item 3) Payload transactions were RLP-decoded twice per newPayload: once in the conversion thread and again in the execution-side tx iterator. The iterator is now the single decoder: it streams each decoded transaction over a bounded channel (capacity = tx count, sends never block) into payload conversion, which assembles the block body from the stream and returns the transactions root computed from the raw payload bytes, skipping the per-tx re-encode in the pre-execution root check. Opt-in via two defaulted PayloadValidator methods; non-Ethereum implementations keep the existing double-decode behavior unchanged. If the stream disconnects before all transactions arrive (malformed tx, aborted execution, early return), conversion falls back to decoding the retained raw bytes, reproducing the existing errors. The block hash is now validated before transaction decoding (engine-API step 1); hive engine-api showed zero delta for this reordering. Deviations from the handoff spec (.scratch/decode-once/spec.md): the sender is also dropped in ensure_ok!/early-return paths because LazyHandle::try_into_inner blocks (spec assumed scope-drop suffices); the tx-root-mismatch test is uncraftable since V1 payloads carry no root field (root is derived from the raw bytes), so root correctness is locked by a seam test instead. Claude-Session: https://claude.ai/code/session_01BaDayhJk8crEcAQFxmvQF4
Encode the drop-sender-before-join invariant in a PendingValidatedBlock wrapper instead of three manual drops, define the body tx sender alias via the canonical PayloadTxStreamSender, and deduplicate the fork-field checks between the streaming and non-streaming payload validators. Claude-Session: https://claude.ai/code/session_01H8bWdbHjv1fuUGgyR3TZcT
The streaming payload conversion returned the header's transactions root so the pre-execution check could skip re-encoding the decoded body. That root is derived by alloy from the raw payload bytes, and the header root the check compares it against is derived the same way, so the comparison compared a value to itself. The re-encode is not redundant work: it is what compares the raw bytes against the re-encoded decoded body, which rejects non-canonical transaction RLP. `validate_block_pre_execution_with_tx_root` states the contract the passthrough violated -- "the caller must ensure this value was derived from `block.body().calculate_tx_root()`". Conversion now returns a plain SealedBlock and the consensus check computes the root itself, as before. This withdraws the root half of the spec's optimization (.scratch/decode-once/ spec.md 4.2, 5.6); the decode-once saving stands at ~53 us/payload rather than the ~70 us claimed in 8. Also asserts latestValidHash's discriminator in the check-ordering tests, compares the fallback's block against the non-streaming path instead of only its error kind, and dedupes the three body_tx send sites. Claude-Session: https://claude.ai/code/session_01H8bWdbHjv1fuUGgyR3TZcT
The existing stream tests all ran below SMALL_BLOCK_TX_THRESHOLD, so the parallel fan-out was never exercised, and none of them could tell a working stream from a silent fallback: the disconnect fallback re-decodes the raw payload bytes and yields an identical block. Stream transactions that differ from the encoded ones to prove the body comes from the stream, and add a 40-tx case for the parallel path. Also correct the benchmark figure in the scratch spec to ~53 µs. The ~18 µs root re-encode saving assumed the raw-bytes tx-root passthrough, which was dropped in d774ea3. Claude-Session: https://claude.ai/code/session_018cVR99v8sB6YRtBU1Q9czL
gr8h
force-pushed
the
feat/decode-once-payload-issue-20431
branch
from
August 26, 2026 09:46
e621dff to
758b60f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Decodes execution payload transactions a single time and streams them to payload-to-block conversion, instead of decoding once for execution and again for the block body.
convert_payload_to_block_with_tx_stream: opt-in streaming conversion onPayloadValidator; default falls back to full decode.latestValidHash: nullper the engine API even when a transaction is malformed.Refs upstream issue paradigmxyz#20431.
https://claude.ai/code/session_01QBN5jDc2WdmkaqM1r2GqTL