release: gloas#795
Open
mattevans wants to merge 76 commits into
Open
Conversation
…928 support fix(proto): implement proper handling of Gloas beacon blocks and their fields refactor(block_access_list.go): add functionality to decode and process block access list data from Gloas blocks fix(execution_transaction.go): adjust processing logic to handle transactions in Gloas blocks fix(beacon_blob.go): modify kzg commitments extraction for Gloas blocks to ensure proper retrieval fix(proposed_validator_block.go): update logic to correctly compute Gloas block size and handle transactions
feat(canonical): add ExecutionPayloadSlotNumber to canonical block batch fix(beacon): update GloasBlock message retrieval to use GetMessage method fix(event-ingester): fix casting of GloasBlock message to use GetMessage method test: add tests for BlockAccessList related functionalities style: reorder imports for clarity in conversion test file chore: refactor existing append functions to include ExecutionPayloadSlotNumber handling
Implement BeaconBlockAccessList type for handling access list events in the ETH V2 protocol. This adds support for new event type, enhances event routing, and allows for better structuring of events in the system.
…r improved stability chore(go.mod): update github.com/ethpandaops/beacon dependency to version v0.67.1 chore(go.sum): update checksum for the new github.com/ethpandaops/beacon version style(block_access_list.go): remove unnecessary empty line for cleaner code readability
… to ensure compatibility and access to bug fixes chore(go.sum): update checksum for ethpandaops/beacon to match new version
…or improved functionality fix(go.mod): change ethcore version to v0.0.0-20260323002709-50f492685a52 for compatibility fix(go.sum): update checksums for go-ethereum, c-kzg-4844, and ethcore dependencies fix(conversion.go): update access.Code to access.CodeChanges for clarity in block access list creation fix(conversion_test.go): adjust test to set CodeChange correctly in the test setup for block access list
… in Kafka and ClickHouse This change enhances the data pipeline by adding the beacon block access list event transformation and corresponding sinks, enabling better handling of this specific event data for analytics and processing downstream.
refactor: bals structure rlp decoded
… properties to use camelCase style(vector-kafka-clickhouse.yaml): standardize field naming conventions to match expected API format
…better customization
* master: chore(buf.yaml): update linting rules to exclude specific checks for better customization
…d storage read tracking fix(conversion): adjust handling of storage reads to use new BlockAccessListStorageRead structure
feat(gloas): add BAL storage reads and bump deps to bal-devnet-3
… with no state changes to enhance parallel execution analysis feat(proto): update BlockAccessListChange to include "touched" in change_type options for better clarity on account status
feat(gloas): add BAL storage reads and touched accounts
feat: prepare epbs types
…ClickHouse tables to improve data structure for valid bid tracking and payload attestation flow feat(migrations): create various new tables for tracking execution payloads, bids, and proposer preferences to enhance transaction handling and performance fix(migrations): reverse EIP-7732 changes in down scripts to ensure consistent rollback functionality and data integrity
feat(migrations): add epbs support
…BS conditional block routing
…): remove execution payload logic for Gloas blocks to simplify the code base and adapt to new event handling requirements
… execution payload and proposer preferences. feat(libp2p): add event names for libp2p Gossipsub events to manage payload attestation and proposer preferences. fix(routes_test): remove deprecated events from test coverage related to pending routes for ePBS. refactor(canonical): implement flattening methods for execution payload bid, payload attestation, and beacon block related events.
…y and readability of withdrawal_type classification
refactor: widen block_access_index from UInt16 to UInt32 per EIP-7928
* master: chore(workflows): update Golang version to 1.26.2 and golangci-lint version to 2.12.2 for consistency and compatibility chore(Dockerfile, go.mod): upgrade base Go image to 1.26.2 and update Go module version to match chore(go.mod): update dependencies to ensure compatibility with Go 1.26.2 and fix potential vulnerabilities
…ields for consistency and maintainability chore(log_fields): create log_fields.go to centralize log field keys for v1 and v2 beacon event handlers style(test): replace hardcoded snapshot IDs and client names in tests with constants for better readability and maintainability
feat: wire ePBS gossipsub producers for the 4 Gloas topics
…internals Adds the producer + consumer side for `BEACON_SYNTHETIC_PAYLOAD_STATUS_RESOLVED` and `BEACON_SYNTHETIC_BUILDER_PENDING_PAYMENT_SETTLEMENT` — TYSM-instrumented events that observe ePBS internal state transitions on every beacon node (multi-witness, not gossip). Naming follows the LIBP2P_TRACE_SYNTHETIC_HEARTBEAT precedent — these are "synthesized" because they don't correspond to any beacon API SSE event; they're emitted from inside prysm via TYSM's hook system. Captured by every TYSM-patched beacon node, every slot / every epoch. What you get from these: - PayloadStatusResolved: empty-slot rate, envelope timing distributions, PTC participation rate, fork-choice latency, INVALID payload transitions. - BuilderPendingPaymentSettlement: builder payment success rate (SETTLED vs DROPPED), lost-revenue totals per builder, quorum-margin distributions, builder market concentration. Changes: - pkg/proto/eth/v1/beacon_synthetic.proto — new PayloadStatusResolved + BuilderPendingPaymentSettlement message types with PayloadStatus (PENDING/FULL/EMPTY/INVALID) and BuilderPendingPaymentOutcome (SETTLED/DROPPED) enums. - pkg/proto/xatu/event_ingester.proto — 2 new Event_Name enum values (102, 103), 2 new AdditionalData messages, 2 new oneof entries in DecoratedEvent (Additional* IDs 108/109, data IDs 223/224). - pkg/clmimicry/ — 2 TraceEvent payload structs, 2 New*Payload builders, 2 TraceEvent_* constants, dispatch map entries, case arms in handleHermesRPCEvent, 2 EventConfig toggles, 2 Group D categorizer registrations, 2 TraceEventTo* converters, 2 handle* methods + 2 deriveAdditionalDataFor* helpers in a new event_beacon_synthetic.go. - pkg/server/service/event-ingester/event/beacon/synthetic/ — 2 new server handler files in a new `synthetic` package (parallels `eth/v1/`); registered in the event router. - deploy/migrations/clickhouse/003_gloas_epbs_support.up.sql — 2 new tables appended (no new migration since 003 is greenfields, not yet deployed). Both use ReplicatedReplacingMergeTree with meta_client_name in the ORDER BY composite to preserve multi-witness records. - pkg/clickhouse/route/beacon/beacon_synthetic_*.go — 2 hand-written FlattenTo routes calling the existing extractBeaconSlotEpochPropagation helper. The matching .gen.go files come from `make clickhouse-routes`. - pkg/clickhouse/route/cmd/generate/manifest.go — 1-line addition registering the `beacon_synthetic_` prefix in the codegen prefix-route table so the generator emits the new files into pkg/clickhouse/route/beacon/. - Updated existing .gen.go files from re-running `make clickhouse-routes` (rowgen tool produces a tighter output than the last commit on a few fields — the regen is consistent with the schema, no functional change). Tests: - pkg/clmimicry/event_categorizer_test.go — counts updated 33→35 events, GroupD ≥10→≥12 (the 2 new synthetic events are Group D — no Topic, no MsgID, same shape as engine API events / custody probes). - pkg/clickhouse/route/beacon/beacon_synthetic_*_test.go — placeholder snapshot tests following the existing `TODO(epbs)` pattern; they currently fail with the standard `"nil … payload: invalid event"` message awaiting real Gloas devnet captures. Verification: - buf lint / buf build → clean - go build ./... → clean - go test -race ./pkg/clmimicry/... → ok - make proto + make clickhouse-routes → clean output Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat: beacon_synthetic_* events
feat: PTC vote three-state breakdown + EIP-7843 slot_number coverage
feat: add OnPayloadAttestationProcessed observe hook
* master: chore: swap go-eth2-client to ethpandaops fork v0.1.2 (#828) feat(output): add s3blobstore sink for blob sidecar archival
Bumps: - go-ethereum replace pin: bal-devnet-3 (8f361e342cb9, 2026-03-23) → bal-devnet-6 HEAD (73944e329925, 2026-05-07). Matches the EL branch deployed on glamsterdam-devnet-3 per ethpandaops/glamsterdam-devnets ansible inventory. - ethpandaops/beacon: 0260912 → ddfa770 (carrying matching go-ethereum pin). - ethpandaops/ethcore: 4e1b5fb → f67d7a6 (carrying matching go-ethereum pin + new eth.NewPeer 5-arg call). bal-devnet-6 keeps AccountAccess.StorageChanges + EncodedStorage.ToHash required by pkg/proto/eth/v1/conversion.go and brings the latest BAL (EIP-7928) refinements plus Amsterdam slot-number support (EIP-7843).
Resolves enum + migration collision by shifting BAL+ePBS up by 1:
- BEACON_API_ETH_V1_EVENTS_FAST_CONFIRMATION keeps event enum 89, oneof
field 95, oneof data field 210, and migration 002.
- BEACON_API_ETH_V2_BEACON_BLOCK_ACCESS_LIST moves 89 → 90 (oneof 96/211,
migration 003).
- All 15 ePBS enums shift up by 1 (90 → 91 ... 104 → 105) with matching
oneof field renumbering.
Accepts master's consumoor-only local stack (deletes vector-http-kafka.yaml
and vector-kafka-clickhouse{,-libp2p}.yaml; adds xatu-consumoor service
config + xatu-server direct kafka output with topicPattern
"xatu-protobuf-${event-name}").
Bumps ethpandaops/beacon to release/gloas tip 81b9ef3 to pick up both ePBS
SSE hooks (release/gloas) and OnFastConfirmation (merged in from master).
# By Sam Calder-Mason # Via GitHub * master: feat(observability): full OTel traces with gRPC propagation (#835) # Conflicts: # pkg/proto/eth/v2/events_vtproto.pb.go # pkg/sentry/event/beacon/eth/v1/beacon_blob.go # pkg/sentry/event/beacon/eth/v1/beacon_committee.go # pkg/sentry/event/beacon/eth/v1/events_attestation.go # pkg/sentry/event/beacon/eth/v1/events_blob_sidecar.go # pkg/sentry/event/beacon/eth/v1/events_block.go # pkg/sentry/event/beacon/eth/v1/events_block_gossip.go # pkg/sentry/event/beacon/eth/v1/events_chain_reorg.go # pkg/sentry/event/beacon/eth/v1/events_contribution_and_proof.go # pkg/sentry/event/beacon/eth/v1/events_data_column_sidecar.go # pkg/sentry/event/beacon/eth/v1/events_finalized_checkpoint.go # pkg/sentry/event/beacon/eth/v1/events_head.go # pkg/sentry/event/beacon/eth/v1/events_single_attestation.go # pkg/sentry/event/beacon/eth/v1/events_voluntary_exit.go # pkg/sentry/event/beacon/eth/v1/proposer_duty.go
… consistency and to avoid unnecessary conversion
feat: port otel work to glammy work
* master: fix(processor): stop per-trace partitioning in BatchItemProcessor (#841)
feat: decode Gloas data_column_sidecar gossip events
… dependencies and reasons for the pinning to bal-devnet-6
refactor: glammy-devnet-4 prysm renamed fields
…ypes for consistency with the new API design This change modifies the event constant names and proto types, ensuring they align with the updated API structure. The changes improve clarity and foster easier integration and maintenance of the event ingestion system. refactor(proto): reorder fields in event_ingester.pb.go for improved organization and readability. This change enhances the maintainability of the code by establishing a more logical structure. refactor(proto): rename fields in Event message for clarity and consistency fix(proto): update method signatures in event_ingester_vtproto for renamed fields to match updated Event message structure
…TxIndex to ensure correct value representation in BlockAccessList
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Glamsterdam fork support to Xatu.
What this covers
EIP-7928 — Block-Level Access Lists (BALs)
Spec-compliant against
bal-devnet-3.BlockAccessList,BlockAccessListEntry,BlockAccessListChange,BlockAccessListStorageRead(incl."touched"change type for accounts with no state changes)BeaconBlockGloas/BeaconBlockBodyGloasproto messagesExecutionPayloadGloaswithblock_access_list+slot_numberBEACON_API_ETH_V2_BEACON_BLOCK_ACCESS_LIST = 89NewBlockAccessListFromGloas()(with test coverage)NewBlockAccessListDeriver(CannonType 16) — wired002_gloas_bals_support(tablecanonical_beacon_block_access_list+execution_payload_block_access_list_rootcolumn oncanonical_beacon_block)NewEventBlockFromGloas()EIP-7732 — Enshrined Proposer-Builder Separation (ePBS)
Internal plumbing wired end-to-end across server, sentry, cannon, fork-choice. Awaiting Gloas devnet for integration testing.
eth/v1files (incl.execution_payload_available.proto), 4 libp2p gossipsub summary types, 12 new event IDs (90–101), 12Additional*Datamessages, 3 newCannonTypeentries (16/17/18)003_gloas_epbs_support— 12 new tables + column additions (builder_index,bid_value,execution_payment,payload_present)event.go(incl. new GOSSIP / AVAILABLE handlers)OnExecutionPayload,OnExecutionPayloadGossip,OnExecutionPayloadAvailable,OnExecutionPayloadBid,OnPayloadAttestationMessage,OnProposerPreferencesNewPayloadAttestationDeriver,NewExecutionPayloadBidDeriver; backfilling checkpoint iterator handles CannonType 16/17/18 in bothGetMarkerandcreateLocationFromEpochNumberForkChoiceNodeV2.payload_statusround-trips viaextra_data["payload_status"]pkg/clickhouse/route/{beacon,canonical,libp2p}/What this does NOT yet cover
Devnet-blocked or out-of-scope:
pkg/clmimicry/) — server handlers and routes already exist; producer side is the gapExecutionTransaction/Withdrawal/BlockAccessList— depends onBeaconBlockBodyGloasno longer carryingExecutionPayload; wired onceenvelope events arrive on devnet
nil … payload: invalid eventawaiting Gloas devnet dataOnBlock/OnHeadGloas paths) — wire once devnet stabiliseseth_getBlockAccessListJSON-RPC, Engine API V5/V6 tracking (newPayloadV5,getPayloadV6,forkchoiceUpdatedV4)Intentionally not included (don't reintroduce without spec confirmation)
payload_voteprojection on attestation tables —committee_indexalready stores rawdata.index; consumers can derive 0/1 for Gloas+ rowssidecar_slot/sidecar_beacon_block_rootprojections on DataColumnSidecar tables — preemptive; revisit when real Gloas SSE payloads arriveMigrations
002_gloas_bals_support003_gloas_epbs_support