Skip to content

perf(events): skip redundant writer dup probe when caller pre-checked#227

Open
Thunder-Blaze wants to merge 1 commit into
hoytech:masterfrom
Thunder-Blaze:perf/consolidate-duplicate-probe
Open

perf(events): skip redundant writer dup probe when caller pre-checked#227
Thunder-Blaze wants to merge 1 commit into
hoytech:masterfrom
Thunder-Blaze:perf/consolidate-duplicate-probe

Conversation

@Thunder-Blaze

Copy link
Copy Markdown
Contributor

Issue

  • writeEvents() was calling lookupEventById() on every event even when WriterPipeline had just run the same lookup in the same thread, so import/sync/router paid two B-tree seeks per event

Description

  • WriterPipeline (used by strfry import / sync / router) already probes lookupEventById under a read-only txn right before opening its txn_rw and calling writeEvents(). The same probe then runs again inside writeEvents for every event. Consolidates to one probe on that path.

  • Adds bool preChecked on EventToWrite. When set, writeEvents() skips lookupEventById but still runs the unconditional intra-batch guard (ev.id() == evs[i-1].id()). WriterPipeline sets the flag right after its pre-filter.

  • RelayIngesterRelayWriter is intentionally not changed: the ingester's ro-txn is a batch-start snapshot and with multiple ingester threads its probe can be stale by the time the writer processes the message; skipping the writer probe there would race and double-insert.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant