Skip to content

ethereum: fix non-transitive trigger ordering causing sort panics - #6725

Open
alter wants to merge 1 commit into
graphprotocol:masterfrom
alter:fix/ethereum-trigger-ordering-transitivity
Open

alter wants to merge 1 commit into
graphprotocol:masterfrom
alter:fix/ethereum-trigger-ordering-transitivity

Conversation

@alter

@alter alter commented Sep 25, 2026

Copy link
Copy Markdown

Fixes #6071

What is going on

EthereumTrigger::cmp compares Call triggers (and Call/Log pairs) by transaction_index, but compares Log/Log pairs by log_index alone, ignoring transaction_index entirely:

(Self::Call(a), Self::Call(b)) => a.transaction_index.cmp(&b.transaction_index),
(Self::Log(a), Self::Log(b)) => a.log_index().cmp(&b.log_index()),

Those two comparisons only agree when log_index happens to increase in lockstep with transaction_index for every trigger in the set being sorted. That is not guaranteed for every trigger source. When it doesn't hold, the combined ordering stops being transitive.

Concretely: a Log in an earlier transaction with a high log_index compares as greater than a Call in the same transaction (correct: events come before calls in a transaction), and that Call compares as less than a Log in a later transaction (correct: earlier transaction sorts first) -- but the two Logs compare directly by log_index alone, so the earlier-transaction Log can come out greater than the later-transaction Log if its log_index happens to be higher. a < b, b < c, but a > c.

This is what crashes graph-node in production with:

thread 'tokio-runtime-worker' panicked at library/core/src/slice/sort/shared/smallsort.rs:860:5:
user-provided comparison function does not correctly implement a total order

inside BlockWithTriggers::new_with_triggers, which calls trigger_data.sort() on a freshly built Vec<Trigger>.

Fix

Key the Log/Log comparison on transaction_index first, falling back to log_index only as a tie-breaker within the same transaction -- the same primary key already used by the Call/Call and Call/Log comparisons. Also rewrote the Call/Log and Log/Call arms with Ordering::then to drop the duplicate guarded/unguarded arm pairs, since both arms now share the same primary-then-secondary-key shape.

Testing

Added test_trigger_ordering_is_transitive in chain/ethereum/src/tests.rs, using the existing create_log test fixture. It constructs the minimal counter-example (a Log in transaction 5 with log_index 50, a Call also in transaction 5, and a Log in transaction 6 with log_index 10), asserts the three pairwise comparisons are consistent, and asserts sorting the Vec doesn't panic.

Confirmed the test fails against the pre-fix code with exactly the expected assertion (log_a.cmp(&log_c) returns Greater instead of Less), and passes after the fix.

Ran cargo test -p graph-chain-ethereum --lib (60 passed, including the two pre-existing trigger-ordering tests), cargo fmt -p graph-chain-ethereum -- --check, cargo clippy -p graph-chain-ethereum --lib -- -D warnings, and cargo check -p graph-chain-ethereum --release, all clean.

EthereumTrigger::cmp compared Call triggers (and Call/Log pairs) by
transaction_index, but compared Log/Log pairs by log_index alone,
ignoring transaction_index entirely. Those two comparisons only agree
when log_index happens to increase in lockstep with transaction_index
across the whole set being sorted, which isn't guaranteed for every
trigger source. When it doesn't hold, the combined ordering is not
transitive: a Log in an earlier transaction can end up compared as
greater than a Log in a later transaction if the earlier one has a
higher log_index within its own transaction.

This crashes graph-node in production with a panic from Rust's stable
sort ("user-provided comparison function does not correctly implement
a total order") inside BlockWithTriggers::new_with_triggers, which
calls trigger_data.sort() on a freshly-built Vec<Trigger>.

Fix the Log/Log comparison to key on transaction_index first, with
log_index only as a tie-breaker within the same transaction, matching
how Call/Call and Call/Log comparisons already work. Also rewrite the
Call/Log and Log/Call arms with Ordering::then to remove the duplicate
guarded/unguarded match arm pairs, now that both arms share the same
primary-then-secondary-key logic.

Added a regression test (test_trigger_ordering_is_transitive) that
constructs the minimal counter-example and asserts both the pairwise
comparisons and that sorting the resulting Vec doesn't panic.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] smallsort panic in tokio task

1 participant