Guard rollback-sensitive transaction flags - #8245
Merged
Merged
Conversation
Amaury Chamayou (achamayou)
force-pushed
the
achamayou-guard-tx-flags
branch
from
August 31, 2026 07:51
4c22cdd to
2cbb194
Compare
This was referenced Aug 31, 2026
A transaction whose view changed while it was committing could apply its writes to the local store and only then be refused replication, leaving state that never reaches consensus - contradicting the documented contract that a failed transaction is rolled back. Validate the view the transaction captured atomically with the allocation of its version, under the same lock a rollback takes, so it is refused before any map is modified. If allocation wins the race instead, the rollback observes the new version and truncates the writes. The unused caller-supplied version resolver is removed: it had no callers and would have bypassed this check. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75d99c5d-6efa-4048-8032-8c78b97208d9
Ledger chunk sizes were appended, and snapshot scheduling rolled back, outside the lock guarding the rollback epoch. A transaction could therefore restore chunk metadata for an entry a concurrent view change had already discarded, leaving the chunker permanently ahead of the store and skewing every later chunk boundary. Take the version lock for both the rollback and the append, and skip the append when the batch's rollback epoch or view no longer holds. A rollback can only discard a batch's writes by truncating, which moves the epoch on; a rollback that does not truncate may still move the view, which consensus rejects - so both are checked. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75d99c5d-6efa-4048-8032-8c78b97208d9
A rollback whose target is at or beyond the store's own version discards nothing, but still reset the chunker to that target. The chunker can legitimately lag the store, because an entry is allocated a version well before its size is recorded, so this moved the chunker forward past the store and left a permanent offset that skewed every later chunk boundary. Clamp the target to the store's version, so a rollback can only ever move chunk metadata back. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75d99c5d-6efa-4048-8032-8c78b97208d9
A transaction may request a forced ledger chunk, or arm a snapshot at the next signature. Both were applied after the transaction's writes, with no recheck that a concurrent view change had not already discarded them, so a transaction that never reached the ledger could still leave a chunk boundary behind, or arm a snapshot. Apply both under version_lock via Store::apply_tx_flags, refusing when the transaction's view or rollback epoch no longer holds, and attach the forced chunk to the transaction's own version rather than whichever version the store had since reached. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75d99c5d-6efa-4048-8032-8c78b97208d9
Amaury Chamayou (achamayou)
force-pushed
the
achamayou-guard-tx-flags
branch
from
August 31, 2026 12:29
2cbb194 to
03bb4a1
Compare
cjen1-msft
approved these changes
Sep 3, 2026
Copilot started reviewing on behalf of
Amaury Chamayou (achamayou)
September 4, 2026 10:19
View session
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The non-truncating rollback path can still advance a lagging chunker and permanently skip entry-size metadata.
Pull request overview
Guards KV transaction commits and side effects against concurrent rollbacks and view changes.
Changes:
- Validates commit terms before applying writes.
- Synchronizes transaction flags and chunk/snapshot metadata with rollback state.
- Adds regression tests and documents the API change.
Custom instructions used:
.github/copilot-instructions.md.github/instructions/changelog.instructions.md.github/instructions/reviewing.instructions.md
File summaries
| File | Description |
|---|---|
src/kv/apply_changes.h |
Supports rejected version resolution. |
src/kv/committable_tx.h |
Guards writes and transaction flags. |
src/kv/kv_types.h |
Extends the store interface. |
src/kv/store.h |
Coordinates rollback-sensitive state. |
src/kv/test/kv_test.cpp |
Adds rollback regression coverage. |
src/node/rpc/frontend.h |
Updates the commit call signature. |
src/node/snapshotter.h |
Updates the commit call signature. |
CHANGELOG.md |
Documents fixes and API changes. |
python/pyproject.toml |
Bumps the package version. |
Review details
Suppressed comments (2)
src/kv/store.h:699
- This clamp still advances a lagging chunker. For example, if the store is at version 2 while
current_tx_versionis 1, a rollback to version 2 callsrolled_back_to(2), whose implementation assignscurrent_tx_version = 2; the size for entry 2 is then permanently skipped. The non-truncating path should not callrolled_back_tounless the requested version is below the chunker's own position (orrolled_back_toitself should be made monotonic backwards).
}
src/kv/test/kv_test.cpp:3609
- This stale case changes both the term and
rollback_count, so it still passes if the new term check inapply_tx_flagsis accidentally removed. Add a separate non-truncating rollback case, where the term changes but the rollback count remains current, and verify that neither flag is applied.
store.set_consensus(consensus);
auto chunker = std::make_shared<InspectableChunker>();
store.set_chunker(chunker);
constexpr ccf::kv::Term initial_term = 2;
store.initialise_term(initial_term);
- Files reviewed: 5/5 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fourth of a stack of small fixes for a family of key-value store races found by an interleaving-exploration harness. Stacked on #8244.
The problem
A transaction can request two side effects that outlive it:
CommittableTx::commit()applied both immediately after allocating a version and applying its writes, beforeStore::commit()decides whether the transaction can be replicated at all. Neither application rechecked the transaction's view or rollback epoch, so if a concurrent election rolled the store back in that window:SNAPSHOT_AT_NEXT_SIGNATUREwas re-armed on the store afterStore::rollback()had deliberately cleared it, so the node takes a snapshot it was told not to take.Separately,
force_end_of_chunkwas called withversionread back off the transaction after the fact. That is the transaction's own version today, but it is trivially fragile: any caller reaching this path with a stale handle attaches the boundary to the wrong entry.The fix
Both effects move behind a single new
AbstractStore::apply_tx_flags, which takesversion_lockonce and applies them only if the transaction's view and rollback epoch still hold. On failure the transaction returnsFAIL_NO_REPLICATE, which is whatStore::commit()would have returned a moment later anyway - the transaction did not reach the ledger, so it must not leave anything behind.The read-only (
NoVersion) case is preserved explicitly: such a transaction has no version to attach a chunk to, but arming a snapshot is still legal and unchanged from before.Why this is minimal
version_lock, taken exactly whereset_flagalready took it. Lock order (commit_lock->version_lock->chunker_lock) is unchanged.Test
Rollback-sensitive transaction flags are not restoredinkv_testcallsStore::apply_tx_flagsdirectly - single-threaded, no harness - and asserts:Mutation-verified: disabling the guard fails (1); using the store's
versioninstead of the transaction's fails (3).Labelled
run-long-test.Review stack
These five PRs come from one investigation and are stacked; review and merge in order.
All were found by an interleaving-exploration harness built over the real KV, consensus and history stack (draft #8238). The harness itself is deliberately not included here; these PRs carry only the fixes and the single-threaded regression tests that pin them.