Skip to content

fix(snapshot): the state cursor marks the whole restore, not the tip - #1297

Merged
scarmuega merged 2 commits into
mainfrom
fix/restore-cursor-last
Sep 3, 2026
Merged

fix(snapshot): the state cursor marks the whole restore, not the tip#1297
scarmuega merged 2 commits into
mainfrom
fix/restore-cursor-last

Conversation

@scarmuega

@scarmuega scarmuega commented Sep 2, 2026

Copy link
Copy Markdown
Member

Re-implements #1283 on top of the stelae repository split, which left that PR unmergeable. Plan: plans/dolos-stelae-restore-operability.md in the txpipe domain root. The crates/stelae half now lives in txpipe/stelae#1 (branch feat/dir-blob-sidecar); this PR is the profile-side half.

What changed

Part (1) — the state cursor marks the whole restore, not the tip (20f2347a)

  • set_cursor moves after the live-UTxO index rebuild, as the last write of the restore. A node reads as restored only when its utxo::* dimensions are there too; an interruption anywhere leaves a node has_existing_data() reports as empty, and --continue repairs it from the epoch checkpoints.
  • crates/snapshot/PROFILE.md §"Restore pipeline" steps 5–6 and restore.rs's module doc say what the order now is, superseding the open question they carried.
  • A new TestFault::IndexApplyError fails only IndexWriter::apply, the restore's one call in the rebuild; a_restore_interrupted_in_the_live_utxo_rebuild_leaves_no_cursor pins the invariant.

Part (2) — a directory restore reads the blob map its stele carries (83a186e9)

  • Docs only on this side: restore_dir's doc, the preflight comment, and ADR-004's --output-dir line now describe the blobs.json sidecar. The code already asks the stele for its map.
  • The stelae pin is v0.2.0. crates/snapshot/Cargo.toml pins stelae and stelae-driver to tag v0.2.0, the release that carries the sidecar (perf(dir): a directory states its blob map instead of rebuilding it stelae#1, merged). The interim pin to the branch commit be10860 is gone; this is the pin that merges.

Verification

At 83a186e9, locally, against the pinned release:

  • cargo +nightly-2026-08-27 fmt --all -- --check — clean
  • cargo clippy --all-targets --all-features -- -D warnings — clean
  • cargo test -p dolos-snapshot -p dolos-testing --all-features — 185 passed, 0 failed (the ignored suites are the live-registry ones)
  • cargo deny check advisories — ok

The same fmt, clippy and test set was also run before the pin moved, against v0.1.0, and passed.

Not in this PR

Closing #1283, whose branch feat/stelae-dir-blob-sidecar this supersedes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RR5d8fir268ZksEocd8kbm

Summary by CodeRabbit

  • New Features

    • Snapshot publishing now creates a blobs.json sidecar file in the output directory alongside blobs and inscription data.
    • Directory restores can use the blob index for faster, more reliable data access while retaining compatibility with older snapshots.
  • Bug Fixes

    • Restore operations now finalize only after live UTxO indexes are rebuilt, preventing incomplete restores from leaving an invalid completion cursor.
    • Interrupted restores are correctly recognized and can be resumed or repaired.

The restore pipeline set the cursor at the end of step 5 and rebuilt the
live-UTxO index dimensions at step 6, so an interruption between the two
left a node `has_existing_data()` reported as restored and that answered
address and asset queries with fewer rows rather than an error.

`set_cursor` now runs after the rebuild, as the last write of the
restore: a node reads as restored only when its `utxo::*` dimensions are
there too, and an interruption anywhere leaves a node that reports as
empty — which is what it is. `--continue` still repairs it from the epoch
checkpoints; without it the stele is restored again from the top over
keyed writes, a rewrite and not a duplication.

Nothing else moves: `rebuild_utxo_indexes` takes the chain point as an
argument and never reads it back off the state store, and the tip is
never checkpointed, so a resumed restore already redid exactly the work
that now follows the cursor.

PROFILE.md §"Restore pipeline" and `restore.rs`'s module doc are amended
to say what the order is, superseding the open question they carried. The
test pins it through a new `TestFault::IndexApplyError`, which fails only
`IndexWriter::apply` — the restore's one call, in the rebuild.

Re-implemented from #1283, which could no longer merge once the stelae
crates left this workspace.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RR5d8fir268ZksEocd8kbm
@coderabbitai

coderabbitai Bot commented Sep 2, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 2a65251f-efec-4929-88c7-83376fd45611

📥 Commits

Reviewing files that changed from the base of the PR and between 9b6fd98 and 83a186e.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (6)
  • adrs/004_stelae_snapshots.md
  • crates/snapshot/Cargo.toml
  • crates/snapshot/PROFILE.md
  • crates/snapshot/src/restore.rs
  • crates/snapshot/tests/restore.rs
  • crates/testing/src/faults.rs

Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review.


📝 Walkthrough

Walkthrough

The Stelae dependencies and snapshot documentation now include blob-index sidecar handling. Restore writes the cursor after live-UTxO index rebuilding. Fault injection tests verify interrupted restores leave no cursor.

Changes

Stelae restore flow

Layer / File(s) Summary
Stelae format and restore inputs
crates/snapshot/Cargo.toml, adrs/004_stelae_snapshots.md, crates/snapshot/src/restore.rs
Stelae dependencies use v0.2.0. Documentation describes the blobs.json sidecar and blob-index lookup behavior.
Restore completion ordering
crates/snapshot/src/restore.rs, crates/snapshot/PROFILE.md
Live-UTxO indexes rebuild before the cursor is set and committed. Checkpoint cleanup remains after the cursor commit.
Interrupted restore validation
crates/testing/src/faults.rs, crates/snapshot/tests/restore.rs
The fault harness can fail IndexWriter::apply. Tests verify that an interrupted rebuild returns the expected error, leaves no cursor, and preserves landed UTxOs.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: 🔵 Low · up to 83a18

The restore now marks completion only after live-UTxO indexes are rebuilt, preventing interrupted restores from being treated as complete. A late rebuild failure can still leave a partial index appearing current to direct index consumers, although normal startup remains gated by the state cursor; this is a bounded low-risk consistency issue for owner awareness or follow-up, not a merge blocker.

Suggested reviewers: akashbhalla-svg

Sequence Diagram(s)

sequenceDiagram
  participant Restore
  participant IndexWriter
  participant CursorStore
  participant CheckpointStore
  Restore->>IndexWriter: Rebuild live-UTxO indexes
  Restore->>CursorStore: Set and commit cursor
  Restore->>CheckpointStore: Clear checkpoint state
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 38.46% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 3 files. (3 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: the state cursor now marks completion of the entire restore rather than only the state-tip step.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 38.46% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 3 files. (3 skipped: 3 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/restore-cursor-last

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

stelae's `SteleWriter::seal` now writes the identity→blob map beside
`inscription.json` as a `blobs.json` sidecar, and `SteleDir::blob_index`
reads it instead of decompressing every blob to rebuild it. Nothing in
this crate changes shape for that — `restore_dir` already asks the stele
for its map — so what lands here is the prose: `restore_dir`'s doc, the
preflight comment that said `blob_index` reads blobs, and ADR-004's
`--output-dir` line, which now names the sidecar among what a publish
writes to disk.

The stelae pin moves to `v0.2.0`, the release that carries the sidecar —
one tag for both crates, per the lockstep rule the dependency comment
states.

Re-implemented from #1283, which could no longer merge once the stelae
crates left this workspace.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RR5d8fir268ZksEocd8kbm
@scarmuega
scarmuega force-pushed the fix/restore-cursor-last branch from 58a79a1 to 83a186e Compare September 2, 2026 19:03
@scarmuega
scarmuega marked this pull request as ready for review September 2, 2026 20:09
@scarmuega
scarmuega requested a review from a team as a code owner September 2, 2026 20:09
@scarmuega
scarmuega merged commit dc33a5f into main Sep 3, 2026
17 checks passed
@scarmuega
scarmuega deleted the fix/restore-cursor-last branch September 3, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant