Repository navigation
SnapshotWriter::open: cut a torn tail off before appending - #33
Merged
Merged
Conversation
A short write (the disk ran out at a block boundary, then a drop whose flush failed) leaves part of a record at the end of the log. A writer opened without a `load` first appended behind it, and the log was then corrupt mid-file for good: every `load` and every `compact` failed with a CRC mismatch. instd's VXLAN network sync hit this on the homelab (network-sync.snap, a 6-byte and a 5-byte torn cursor record at a 4 KiB boundary, then complete records) and logged "failed to compact snapshot" every 30 s from then on. open now validates the log and truncates a torn tail (a truncate needs no free space, unlike load's rewrite), replaces a torn header instead of appending after it, and refuses mid-file corruption, as load does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XoaQgogNq1z3dNoM9feGoH
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A short write (the disk ran out at a block boundary, then a drop whose flush failed) leaves part of a record at the end of the log.
SnapshotWriter::openappended behind it whenever noloadran first. From then on the log was corrupt mid-file for good: everyloadand everycompactfailed withCorrupted.This happened in production-shaped use. On the beyond homelab, instd's VXLAN
network-sync.snaphas a 6-byte and a 5-byte torn cursor record just before a 4 KiB block boundary (offset 139253, boundary 0x22000), followed by complete records. instd has logged "failed to compact snapshot: snapshot corrupted (CRC mismatch)" every 30 s ever since, and the file never compacts.What
SnapshotWriter::opennow validates the log before appending:replay_log; a truncate needs no free space, unlikeload's rewrite;Corrupted, asloaddoes.The ARCHITECTURE format section is updated, and the version is bumped to 0.8.6.
Proof
open_after_a_torn_tail_never_appends_behind_it,open_after_a_torn_header_starts_a_fresh_log,open_refuses_mid_file_corruption.mise exec -- cargo test: 0 failures. clippy is clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01XoaQgogNq1z3dNoM9feGoH