Skip to content

write chunkindex fragment(s) before the archive pointer - #10248

Merged
ThomasWaldmann merged 1 commit into
borgbackup:masterfrom
ThomasWaldmann:fix-10239-index-before-pointer
Aug 29, 2026
Merged

write chunkindex fragment(s) before the archive pointer#10248
ThomasWaldmann merged 1 commit into
borgbackup:masterfrom
ThomasWaldmann:fix-10239-index-before-pointer

Conversation

@ThomasWaldmann

Copy link
Copy Markdown
Member

Fixes #10239.

The archive pointer write (archives/<archive_id>) is the commit point of a backup. Previously, the chunkindex fragment covering the session's new chunks was only written at cache close (and every 600s during the backup), i.e. after the pointer: a crash in between left a committed archive whose chunks were partly not covered by index/*, so reads (which are routed through the index) raised ObjectNotFound until borg check rebuilt the index.

Now Archive.save() and Archive.set_meta() (rename/tag/copy) persist the index fragment(s) covering everything the session stored so far before writing the pointer, so a committed archive always has complete index coverage. A crash before the pointer write merely leaves an index fragment referencing uncommitted objects, which is harmless (compact/rebuild prunes it). This also covers import-tar, recreate and transfer, which commit archives through Archive.save(); the check --repair paths already rebuild the whole index in finish().

Notes:

  • The forced write resets the periodic (600s) index write timer, so no duplicate write follows right after committing.
  • The close-time index write is now usually a no-op (no new chunks left), but is kept for sessions that store chunks without committing an archive.
  • Multi-archive sessions (e.g. borg transfer) write one fragment per committed archive, covering the chunks stored since the previous write; the bounded repack at cache close consolidates small fragments as before.
  • docs: removed the packs.rst TODO about this and updated the write-order steps to match the implementation (the figure already showed the intended order).
  • test: regression test simulates dying right after the commit point (close-time index write suppressed) and asserts the archive is still fully readable via index/* only. It fails on master: there, the archive is not even findable, as its metadata chunk cannot be read through the index.

🤖 Generated with Claude Code

…ckup#10239

The archive pointer write (archives/<archive_id>) is the commit point of
a backup. Previously, the chunkindex fragment covering the session's new
chunks was only written at cache close (and every 600s during the
backup), i.e. AFTER the pointer: a crash in between left a committed
archive whose chunks were partly not covered by index/*, so reads (which
are routed through the index) raised ObjectNotFound until "borg check"
rebuilt the index.

Now Archive.save() and Archive.set_meta() (rename/tag/copy) persist the
index fragment(s) covering everything the session stored so far before
writing the pointer, so a committed archive always has complete index
coverage. A crash before the pointer write merely leaves an index
fragment referencing uncommitted objects, which is harmless
(compact/rebuild prunes it). This also covers import-tar, recreate and
transfer, which commit archives through Archive.save().

The close-time write is now usually a no-op (no new chunks), but is kept
for sessions that store chunks without committing an archive.

Also: remove the packs.rst TODO about this and update the write-order
steps to match the implementation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.47%. Comparing base (6299333) to head (c82a280).
⚠️ Report is 3 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #10248   +/-   ##
=======================================
  Coverage   87.47%   87.47%           
=======================================
  Files         103      103           
  Lines       18534    18538    +4     
  Branches     2843     2843           
=======================================
+ Hits        16213    16217    +4     
  Misses       1623     1623           
  Partials      698      698           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@ThomasWaldmann

Copy link
Copy Markdown
Member Author

@mr-raj12 ^

@ThomasWaldmann
ThomasWaldmann merged commit 39bd771 into borgbackup:master Aug 29, 2026
25 checks passed
@ThomasWaldmann
ThomasWaldmann deleted the fix-10239-index-before-pointer branch August 29, 2026 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

borg2: write chunkindex fragment before archive pointer

1 participant