borg-200b20 check --find-lost-archives --repair hits invalid filename

[tve]

I'm continuing to mess around with a repo that has suffered some corruption (#9306) and I hit the following error in borg check:

BORG_REPO=/mnt/zerofs/v2.0.0b20 ./borg-200b20 check --find-lost-archives --repair --progress -v                  
This is a potentially dangerous function.            
check --repair might lead to data loss (for kinds of corruption it is not                                  
capable of dealing with). BE VERY CAREFUL!           

Type 'YES' if you understand this and want to continue: YES                                                
Starting full repository check                       
Starting from beginning.                             
Checkpointing at key data/01148af662e7657967c3e288759f9850f464881a53ed42b15269593e18a1dd7c.                
Checkpointing at key data/02a66db6631e78d3c2a6f3e079f73a970d4f5a6c7eeeb5862f8743bf6b0aa9ec. 
...
Checkpointing at key data/940ecf2d6512c4df8c878bd8f3a389d2274b323321a3fdb9875689391032214b.  
Local Exception

Error:

ValueError: name must be lowercase, but got: data/.9/5a/.95a99eb256e8685192973ca84a1ef1b823dba0da3cca2ad8da4c4bc1813186b3.wOJKSW

Trying to figure out what is going on:

# find . -name \*56e8685192973\*                          
./data/95/a9/.95a99eb256e8685192973ca84a1ef1b823dba0da3cca2ad8da4c4bc1813186b3.wOJKSW        
./data/95/a9/95a99eb256e8685192973ca84a1ef1b823dba0da3cca2ad8da4c4bc1813186b3                
# ls -ls ./data/95/a9/.95a99eb256e8685192973ca84a1ef1b823dba0da3cca2ad8da4c4bc1813186b3.wOJKSW                                                                   
768 -rw------- 1 root root 786432 Jan 21 13:12 ./data/95/a9/.95a99eb256e8685192973ca84a1ef1b823dba0da3cca2ad8da4c4bc1813186b3.wOJKSW                                                      
# ls -ls ./data/95/a9/95a99eb256e8685192973ca84a1ef1b823dba0da3cca2ad8da4c4bc1813186b3                                                                           
914 -rw------- 1 root root 935779 Jan  2 09:20 ./data/95/a9/95a99eb256e8685192973ca84a1ef1b823dba0da3cca2ad8da4c4bc1813186b3                                                              

Is this borg tripping over its own temp file? Or did I run some other tool that created that temp file?
And why does an invalid filename stop the operation?

(Edit: and why, when I restart the borg check, does it not continue at the last checkpoint?)

[ThomasWaldmann]

Yeah, looks like a temp file.

That's a bit strange, because the current borgstore code either:

• renames the tempfile over the desired final filename (if writing to the tempfile was completed successfully), or
• deletes the tempfile if that rename does not succeed.

Also, borgstore creates the temp file using TMP_SUFFIX (== ".tmp") and I don't see that there.

Could it be that this temp file was created by a much older borg or borgstore version than the current one? There were no significant borgstore changes recently.

[ThomasWaldmann]

Hmm, could that be an artifact of how your filesystem works?

[tve]

I had to rsync that directory, I think it uses this type of temp filename. Didn't think about that before. OK, one mystery solved, apologies for the noise.

[ThomasWaldmann]

Just checked: borgstore PosixFS.list will never yield a filename that ends with TMP_SUFFIX (its own temp files).

[ThomasWaldmann]

TODO: Repository.check or PosixFS.list / store.list could validate the names and skip if it does not look valid (== all hex, 64 chars, all lowercase).

borgbackup/borgstore#130

[ThomasWaldmann]

About continuing at checkpoint: it does that, but only when the partial flag is given. The default is to play safe and do a full check.

[tve]

So borg check --find-lost-archives --repair does not allow --max-duration but it does create checkpoints. Is there a separate undocumented "partial flag"?

[ThomasWaldmann]

IIRC, checkpointing (and partial checks) is only supported for pure repo (not: repo+archive) checks.

For repo+archive we need a fully consistent index built from all repo objects.

[ThomasWaldmann]

Continued there: borgbackup/borgstore#130