fix(worker): One storage config with a missing root no longer stops the Worker (#282)

[bitbiter-dev]

Closes #282.

Summary

One storage config whose root is missing or unreadable no longer stops the whole Worker. A failed crawl is logged with the config's root path and user, and the loop moves on to the next config.

 Worker.CrawlAllAsync
   for each active config
     CrawlAsync(config)
-      ingestionPipeline.RunAsync(config)        # root missing → throws → BackgroundService faults → StopHost
+      try ingestionPipeline.RunAsync(config)
+      catch when not a shutdown (IngestionShutdown.IsInProgress)
+        Log.CrawlFailed(rootPath, userId, ex)   # Error; next config runs

• A shutdown in the middle of a crawl still propagates and is not logged as a failure.
• FileIngestionPipeline still throws for a missing root. Only the per-config loop catches it.
• CLAUDE.md now says a missing root fails that config's crawl, not the process.
• The IngestionShutdown comment names its third caller.

Loading the config list itself (the database being unreachable, for example) can still stop the host. That is outside #282 and is noted on #281, which replaces this loop with the work queue.

Evidence

• Before: CrawlAllAsync_ConfigCrawlFails_LogsItsRootAndCrawlsTheNextConfigAsync failed, because the exception escaped and the second config was never crawled.
  After: it passes, together with:
  • ExecuteAsync_CrawlFails_KeepsRunningAndCrawlsAgainAfterTheIntervalAsync: crawl fails → timer re-armed → crawl again after 3 h, and the Worker is still running;
  • CrawlAllAsync_ShutdownDuringACrawl_StopsWithoutLoggingAFailureAsync: throws OperationCanceledException, no next config, no error log;
  • CrawlAllAsync_CancellationWithoutShutdown_IsLoggedAsACrawlFailureAsync.
• All test projects pass, and dotnet format --verify-no-changes is clean.
• scripts/mutation-changed.sh: 100.00% (5 killed, 0 survived).

Merge Danger

Door: two-way

Blast Radius: Worker crawl loop

A crawl error that used to crash the Worker is now logged at Error level and retried on the next interval. An operator who relied on the container restarting to notice a missing mount now has to watch the Worker logs for the error.

🤖 Generated with Claude Code