Two cleanup paths exist in the codebase but are never invoked, so state accumulates unbounded on long-running instances.
3a — DownloadProcessingJobs grows unbounded. IDownloadProcessingJobService.CleanupOldJobsAsync(retentionDays) is implemented (EfDownloadProcessingJobRepository.CleanupOldJobsAsync) but has no caller. The table accumulates completed/failed rows indefinitely, and every queue-snapshot reconciliation that queries it pays for the bloat.
→ PR incoming: a small hosted DownloadProcessingJobCleanupService that calls CleanupOldJobsAsync on a schedule (shortly after startup, then daily), with tests. Already implemented and running locally.
3b — orphaned downloads are never terminalized. The orphan branch in DownloadQueueService.GetQueueSnapshotAsync only logs ("keeping records for resilient monitoring/import handling") and never moves long-dead rows to a terminal state, so they linger in the queue-matching pool indefinitely.
Design care if we act on 3b:
- preserve the existing empty-client guard (a temporarily-unreachable client returning 0 items must not mass-terminalize);
- only terminalize an allowlist of "should be in the client but isn't" states (Queued/Paused) — never Ready/ImportPending/ImportBlocked, which are legitimately absent and would destroy pending imports;
- key off continuous absence (missing-since), not
StartedAt, so a single partial snapshot can't false-terminalize.
Open question for maintainers: should orphans ever auto-terminalize, or stay user-driven? Given the deliberate log-only choice, an opt-in setting (default off) seems safest.
Related: #631 (downloads getting Import Blocked) for 3b context. Surfaced during a scale-testing discussion in Discord.
Two cleanup paths exist in the codebase but are never invoked, so state accumulates unbounded on long-running instances.
3a —
DownloadProcessingJobsgrows unbounded.IDownloadProcessingJobService.CleanupOldJobsAsync(retentionDays)is implemented (EfDownloadProcessingJobRepository.CleanupOldJobsAsync) but has no caller. The table accumulates completed/failed rows indefinitely, and every queue-snapshot reconciliation that queries it pays for the bloat.→ PR incoming: a small hosted
DownloadProcessingJobCleanupServicethat callsCleanupOldJobsAsyncon a schedule (shortly after startup, then daily), with tests. Already implemented and running locally.3b — orphaned downloads are never terminalized. The orphan branch in
DownloadQueueService.GetQueueSnapshotAsynconly logs ("keeping records for resilient monitoring/import handling") and never moves long-dead rows to a terminal state, so they linger in the queue-matching pool indefinitely.Design care if we act on 3b:
StartedAt, so a single partial snapshot can't false-terminalize.Open question for maintainers: should orphans ever auto-terminalize, or stay user-driven? Given the deliberate log-only choice, an opt-in setting (default off) seems safest.
Related: #631 (downloads getting Import Blocked) for 3b context. Surfaced during a scale-testing discussion in Discord.