Report duplicate content (report-only; never merges or deletes) #36
Copy link
Copy link
Closed
Labels
enhancementNew feature or requestNew feature or requestready-for-agentFully specified, ready for an AFK agentFully specified, ready for an AFK agentv1.0Required for the v1.0 releaseRequired for the v1.0 release
Description
Decision (supersedes the original text)
Report-only. Deduplication stays per storage config: the same bytes under two configs are two assets, which is what makes multi-user work (CLAUDE.md). This issue never merges, hides or deletes anything. It tells the user where duplicate content lives so they can clean up the NAS.
What counts as a duplicate
- Within one config: the same
ContentHashat two paths that both exist. Only one row exists, because the partial unique index allows one active asset per hash. The second copy is seen during the crawl, and Re-link moved/renamed files and refresh LastSeenOnNas on every crawl #35 identifies it ("different path, stored path still exists"). - Across one user's own configs: active rows with the same
ContentHashin two configs owned by the same user.
Duplicates across different users are not reported. They are expected and private.
Scope for v1.0
- Log both kinds at
Informationlevel, structured withContentHash, the config id(s) and the paths, once per crawl. - An API listing or UI for duplicates is out of scope; open a follow-up if wanted.
- A within-config duplicate can first be reported on the crawl after a move. If the old file is gone and two new copies are processed at the same moment, both are treated as the move and the row ends up at one of them. On the next crawl the other copy sees that path still exists and is reported. Expected, not a bug.
Acceptance
- Both kinds are detected and logged with the fields above. Each pair is logged once per crawl.
- Nothing is written to the database.
-
scripts/mutation-changed.shat 100%.
Blocked by #35, which produces the within-config signal.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestready-for-agentFully specified, ready for an AFK agentFully specified, ready for an AFK agentv1.0Required for the v1.0 releaseRequired for the v1.0 release
Activity
[-]4.4 Implement periodic deduplication scan[/-][+]Implement periodic deduplication scan[/+][-]Implement periodic deduplication scan[/-][+]Report duplicate content (report-only; never merges or deletes)[/+]feat(worker): Report duplicate content, never merge it (#36) (#291)