Repository navigation
Update the row when a file's content changes at the same path #250
Copy link
Copy link
Closed
Labels
ready-for-agentFully specified, ready for an AFK agentFully specified, ready for an AFK agent
Description
Problem
A file edited in place (re-exported, rotated and saved, metadata stripped) fails ingestion on every crawl.
- Its new bytes have a hash no row carries, so
IdempotencyCheckMiddlewarepasses it on as new. PersistenceMiddlewareinserts a row at the same(StorageConfigId, FilePath)as the existing one and hitsIX_MediaAssets_StorageConfigId_FilePath(MediaAssetIndexNames.FilePathUnique). That violation is rethrown, so the item fails, and fails again on the next crawl.- The old row keeps pointing at the path but describes bytes that are no longer there. Once originals are served (Implement on-demand original file serving endpoint #44), the thumbnail and the original disagree.
#35 adds a second way in: if a copy of the old content exists elsewhere, the old path still exists, so that copy is classed as a duplicate and the row stays on the wrong bytes.
Decision
Update the existing row in place. An edited photo is the same memory, so it keeps its Id and its interactions (stars, likes, hidden).
Behaviour
When the hash is unknown but an active row already holds (StorageConfigId, FilePath):
- Run the item through the pipeline as usual (EXIF, proxies). At persistence, update that row instead of inserting: new
ContentHash,FileName, date fields (DateCaptured,Month,Day,Year),MediaType,Metadata, andLastSeenOnNas. - Replace its
ProxyFilerows with the newly generated ones. The new proxies live in the new content's directory (IProxyDirectoryStrategykeys by content). The old directory is orphaned; delete it after the save commits, or leave it to feat(worker): Orphaned proxy file sweeper (proxies ↔ DB reconciliation) #174, and say which. - Keep
Id,Interactions,BurstIdandPairedAssetId. A Live Photo whose HEIC was edited stays paired to its MOV. - Once the row carries the new hash, a copy of the old content elsewhere no longer finds a row by hash, so it ingests as a new asset. Moves (Re-link moved/renamed files and refresh LastSeenOnNas on every crawl #35) behave as before.
Out of scope: a soft-deleted row at the path also blocks the insert, because the FilePath index covers soft-deleted rows. That belongs to restore (#34).
Acceptance
- A file whose bytes change at the same path updates its existing row.
Idand interactions are unchanged; hash, metadata, dates and proxies are new. - The item no longer fails on later crawls.
- Old proxy files are deleted after commit or explicitly left to feat(worker): Orphaned proxy file sweeper (proxies ↔ DB reconciliation) #174, and that choice is documented in CLAUDE.md.
- Pipeline compensation on failure still deletes only this attempt's proxies, never the row's previous ones.
- Tests at the middleware seam.
-
scripts/mutation-changed.shat 100%.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
ready-for-agentFully specified, ready for an AFK agentFully specified, ready for an AFK agent
Activity
refactor(worker): Classify each crawled file once, then one handler p…
bitbiter-dev commentedon Oct 18, 2026
Scope note from the 2026-10-18 architecture review (#275)
PR #274 planned #250 as "one new outcome plus its writer, no middleware gains a branch". Following a
ContentChangedfile through master (917f973) shows one more seam change that this issue has to include.IngestionContext.AssetIddoes two jobs. It is set to a new GUID when the item is queued (FileIngestionPipeline), and it is used as:ProxyStagingWriterand the temporary-file cleanup inIngestionPipelineRunner.Compensate;ImageProxyMiddlewareandVideoProxyMiddlewarepass it asownerAssetId, andNewContentWriteruses it as the new row'sIdand to pick that row's proxies.For
ContentChanged, the proxies belong to the stored row, whose id is only known after classification (order 20, before the proxy middlewares at 40 and 50). RewritingProxyFile.AssetIdinside the writer would make the context's registrations, andReleaseProxyFiles(ownerAssetId), disagree with what is saved. So:IngestionContext. Classification sets the owner: the stored row's id forContentChanged, a new id forNew. The proxy middlewares andNewContentWriterread the owner, while temporary files and compensation keep the attempt id. This changes what the middlewares read, but adds no branch to them.Two more things to settle while implementing:
ProxyFileper type, andProxyFileIndexNames.OnePerAssetAndTypeis unique. Updating the existing rows'ProxyPath,SizeBytesandCreatedAtavoids a delete and insert of the same key in one save. It also keepsLostIngestionRacefrom reading this writer's own collision as "lost a race", which is only true for new assets.Month,DayandYearmust be set again. The review's grouped issue moves that rule into one place first, so this issue calls it instead of making a third copy.Design part unchanged. This is a seam inside the Worker, with no new abstraction or contract.
Merge master (#288) into feat/250-content-change-in-place
feat(worker): Update the row in place when a file's content changes a…