Skip to content

Update the row when a file's content changes at the same path #250

Closed

Description

@bitbiter-dev

Problem

A file edited in place (re-exported, rotated and saved, metadata stripped) fails ingestion on every crawl.

  • Its new bytes have a hash no row carries, so IdempotencyCheckMiddleware passes it on as new.
  • PersistenceMiddleware inserts a row at the same (StorageConfigId, FilePath) as the existing one and hits IX_MediaAssets_StorageConfigId_FilePath (MediaAssetIndexNames.FilePathUnique). That violation is rethrown, so the item fails, and fails again on the next crawl.
  • The old row keeps pointing at the path but describes bytes that are no longer there. Once originals are served (Implement on-demand original file serving endpoint #44), the thumbnail and the original disagree.

#35 adds a second way in: if a copy of the old content exists elsewhere, the old path still exists, so that copy is classed as a duplicate and the row stays on the wrong bytes.

Decision

Update the existing row in place. An edited photo is the same memory, so it keeps its Id and its interactions (stars, likes, hidden).

Behaviour

When the hash is unknown but an active row already holds (StorageConfigId, FilePath):

  • Run the item through the pipeline as usual (EXIF, proxies). At persistence, update that row instead of inserting: new ContentHash, FileName, date fields (DateCaptured, Month, Day, Year), MediaType, Metadata, and LastSeenOnNas.
  • Replace its ProxyFile rows with the newly generated ones. The new proxies live in the new content's directory (IProxyDirectoryStrategy keys by content). The old directory is orphaned; delete it after the save commits, or leave it to feat(worker): Orphaned proxy file sweeper (proxies ↔ DB reconciliation) #174, and say which.
  • Keep Id, Interactions, BurstId and PairedAssetId. A Live Photo whose HEIC was edited stays paired to its MOV.
  • Once the row carries the new hash, a copy of the old content elsewhere no longer finds a row by hash, so it ingests as a new asset. Moves (Re-link moved/renamed files and refresh LastSeenOnNas on every crawl #35) behave as before.

Out of scope: a soft-deleted row at the path also blocks the insert, because the FilePath index covers soft-deleted rows. That belongs to restore (#34).

Acceptance

  • A file whose bytes change at the same path updates its existing row. Id and interactions are unchanged; hash, metadata, dates and proxies are new.
  • The item no longer fails on later crawls.
  • Old proxy files are deleted after commit or explicitly left to feat(worker): Orphaned proxy file sweeper (proxies ↔ DB reconciliation) #174, and that choice is documented in CLAUDE.md.
  • Pipeline compensation on failure still deletes only this attempt's proxies, never the row's previous ones.
  • Tests at the middleware seam.
  • scripts/mutation-changed.sh at 100%.

Blocked by #35. Blocks #44.

Activity

  1. bitbiter-dev commented on Oct 18, 2026

    @bitbiter-dev
    OwnerAuthor

    Scope note from the 2026-10-18 architecture review (#275)

    PR #274 planned #250 as "one new outcome plus its writer, no middleware gains a branch". Following a ContentChanged file through master (917f973) shows one more seam change that this issue has to include.

    IngestionContext.AssetId does two jobs. It is set to a new GUID when the item is queued (FileIngestionPipeline), and it is used as:

    • the attempt id: the temporary-file suffix in ProxyStagingWriter and the temporary-file cleanup in IngestionPipelineRunner.Compensate;
    • the owner of every proxy: ImageProxyMiddleware and VideoProxyMiddleware pass it as ownerAssetId, and NewContentWriter uses it as the new row's Id and to pick that row's proxies.

    For ContentChanged, the proxies belong to the stored row, whose id is only known after classification (order 20, before the proxy middlewares at 40 and 50). Rewriting ProxyFile.AssetId inside the writer would make the context's registrations, and ReleaseProxyFiles(ownerAssetId), disagree with what is saved. So:

    • Separate the attempt id from the owning asset id on IngestionContext. Classification sets the owner: the stored row's id for ContentChanged, a new id for New. The proxy middlewares and NewContentWriter read the owner, while temporary files and compensation keep the attempt id. This changes what the middlewares read, but adds no branch to them.

    Two more things to settle while implementing:

    • Replacing proxy rows. The stored row already has one ProxyFile per type, and ProxyFileIndexNames.OnePerAssetAndType is unique. Updating the existing rows' ProxyPath, SizeBytes and CreatedAt avoids a delete and insert of the same key in one save. It also keeps LostIngestionRace from reading this writer's own collision as "lost a race", which is only true for new assets.
    • Capture date. Month, Day and Year must be set again. The review's grouped issue moves that rule into one place first, so this issue calls it instead of making a third copy.

    Design part unchanged. This is a seam inside the Worker, with no new abstraction or contract.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions