detect_incremental legacy-manifest branch misses backwards-moving mtimes (git checkout, tar restore)

[thejesh23]

Summary

detect_incremental uses current_mtime > stored (a strict greater-than) on the legacy manifest schema and does no content-hash verification. Files whose mtime moved backwards — after git checkout <older-commit>, tar -xf restore, rsync --times, or a manual timestamp adjustment — are silently treated as unchanged and never re-extracted. The graph then reflects the newer content while the corpus on disk holds the older content.

The parallel dict-schema branch handles this correctly (!= + hash-verify). The legacy-schema branch does not.

Where

graphify/detect.py:1583-1610:

# Legacy manifest: plain float value — treat as ast_hash only
if isinstance(stored, (int, float)):
    changed = stored is None or current_mtime > stored
elif isinstance(stored, dict):
    # …
    if stored_mtime is None or current_mtime != stored_mtime:
        # mtime bumped — verify with content hash before re-extracting
        changed = _md5_file(Path(f)) != stored_hash
    else:
        changed = False

Two issues in the float branch:

1. > misses backwards mtime motion. git checkout <older-commit> restores older file timestamps; those files' current_mtime < stored and the check reports changed = False. The stored graph is now stale relative to what's on disk. The dict branch's != catches this.
2. stored is None is dead code. The enclosing isinstance(stored, (int, float)) already excludes None.

The legacy branch also skips the _md5_file verification the dict branch performs, so even when the float mtime does differ it re-extracts without checking whether the AST hash actually changed — the reverse cost of the same asymmetry.

Reproduction

1. Build a graph from a clean checkout at commit A.
2. Copy the manifest into legacy form (single float mtime per file — the shape written by graphify prior to the dict-schema migration).
3. git checkout <older-commit-B> where at least one tracked file's content differs and its mtime is older than at A.
4. Run graphify update (which calls detect_incremental).
5. Observe: new_files does not contain the reverted file; its cached AST from A is retained. The graph now describes commit A's version of a file whose bytes are commit B.

Suggested fix

Align the float branch with the dict branch: compare with != and content-hash-verify before re-extracting:

if isinstance(stored, (int, float)):
    if current_mtime != stored:
        # mtime moved (either direction) — verify with content hash
        changed = _md5_file(Path(f)) != ""    # legacy schema stored no hash;
        # re-extract on any mtime change since we have no hash to compare against
        changed = True if current_mtime != stored else False
    else:
        changed = False

Or, cleaner — normalise legacy floats up-front (already done in the dict branch for the {mtime, hash} shape) so a plain float is treated as {"mtime": stored, "ast_hash": "", "semantic_hash": ""} and falls through to the dict logic, which then correctly re-extracts (empty stored_hash → changed = True) and rewrites the entry in the new schema on the next save.

Happy to open a PR with either shape and a regression test that emulates a backwards-mtime checkout.

[thejesh23]

PR up: #1862.

[safishamsi]

Fixed in e32be08 (ships in 0.9.16), via @thejesh23's #1862. Legacy-manifest branch now re-extracts on backwards mtime motion.