Skip to content

Detect same-timestamp source changes with verified hashes#1051

Merged
TheGreenCedar merged 1 commit into
dev/codestory-nextfrom
codex/911-content-freshness
Jul 13, 2026
Merged

Detect same-timestamp source changes with verified hashes#1051
TheGreenCedar merged 1 commit into
dev/codestory-nextfrom
codex/911-content-freshness

Conversation

@TheGreenCedar

Copy link
Copy Markdown
Owner

Context

Closes #1050
Refs #911
Refs #899

CodeStory's incremental refresh planner decides whether a stored file projection must be rebuilt. #978 made parser-backed projections carry a verified SHA-256 identity for the source bytes that produced them, but planning still treated a matching millisecond modification time as unchanged. Multiple writes inside one timestamp tick—or a coarse filesystem clock—could therefore leave stale graph data current even though the store had a stronger identity available.

This PR owns only the planner-side same-timestamp gap. It does not close the broader freshness and completeness contract in #911.

Outcome

When a parser-backed row has a verified source hash and its mtime matches, refresh planning now verifies the current bytes and schedules reindexing on a mismatch or unstable read. Rows without verified parser identity preserve the existing metadata fallback.

What changed

  • Carried nullable parser content hashes through the compact store-to-workspace inventory contract and loaded them in one bulk SQLite query.
  • Extracted a small freshness helper that keeps the existing unindexed, retry, unreadable, and mtime checks before streaming SHA-256 verification in 64 KiB chunks.
  • Limited content reads to rows that already contain a verified parser hash, leaving legacy, structural, text-only, and oversized-file behavior unchanged.
  • Added one same-timestamp behavior regression, reused the existing unchanged-file test for no-spin coverage, and extended the existing store inventory test for hash propagation.
  • Updated the changelog, indexing architecture docs, and all three repository-scale statistics tables.

How to review

  1. Review crates/codestory-contracts/src/workspace.rs and crates/codestory-store/src/file_store.rs for the compact inventory boundary and bulk-load behavior.
  2. Review stored_file_needs_index and current_content_hash in crates/codestory-workspace/src/lib.rs for precedence, fallback, and fail-closed reads.
  3. Review the focused regression and the 72c674e5+1050wt rows in docs/testing/codestory-e2e-stats-log.md for behavior and measured evidence.

Verification

  • cargo fmt --all -- --check — passed.
  • cargo test -p codestory-contracts --locked — 28 passed.
  • cargo test -p codestory-store --locked inventory_retries_file_errors_but_not_parser_partial_coverage — passed.
  • cargo test -p codestory-workspace --locked incremental_refresh_detects_changed_content_with_matching_mtime — passed.
  • cargo test -p codestory-workspace --locked incremental_refresh_uses_millisecond_precision_for_unchanged_files — passed.
  • cargo check --workspace --locked — passed.
  • cargo test --workspace --locked — passed on macOS after rebasing onto merged PR Make macOS a release-gated supported platform #1048.
  • cargo clippy --workspace --all-targets --all-features --locked -- -D warnings — passed.
  • cargo build --release -p codestory-cli — passed.
  • Mandatory ignored codestory_repo_e2e_stats gate — 2 passed with proof_tier=full_sidecar, explicit CPU allowance, and the real-repository drill intentionally skipped. The run recorded 19.72 s initial indexing, 17.86 s repeat refresh, 31.65 s retrieval indexing, 2.23 s report generation, 148,437 nodes, 123,824 edges, 307 files, zero index errors, and 1,007 dense anchors.
  • node .github/scripts/check-doc-links.mjs — 69 files and 281 links passed.
  • git diff --check — passed.

Risk or follow-up

@TheGreenCedar

Copy link
Copy Markdown
Owner Author

Review context

PR #1051 closes the same-timestamp freshness child #1050 without closing the broader content/completeness program in #911. I independently reviewed exact head 473b4f85cbc0c4cd2102a77c957d4e6c861d4636 against base 72c674e5484b9175214846be3dcc7333105d200b.

Review scope

  • Inspected the compact workspace inventory DTO, the SQLite bulk hash read, refresh-planning precedence, streaming SHA-256 verification, focused regressions, architecture copy, and recorded repo-scale statistics.
  • Confirmed verified hashes cross the store/workspace boundary in one bulk query rather than per-file lookups.
  • Confirmed content reads occur only after the existing indexed/retry/mtime gates and only for rows carrying a verified parser hash; legacy, structural, text-only, and oversized-file rows keep metadata fallback.
  • Confirmed unreadable or unstable verification reads schedule reindexing, while the existing matching-hash regression prevents unchanged-file spin.
  • P1/P2 code findings: none.

Disposition

  • Disposition: keep draft until the two GitHub-hosted macOS rerun jobs finish.
  • Merge condition: macOS workspace cargo checks (macos-x64) and concurrent publication (macos-latest) pass on this exact head; all other substantive hosted checks are already green.

This review validates the code contract and reviewer navigation. It does not replace the pending hosted macOS evidence or turn the disclosed retrieval-timing warning into a release-performance claim.

@TheGreenCedar
TheGreenCedar marked this pull request as ready for review July 13, 2026 13:58
@TheGreenCedar
TheGreenCedar merged commit 9c97808 into dev/codestory-next Jul 13, 2026
27 of 33 checks passed
@TheGreenCedar
TheGreenCedar deleted the codex/911-content-freshness branch July 13, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant