fix(code-index): a replaced engine no longer drops changes made since the last index run

[wongk]

Code search stops seeing changes whenever its engine is replaced

The server's engine cache replaces a repo's CodeContextEngine every time the index version moves, and every re-index moves it. That's 8ea1eb15, which stops a server from serving an old index version forever.

The replacement engine's autosync loop took its starting point from the working tree as it found it. On its first tick it:

• recorded the current HEAD as the baseline;
• recorded the current tree signature as already indexed (bootstrap / seed_signature).

So any change made between the old engine's last check and the new engine's first tick was treated as already indexed. That includes a commit, a checkout, or a file written outside LemonCrow's edit. It stayed out of the index until some later change happened to touch the same files.

In a busy session the index version moves on almost every edit, so this happened constantly.

Found in practice. A worktree's index kept engine.py from Sep 22 through 7 HEAD moves and 3 days. code_search could not find constants that were plainly in the file, and code_coverage_check reported the file stale.

Reproduced with the installed build. In a scratch repo I committed a new function while a loop searched every 2 s, as a working session does. The search went through mcp_server._code_context_engine. The function never became searchable. The first search after the commit got a new engine. That engine's first tick recorded the post-commit HEAD and tree as its baseline and re-indexed nothing.

Server restarts dropped changes the same way.

The fix: the index records what it covers

• Every index run records its baseline. Each completed index_repo run stores the tree signature and HEAD it started from in engine_state, under autosync_baseline:<repo_id>. The key is per repo because several repos can share one database.
• The baseline is read before the scan. A change landing mid-scan then differs from the baseline, and the next check re-indexes it. Recording it after the scan would mark that change as done.
• Autosync measures against that baseline. It no longer uses its own first look at the tree. A fresh engine, whether a replacement or a new process, re-indexes on its first tick if HEAD or the tree has moved since the last index run.
• After each re-index the loop adopts the baseline that run recorded. It no longer re-reads the tree. So the server process skips its extra stat walk after a re-index; the walk moves into the index run.
• An index with no recorded baseline is re-indexed once, incrementally. That covers any index built before this change, and the event reason is no_indexed_baseline.

Cost. Each index run now walks the source tree once more to compute the signature. That's 1.25 s on a 17.7k-file repo, against a full rebuild of about 19 s. For autosync re-indexes the total is unchanged, because the server no longer walks the tree after each one.

Also: Zoekt snapshot thread crash in linked worktrees

The crash:

• ZoektIndexer stores its line-count snapshot at <repo>/.git/lemoncrow/zoekt_snapshot.json and reads HEAD from <repo>/.git/HEAD.
• In a linked worktree .git is a file, so loading the snapshot raised NotADirectoryError, which the loader didn't catch.
• That killed the lemoncrow-zoekt-snapshot thread and printed a traceback every time a worktree engine started.
• HEAD also always read as None in a worktree, so the saved snapshot could never be reused.

The fix:

• The indexer now uses zoekt/server.py's existing _git_dirs() / _read_git_head(), which already handle linked worktrees.
• The snapshot now lives in the checkout's own git admin directory. For a worktree that's .git/worktrees/<name>/, which git worktree remove deletes along with the worktree.
• The loader now treats any OSError as "no snapshot".

Tests

New:

• test_a_replacement_engine_indexes_a_commit_made_since_the_last_index_run
• test_a_replacement_engine_indexes_a_working_tree_change_made_since_the_last_index_run
• test_a_change_made_during_an_index_run_is_reindexed_by_the_next_check
• test_an_index_without_a_recorded_baseline_is_reindexed_on_the_first_check
• test_a_linked_worktree_keeps_its_snapshot_in_its_own_git_admin_directory
• test_a_linked_worktree_snapshot_is_rebuilt_once_its_head_moves

All six fail against the old engine and indexer and pass with this change.

The existing autosync test probe now records a baseline the way a real index run does. The existing autosync tests pass unchanged (15 passed).

Other checks:

• The original scenario, rerun against this branch: the server's engine cache, a commit, and a search every 2 s. The replacement engine re-indexed on its first tick (head_moved), and the commit became searchable.
• Full suite (-m 'not slow'): 6972 passed, 9 failed, 2 errors.
  • Seven of the failures also fail on main: 5 in test_daemon_ownership_races.py, test_mcp_read_batch_budget, test_savings_credit, and test_index_pool_does_not_fork_live_parent_state.
  • test_edit_mcp_handler::test_workspace_root_edit_reindexes_once_after_the_response failed under parallel load and passes on its own.
  • The 2 test_retriever_eval setup errors come from lemoncrow init in a worktree reading a prepare-commit-msg hook that a concurrent test run had just written into the main checkout's shared .githooks. They pass on rerun, and this change doesn't touch that path.
• ruff, black and mypy are clean on the changed files.