Prevent worktrees.json registry loss under concurrent reads and writes

[germanescobar]

Summary

The shared Controller worktree registry can be catastrophically reset when a read overlaps a write.

During cleanup of merged worktrees in the Coliseo project, Controller initially listed 45 secondary worktrees. Several sequential deletions were issued through the supported CLI while the Controller UI was open. Shortly afterward, every secondary worktree disappeared from Controller across the project, including worktrees that were not deletion targets.

The directories for 20 existing worktrees were still present and registered with Git, but their Controller records were gone. Seven of those directories contained uncommitted work.

Evidence

• Before cleanup, controller worktrees list Coliseo returned the main checkout plus 45 managed worktrees.
• After the failure, the same command returned only a newly generated main entry.
• Git still reported 20 secondary worktree directories.
• worktrees.json showed newly generated main-only records for many projects with timestamps clustered between 2026-08-06T06:57:20.043Z and 2026-08-06T06:57:20.085Z.
• This affected projects unrelated to the deletions, which strongly suggests concurrent lazy main-entry recreation after the registry was observed as empty.
• A worktree created afterward was persisted normally.
• The surviving Coliseo records had to be reconstructed manually using their original IDs, paths, branches, and port offsets.

A backup of the post-failure registry was retained locally as worktrees.json.bak.20260806-recovery.

Likely cause

The registry implementation currently has three unsafe behaviors:

1. writeRegistry() writes directly to worktrees.json with fs.writeFile(), which truncates the destination before the new JSON is fully written.
2. Registry read-modify-write operations are not serialized.
3. readRegistry() catches every read or parse error and returns [].

A concurrent reader can therefore observe an empty or partially written file, interpret the parse failure as an empty registry, and persist a new registry containing only a lazily created main record. Concurrent UI reads can repeat this for multiple projects.

Reproduction direction

1. Populate worktrees.json with multiple projects and worktrees.
2. Keep UI clients active so they issue concurrent project/worktree reads.
3. Rapidly issue supported worktree delete or create operations.
4. Race a registry read against the truncate/write window.
5. Observe a parse failure becoming [], followed by main-only registry writes.

A deterministic test could inject a delay between file truncation and content write, then issue getProjectWorktrees() concurrently.

Expected behavior

• A transient read or JSON parse failure must never be treated as a valid empty registry.
• Registry mutations must not lose unrelated records.
• Deleting worktrees in one project must not affect registry entries in any other project.

Proposed safeguards

• Serialize all registry read-modify-write operations with a process-level mutex or queue.
• Write to a temporary file in the same directory, flush it, and atomically rename it over worktrees.json.
• Return [] only for an explicit ENOENT first-run case. Surface parse and I/O errors without writing replacement state.
• Keep a last-known-good backup and support automatic recovery from invalid JSON.
• Add concurrency tests covering getProjectWorktrees, addWorktree, updateWorktree, and removeWorktree.
• Add a reconciliation command that can safely re-register existing git worktree list --porcelain entries while preserving known worktree IDs and sessions.
• Emit or persist sufficient audit information to reconstruct registry records after failure.

Additional deletion safety

The delete route uses git worktree remove --force followed by recursive filesystem removal after checking only for an active runtime session. A paused worktree can still contain uncommitted changes. Before destructive removal, Controller should refuse dirty worktrees unless an explicit force confirmation is supplied, and should create a recoverable snapshot or archive when force deletion is approved.

Impact

This failure makes active conversations disappear from the UI and orphans valid Git worktrees. If a targeted worktree contains paused uncommitted changes, the forced deletion path can permanently remove those files. In this incident, all seven dirty surviving worktrees were recovered and re-registered, but the risk of unrecoverable paused work is significant.