v5 tests: one suite per command, retire #257 oracles

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:Claude Opus 5.5

---

Workstream W2 of the v5 waste-review follow-ups (#286).

Finding	Outcome
F03: delete #257 "verbatim previous implementation" oracles after golden snapshots	done (partial). The rewriter and parser oracles are deleted: hosted npm, yarn classic, pnpm, cargo lock, composer, golang, uv/python lock, pdm plan, pdm/poetry parse reuse, and the reference POM parser. Each seeded sweep now replays against per-case golden digests (crates/socket-patch-core/src/golden.rs, tests/equivalence/*.golden). The goldens were blessed while the oracles still asserted equality (a19da46, then d61f6ba). Kept: the crawler oracles (cargo, composer, go, maven, npm, nuget, python). A crawler's output follows the filesystem's readdir order, and every crawler golden blessed on ext4 failed when replayed on tmpfs. A golden therefore can't pin them portably, whereas the oracle walks the same tree on every OS (912d005). Also kept: about 200 LOC of small memo-vs-fresh and pool-vs-serial oracles over real I/O (copy_tree, cargo_lock memo, reuse, go_mod_edit, common repack, verify, registry_fetch, pypi_wheel), plus group_equivalence's 38-line serial_oracle chain-order spec.
F04: collapse the covgap_/coverage_fix_/in_process_ categories into one suite per command	done (partial). Category files for apply, get, remove, rollback, repair, scan, vendor, update and the global CLI now live in tests/<cmd>/, one binary per command. Files keep their names as modules for traceability. Deferred: the 82 in-process or env-mutating binaries (in_process_*, most covgap_commands_*, cli_parse_*). They share process-global state (notice SILENT/JSON, the socket-cli config OnceLock, the API client's *_SHOWN flags), so merging them changes test isolation until those statics are made resettable.
F58: cut the CLI integration-test binaries by merging small ones	done. 67 subprocess-only binaries → 9 suites, so the CLI goes from 223 → 165 test binaries. Kept separate: every binary a workflow names (64), docker_e2e_*, and crawl_fd_limit_e2e (process-global walk pool).
F10: dedupe the CRLF/corrupt/dry-run/round-trip invariants into table-driven tests	deferred. Re-verified: the ~17 vendor backends each build their own fixture (blobs, installed tree, record, run fn), and no shared fixture interface exists at the LockModel level. Table-driving them needs a new cross-backend fixture trait, which is a design change for its own PR. It would save LOC, but no CI time: all of these tests live in the single core lib binary.

Measured savings

• LOC (.rs, this PR against the current base, merges excluded): +794 / −3334, net −2540, plus 21 golden data files (+4516 lines of digests).
• CLI test binaries: 223 → 165, i.e. 58 fewer full-CLI links.
• Tests: all 1062 test runs of the 67 old binaries map to a test in the new suites.
  • The 596 file tests are preserved one-for-one.
  • The other 466 were common/update_fixture/cache_env helper self-tests that ran again in every binary; they now run once per suite.
  • v5: remove dead code and v3 compatibility shims #296 later removed 6 --one-off tests from these files; that removal is v5: remove dead code and v3 compatibility shims #296's change.
• CI job-minutes: this PR's CI run 36498252115 on dbb5848 took 294.5 job-min over 186 jobs. Run 36525717340 on 5e16953 took 288.4.
  • Same-base comparison: the post-v5 CI: build e2e binaries once and tier the PM matrix #291 v5: socket.yml patch rollout config and filtering #293 run 36497388949 took 298.9 job-min over 186 jobs.
  • That's −4.4 job-min (−1.5%), which is within run-to-run noise. The test jobs total 78.5 vs 78.7 min.
  • The pre-v5 CI: build e2e binaries once and tier the PM matrix #291 base run 36474481876 took 818.6 job-min over 233 jobs. v5 CI: build e2e binaries once and tier the PM matrix #291's tiering explains that drop, not this PR.
  • So the binary merge saves link time and disk, but no measurable wall time in CI.
• Line coverage (host cargo llvm-cov, on dbb5848): 97.35% → 97.35%, with missed lines 6161 → 6136.
  • Lines: base 232,260 total / 6161 missed; this PR 231,573 total / 6136 missed.
  • The total shrinks by the deleted test-only oracle code.
  • Functions went 96.46% → 96.49% and regions stayed at 97.20%.

CI status on fe293ed (after merging #292 and #296)

• Green: CI test (all 3 OSes), release test, coverage and docker coverage; PDM, Bun, pnpm, npm, Go and Poetry compatibility.
• Earlier heads each had one PDM native leg flake and pass on its single re-run: 2.3.4 optional agent on dbb5848, and 2.17.3 direct hosted on 5e16953. PDM is green on fe293ed.
• Red, base-inherited:
  • yarn-classic 1.0.2 / 1.6.0 / 1.7.0 / 1.9.4. Also red on the base run 36474481876 and on v5: socket.yml patch rollout config and filtering #293 after v5 CI: build e2e binaries once and tier the PM matrix #291.
  • vlt install-proof: ubuntu 0.0.0-1, 0.0.0-1 on node 22.0.0, 0.0.0-11 and windows 0.0.0-11. On ubuntu the e2e_redirect_vlt_build hosted rollback legs fail; the same jobs are red on v5: cap new patches per scan, most critical first #294.
  • See the comments below. Neither area is touched by this PR, and no fix exists yet.
• v5: remove dead code and v3 compatibility shims #296 merge: v5: remove dead code and v3 compatibility shims #296 drops DepOverride.berry_zip_url, so the goldens' input digests were re-blessed. 1848 lines in 12 files changed only their input digest, and every case key and output digest is unchanged (fe293ed).

Test evidence (local, Linux)

• socket-patch-core lib on fe293ed: 4492 passed as non-root, with TMPDIR on tmpfs as well as on ext4, so the goldens are filesystem-independent.
  • As root, 4 known root-only permission tests fail, and they fail identically on 14a9cb0.
  • On 14a9cb0 the stage_local_artifact peak-RSS test also fails as root; it inherits the parent's RSS under a full parallel run.
• Goldens replay deterministically across repeated runs as root and as nobody.
• The 9 merged CLI suites pass. The only failures are 2 root-only "unremovable file" repair tests; they fail identically in the pre-merge binaries, and repair passes 116/116 as non-root.
• Before the base merges, the suites ran 3 rounds with the same results each time.
• After the v5: remove dead code and v3 compatibility shims #296 merge, each moved CLI test file equals v5: remove dead code and v3 compatibility shims #296's version plus only this PR's mechanical #[path]/use crate:: edits.
• cargo clippy --workspace --all-features -- -D warnings is clean, and --all-targets is clean for core and CLI.
• scripts/tests (118 tests) pass.
• Deletions: no remaining references to any removed oracle symbol or file (grep), and the build is clean.
• Adversarial review ran with four parallel reviewers: correctness, coverage lost, CI equivalence, and owner decisions.
  • The crawler-golden portability finding came from that review and led to 912d005.
  • No workflow, script or path filter names a moved binary.
  • The owner decisions hold: vlt and PM-version coverage untouched, no setup or hosted ledger, no maven/nuget vendoring code changed.

Notes for reviewers

• Golden digests cover the Debug/JSON form of each result, so a dependency bump that rewords an error, a new lock fixture, or a changed DepOverride shape needs a re-bless (SOCKET_PATCH_BLESS_GOLDEN=1). Review that diff like any golden change: for a pure input-shape change, only the input-digest column should move.
• Three env-gated benchmark oracle tests are deleted: SOCKET_PATCH_{COMPOSER_FIXTURE,GO_FIXTURES,PY_LOCK_FIXTURES}. No CI job sets these.

🤖 Generated with Claude Code

https://claude.ai/code/session_01N9YdJqaGiT9Jf5LN1hDFhB

[Mikola Lysenko (mikolalysenko)]

[agent] vlt patch compatibility / install-proof is red for vlt 0.0.0-11 (ubuntu and windows) and 0.0.0-1 (ubuntu). The ubuntu 0.0.0-11 job failed on 912d005 (job). All three failed again on dbb5848: ubuntu 0.0.0-1, windows 0.0.0-11.

On ubuntu, three hosted tests in e2e_redirect_vlt_build fail: crlf_lock, idempotence and rollback_byte_exact. In each, rollback doesn't restore the lock's original tarball URL. On windows the same job exits 1. The tail of its log shows only the later suites passing; e2e_redirect_vlt_build runs before them, so I expect the same three tests but haven't confirmed that from the log.

This isn't this PR's failure:

• The PR doesn't touch e2e_redirect_vlt_build or any rollback/redirect production code. Only test modules and goldens change.
• The workflow ran here only because path filters matched the test-only core changes.
• The same failure shows up on v5: cap new patches per scan, most critical first #294 (run 36492370868), whose base includes v5 WS4/WS6: one hosted engine for disk + memory; unified Ledgers view #282.

I haven't found a fix to port yet. I merged the new base (#291) in dbb5848, and the jobs still fail there.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] PDM patch compatibility / native (ubuntu-latest, 2.3.4) is red on dbb5848 (job). One cell fails: optional agent. Its checks rollbackExit0, rollbackRestoresUpstreamBytes and rollbackClearsManifest all fail. The other 35 cells pass or refuse as expected.

I don't think this is from this PR. The same workflow passed on the previous head 912d005 (run 36497447814) and on #293 and #294, which have the same base. The only change since 912d005 is the base merge, which touches CI, scripts and e2e_nuget.rs, not the CLI or the pdm code. I'll re-run the failed job once when the run finishes. A second failure would make it real, and I'll root-cause it then.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] CI / yarn-classic 1.7.0 is red on dbb5848 (job). It's base-inherited: yarn-classic 1.0.2, 1.6.0, 1.7.0 and 1.9.4 all failed on the base's last full CI run for #282's head (run 36474481876), before this PR existed. This PR doesn't touch yarn-classic code or its e2e tests. I haven't found a fix to port.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] ready to land

CI on dbb5848 is green apart from base-inherited failures:

• yarn-classic ×4, also red on the base run and on v5: socket.yml patch rollout config and filtering #293.
• vlt 0.0.0-1/0.0.0-11 install-proof, also red on v5: cap new patches per scan, most critical first #294.

The one PDM flake passed on re-run. There's no merge conflict. Coverage is 97.35% → 97.35%, with 25 fewer missed lines. The findings table, measured savings and test evidence are in the description.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#292 landed on release/v5-prerelease as a7b0d00; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] Merged release/v5-prerelease at a7b0d00 (#292) into 5e16953. There were no conflicts, and #292 doesn't touch any of the moved test files.

Local checks:

• workspace clippy is clean;
• the merged apply, get, scan and vendor suites pass;
• repair passes 116/116 as non-root (the only two root-run failures are the known "unremovable file" tests).

CI is re-running on the new head. The PR stays ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] PDM patch compatibility / native (ubuntu-latest, 2.17.3) is red on 5e16953 (job). One case fails: direct hosted, on the appliedExactlyOne check. It was the first case in the run. The other 37 cases pass, and 6 refuse or are unsupported as expected.

This PR changes only test code and goldens, but the base merge brought in #292, which changes the hosted fetch path (api/client.rs, vendor_prefetch.rs, vendor/pypi.rs). So the cause is either a flake or something from #292. The same workflow is running now on #293, #294 and #296, which merged #292 too. I'll compare against those and re-run this job once when the run finishes.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#296 landed on release/v5-prerelease as 1e3ace6; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] Merged release/v5-prerelease at 1e3ace6 (#296) as fe293ed.

Conflicts: two, in composer_equivalence_tests.rs and golang_equivalence_tests.rs. #296 edited the env-gated fixture tests that this PR deletes, so I kept them deleted. #296's other changes to the moved CLI test files (the --one-off removals) came across through rename detection. After the merge, each moved file equals #296's version plus only this PR's mechanical #[path]/use crate:: edits.

Goldens re-blessed: #296 removes DepOverride.berry_zip_url, and the golden inputs serialize the deps, so their digests changed. Across 12 golden files, 1848 lines changed only in the input-digest column. Every case key and every output digest is byte-identical, so the rewriters' behavior is unchanged.

Local checks:

• core lib: 4492/4492 as non-root on tmpfs;
• the 9 merged CLI suites pass (as root, only the 2 known root-only repair tests fail);
• workspace clippy and --all-targets clippy are clean.

CI is re-running. The PR stays ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Coordinator: #297 is next in the landing order (#287 is still a draft and is skipped). It is blocked only because there is no [agent] ready to land comment newer than the last push, fe293ed (07:28Z). The branch contains the current base (1e3ace6). Once CI on fe293ed finishes, post [agent] ready to land with its results and it will land on the next run.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] ready to land

CI on fe293ed has finished. The branch contains the current base (1e3ace6, #296).

• Green: CI test (ubuntu, macOS, windows), release test, coverage and docker coverage; PDM, Bun, pnpm, npm, Go and Poetry compatibility.
• Red, base-inherited:
  • yarn-classic 1.0.2 / 1.6.0 / 1.7.0 / 1.9.4, also red on base run 36474481876 and on v5: socket.yml patch rollout config and filtering #293.
  • vlt install-proof for 0.0.0-1 (ubuntu, and ubuntu on node 22.0.0) and 0.0.0-11 (ubuntu, windows), also red on v5: cap new patches per scan, most critical first #294.
  • This PR doesn't touch either area.
• No merge conflict. Local checks: core 4492/4492 as non-root, and the merged CLI suites pass. Golden output digests are unchanged across the v5: remove dead code and v3 compatibility shims #296 re-bless.

---

Generated by Claude Code