Vendored requirements.txt: `vendor --revert` / `remove` delete the vendored wheel while a `-r` include still points at it (exit 0), so every later `pip install -r requirements.txt` fails

[mikolalysenko]

[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).

Summary

revert_requirements has a guard for vendor lines that drifted: when a vendor line no longer matches its ledger record, the revert leaves it in place, and a residual-reference sweep keeps the .socket/vendor/pypi/<uuid>/ artifact so the line still installs (vendor_revert_residual_reference → vendor_artifact_kept / vendor_revert_kept). That works when the drifted line is still in the file the ledger recorded.

The sweep only scans the files named in the ledger's wiring records, though. If the user moves the vendored line into a -r include (for example while splitting requirements.txt into requirements/base.txt), the revert:

1. skips the recorded requirements.txt:1 record as vendor_revert_line_drifted;
2. sweeps only requirements.txt, which no longer mentions the uuid, so it finds no residual reference;
3. deletes .socket/vendor/pypi/<uuid>/ (and all of .socket/) and exits 0 with status: success.

requirements/base.txt still carries ./.socket/vendor/pypi/<uuid>/six-1.16.0-py2.py3-none-any.whl # socket-patch vendor: six==1.16.0, so pip fails on every install from then on. remove pkg:pypi/six@1.16.0 goes through the same path and gives the same result.

This is also the remedy socket-patch itself prescribes. In the same layout, a superseding patch's re-vendor fails with pypi_requirements_already_vendored ("requirements/base.txt: already routes six … (requirements.txt: the vendor line changed since vendoring); run socket-patch vendor --revert before re-vendoring"). Following that advice breaks the project.

Impact

After a vendor --revert or remove that reports success, pip install -r requirements.txt fails on every machine and in CI with OSError: [Errno 2] No such file or directory: '…/.socket/vendor/pypi/<uuid>/six-1.16.0-py2.py3-none-any.whl'. The wheel is gone, and the ledger entry is too, so neither vendor --revert nor repair can recover it. The user has to restore it from git or hand-edit the include.

Repro (Linux, main 99f61d2, pip 26.2.1 and 24.0 / CPython 3.11)

This uses a local mock of the patch API and vendoring service (batch / by-package / view/<uuid> plus prebuilt_common::mount_view_from_source over the real installed six.py), run through a throwaway driver that wasn't committed.

python3 -m venv venv && venv/bin/pip install six==1.16.0 idna==3.7
export VIRTUAL_ENV=$PWD/venv
printf 'six==1.16.0\nidna==3.7\n' > requirements.txt
F="--json --yes --api-url $MOCK --api-token fake --org test-org --vendor-url $MOCK --patch-server-url $MOCK"
socket-patch scan --mode vendored $F     # exit 0; line 1 -> ./.socket/vendor/pypi/<uuid>/six-…whl  # socket-patch vendor: six==1.16.0

# The user moves the vendored line into an include (pip resolves it exactly as before)
L=$(head -1 requirements.txt); mkdir requirements
printf -- '-r requirements/base.txt\nidna==3.7\n' > requirements.txt
printf '%s\n' "$L" > requirements/base.txt
socket-patch scan --mode vendored $F     # exit 0, "already_vendored: artifact and lockfile wiring already in sync"

socket-patch vendor --revert --json --yes
# exit 0, status success
# events: skipped vendor_revert_line_drifted ("requirements.txt: the vendor line for requirements.txt:1 changed since vendoring; left untouched"), removed
# no vendor_revert_residual_reference; .socket/ is gone
cat requirements/base.txt                # still ./.socket/vendor/pypi/<uuid>/six-1.16.0-py2.py3-none-any.whl …
python3 -m venv fresh && fresh/bin/pip install -r requirements.txt
# ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '…/.socket/vendor/pypi/<uuid>/six-1.16.0-py2.py3-none-any.whl'   (exit 1)

socket-patch remove pkg:pypi/six@1.16.0 --json --yes in the same state gives the same result: exit 0, vendor_reverted plus vendor_revert_line_drifted, and the artifact is deleted.

Control (same file): when the vendored line stays in requirements.txt but has drifted, because the trailing comment was edited or a space was dropped before #, the revert correctly reports vendor_revert_residual_reference, vendor_artifact_kept and vendor_revert_kept, and the wheel stays. So the guard exists. It just doesn't look past the recorded files.

Expected vs actual

The function's own contract (pypi_requirements.rs, doc comment of revert_requirements): "any surviving reference to the vendored uuid dir afterwards raises vendor_revert_residual_reference", and the sweep comment: "a leftover line pointing at the (about to be deleted) uuid dir would break installs." The #786 in-use probe (requirements_entry_in_use) already treats the requirements tree as "the root requirements.txt plus in-root -r includes".

• Expected: the residual sweep covers the same tree (root plus reachable in-root -r includes). The moved line keeps the artifact (vendor_revert_residual_reference, vendor_artifact_kept), so pip keeps installing.
• Actual: only the recorded files are swept. The artifact is deleted under a live reference, with exit 0.

OS × version

OS	pip / Python	moved into -r include → vendor --revert	→ remove	drifted line in root file (control)
Linux	26.2.1 / 3.11	reproduces (×2)	reproduces	artifact kept (correct)
Linux	24.0 / 3.11	reproduces (pip install fails the same way)	—	—
macOS / Windows	—	not probed: the logic is pure text and path joining with forward-slash keys; pip fails on a missing file on every OS	—	—

First bad version

Not bisected. The sweep has only ever covered reverted (the recorded files).

Suspect code

• crates/socket-patch-core/src/vendor/pypi_requirements.rs:462-470: the residual-reference sweep iterates &reverted, which holds only the files named in entry.wiring. It should also scan the rest of the requirements tree (requirements_include_names(root), the same walk the Vendored requirements.txt after the user removes or bumps a vendored pin: the rescan re-adds the removed package as a "(transitive)" line (exit 0), or exits 1 forever after a bump, and scan --prune never reverts the entry #786 in-use probe uses) before it allows the artifact to be deleted.
• The same tree-wide check probably belongs in plan_rewire (pypi_requirements.rs:~697), whose doc comment says "present verbatim in an editable file of the tree" but only looks in the recorded file. That's why the superseding re-vendor sends the user to vendor --revert in the first place.

[mikolalysenko]

[agent] Triaged as priority:p1 (pip). Still present on main 0d302dc: the residual-reference sweep at the end of revert_requirements (vendor/pypi_requirements.rs, for (file, content) in &reverted) scans only the files named in the ledger's wiring records, never the in-root -r include tree that requirements_entry_in_use walks. So a vendor line moved into an include goes undetected, and the artifact is deleted. This isn't a duplicate, and no open PR addresses it yet.

---

Generated by Claude Code

[mikolalysenko]

[agent] More information (main 9c43dfc, real pip 26.2.1 / py3.11): the same failure also happens when the vendored line is moved into a sibling file that requirements.txt does not include. A fix that only walks the -r tree from the root won't cover this case.

# after `socket-patch vendor` on `six==1.16.0`
L=$(cat requirements.txt)
printf 'idna==3.7\n' > requirements.txt
printf '%s\n' "$L" > requirements-dev.txt         # the vendor line now lives only here
socket-patch vendor --check    # exit 1: "wiring missing: no lockfile or config references .socket/vendor/pypi/<uuid> any more"
socket-patch vendor --revert   # exit 0, `vendor_revert_line_drifted`; .socket/vendor/pypi/ is deleted
pip install -r requirements-dev.txt   # ERROR: [Errno 2] No such file or directory: '…/.socket/vendor/pypi/<uuid>/six-1.16.0-py2.py3-none-any.whl'

remove pkg:pypi/six@1.16.0 behaves the same way (exit 0, vendor_reverted + vendor_revert_line_drifted, wheel deleted). Reproduced twice. A scan --mode vendored in between re-adds a root six (transitive) line, which is fine, but a later revert still deletes the wheel under the requirements-dev.txt reference.

A related cosmetic problem shows up in the same output: the drift warning prints a Rust Debug value, requirements.txt: the vendor line for Some("requirements.txt:1") changed since vendoring; left untouched (vendor/pypi_requirements.rs drift_warning, {:?} of rec.key). The same pattern is in vendor/common.rs:806, pypi_uv.rs:847 and pypi_pipenv.rs:468.

---

Generated by Claude Code

[mikolalysenko]

[agent] Shares root cause with #996: a wired PyPI revert deletes .socket/vendor/pypi/<uuid>/ without first probing every Python project file for the uuid dir. The requirements flavor sweeps only its own recorded files, and revert_pypi_opts runs the shared probe (unwired_pypi_reference_clause, which already walks the -r include tree) only for unwired entries. Will be fixed together.

---

Generated by Claude Code

[mikolalysenko]

[agent] Claiming this issue (with #996; shared root cause: a wired PyPI vendor revert deletes the vendored wheel without probing every Python project file for the uuid dir). Branch: agent/fix-pypi-revert-residual-probe. Claim-ID: 2026-10-07T09:20:53Z-6884c3

---

Generated by Claude Code

[mikolalysenko]

[agent] Draft PR: #997

---

Generated by Claude Code