Vendored scan of a fresh Pipenv checkout (no venv yet) fails with exit 1 on packages that exist only in the system Python, because the crawler falls back to the global site-packages

[mikolalysenko]

[agent] Found by the scheduled Pipenv bug-hunt routine (ledger #313).

Summary

Vendored mode is meant for lock-only checkouts. The docs say "Lock-only checkout (nothing installed): … Discovered from the lock" (docs/testing/pipenv-compatibility.md, Formats table). But in a Pipenv project whose venv doesn't exist yet (a fresh CI checkout before pipenv install), the Python crawler still falls back to the global interpreter's site-packages. That's the same fallback as #504: get_site_packages_paths → is_python_project → get_global_python_site_packages(). So scan --mode vendored collects every patchable package from the OS Python (/usr/lib/python3/dist-packages on Debian/Ubuntu) as if it belonged to the project, then tries to vendor it into Pipfile.lock:

Error: Cannot vendor pkg:pypi/pyyaml@6.0.1: Pipfile.lock names pyyaml in neither default nor develop; run `pipenv lock` first

The run ends with status: partial_failure, exit 1, pypi_pipenv_lock_package_missing. The real lock packages (six, idna) do get vendored. The suggested remedy (pipenv lock) can't help, because pyyaml isn't a project dependency at all. --dry-run reports Would download and vendor 3 patches and doesn't flag the refusal, so the preview and the wet run disagree.

Hosted mode in the same state lists the system package as a project package with a patch and warns redirect_pipenv_skipped: Pipfile.lock has no entry for pyyaml (exit 0). scannedPackages is 149 (the whole system Python) instead of the lock's 2.

Impact

• CI breakage: the documented vendored flow runs on a fresh checkout. GitHub's ubuntu-* runners and Debian/Ubuntu Docker images ship a system Python with packages like requests, urllib3, jinja2, cryptography and PyYAML. Whenever Socket has a patch for any of them, every vendored scan of a Pipenv project exits 1. That stays true while the project's own packages vendor fine, and the error tells the user to run a command that won't fix it.
• Patch counts, rollout (--max-new-patches) and the report include packages that aren't part of the project, so a rollout budget can be spent on the OS Python's packages.
• Unlike Agent-mode scan in a Pipenv project without a Pipenv venv patches the system Python's site-packages in place instead of the project's venv/ (regression from #388) #504 (agent mode writing to the system Python), nothing outside the project is written here. A fix for Agent-mode scan in a Pipenv project without a Pipenv venv patches the system Python's site-packages in place instead of the project's venv/ (regression from #388) #504 that only blocks agent-mode writes would therefore leave this failure in place. The crawl itself must not fall back for a Pipenv project, because lock-only packages already come from Pipfile.lock.

Repro (Linux, Debian/Ubuntu system Python with python3-yaml 6.0.1; mock patch API offering free patches for six 1.16.0, idna 3.7 and pyyaml 6.0.1)

mkdir app && cd app
cat > Pipfile <<'EOF'
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
six = "==1.16.0"
idna = "==3.7"
EOF
pipenv lock                                   # Pipenv 2026.8.0; lock only
export WORKON_HOME=$PWD/../empty-workon       # no Pipenv venv yet (fresh checkout)
mkdir -p "$WORKON_HOME"
socket-patch scan --mode vendored --yes --ecosystems pypi --json > out.json; echo "exit=$?"
# exit=1, status partial_failure
# vendor.events: pkg:pypi/pyyaml@6.0.1 failed pypi_pipenv_lock_package_missing
#                six / idna applied
python3 -c "import yaml; print(yaml.__file__)"   # /usr/lib/python3/dist-packages/yaml/__init__.py

Control: run pipenv sync first (the WORKON_HOME venv now exists). The same scan crawls only the venv (Found 4 packages), vendors six and idna, and exits 0.

Expected vs actual

• Expected: CLI_CONTRACT.md "cwd-only (single project)" says the crawler inspects only the project rooted at --cwd, and global packages are the -g / --global-prefix surface. The Pipenv branch of find_local_venv_site_packages returns nothing on purpose when Pipenv has no venv yet. For vendored mode, the lock-only packages in Pipfile.lock are the whole candidate set. So a fresh checkout should vendor six and idna, exit 0, and never mention pyyaml.
• Actual: system-Python packages join the candidate set. Vendored exits 1 (pypi_pipenv_lock_package_missing, with a misleading pipenv lock remedy). --dry-run promises they'll vendor. Hosted warns about them.

Cells (main 9c43dfc, CLI 4.0.0, Linux; lock written by Pipenv 2026.8.0, py3.11)

Shape	vendored	hosted	dry-run
Pipfile + lock, no Pipenv venv, system Python has a patchable pkg not in the lock	❌ exit 1, partial_failure (2/2)	⚠️ exit 0, redirect_pipenv_skipped for the system pkg, 149 scanned	❌ "would vendor 3 patches"
Same, after pipenv sync (WORKON venv present, control)	✅ exit 0, 2 vendored	✅	—

The crawl doesn't depend on the Pipenv version (only the lock is Pipenv's, and every spec-6 lock behaves the same). macOS and Windows weren't probed: probe branches are blocked this run, see the ledger. Not bisected. The global fallback predates vendored Pipenv support, and #504 dates the Pipenv-branch half to #388 (ccd43f5).

Suspect code

• crates/socket-patch-core/src/crawlers/python_crawler.rs:3047-3049: if is_python_project(&options.cwd) { return Ok(get_global_python_site_packages().await); } in non-global mode. Its doc comment justifies the fallback with "a fresh clone … would silently return zero packages", but lockfile-only packages are now merged from the lock, so that fallback is no longer needed for a Pipenv project.
• crates/socket-patch-core/src/crawlers/python_crawler.rs:360-362: the Pipenv branch's empty result ("no venv yet, nothing to patch") can't be told apart from "nothing probed".
• crates/socket-patch-core/src/vendor/pypi_pipenv.rs:172-178: the pypi_pipenv_lock_package_missing detail recommends pipenv lock even when the package isn't a project dependency.

Related: #504 (same fallback, agent mode writes the system Python). That triage noted #476 (Poetry) and #502 (PDM) end in the same fallback.

[mikolalysenko]

[agent] Triaged as priority:p1 (Pipenv). Confirmed on main 9c43dfc: PythonCrawler::get_site_packages_paths (crates/socket-patch-core/src/crawlers/python_crawler.rs:3043-3049) falls back to get_global_python_site_packages() whenever find_local_venv_site_packages returns nothing and is_python_project(cwd) holds, and the Pipenv branch returns an empty list by design when Pipenv has no venv yet. Not a duplicate of #504 (different symptom: vendored refusal / hosted warning instead of a system-Python write).

Shares root cause with #504: the crawler can't tell the Pipenv branch's authoritative "no venv yet" from "nothing probed", so it falls back to the global site-packages. Will be fixed together.

---

Generated by Claude Code

[mikolalysenko]

[agent] Claiming this issue (with #504; shared root cause: Python crawler falls back to global site-packages when a Pipenv project has no Pipenv venv). Branch: agent/fix-pipenv-global-fallback. Claim-ID: 2026-10-06T16:20:54Z-fc3a41

---

Generated by Claude Code

[mikolalysenko]

[agent] Fix PR: #950

---

Generated by Claude Code

[mikolalysenko]

[agent] Shares root cause with #964, #504: get_site_packages_paths falls back to the global site-packages when a project marker exists but no project env does. #964 is the uv / script-lock half, which open PR #950 doesn't cover yet. Will be fixed together.

---

Generated by Claude Code

[mikolalysenko]

[agent] New variant from the Pipenv bug-hunt routine (ledger #313): the system-Python fallback also spends the rollout budget in vendored mode.

Repro on main 9c43dfc (2/2), Pipenv 2026.8.0 lock pinning six==1.16.0, fresh checkout with no Pipenv venv. The system Python (/usr/lib/python3/dist-packages) holds blinker 1.7.0, which also has a (mock) patch:

socket-patch scan --mode vendored --max-new-patches 1 --json
# exit 1, status partial_failure
# rollout.counts: {"new": 1, "deferred": 1}; deferred: [pkg:pypi/six@1.16.0]
# Error: Cannot vendor pkg:pypi/blinker@1.7.0: Pipfile.lock names blinker in neither default nor develop
# Pipfile.lock untouched

So the one budget slot goes to the system package, which can't be vendored, and the project's real dependency (six) gets deferred. Nothing is patched and the scan exits 1. With the same flags, hosted mode spends the slot on six (pass). Control: after pipenv sync the vendored scan crawls only the venv, so six is vendored and the scan exits 0.

PR #950 head (a21968e) fixes this as well. The vendored and hosted --max-new-patches 1 scans both exit 0 with scannedPackages: 1, six is wired and none are deferred, and the base #947 case (vendored, no budget) also exits 0.

---

Generated by Claude Code