Fix Pipenv venv discovery order (#334, #384)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #334
Fixes #384

Summary

For a Pipenv project, agent mode now uses the venv Pipenv itself would use. Before, it could patch the wrong venv, or none, and still exit 0.

Root cause

find_local_venv_site_packages (crates/socket-patch-core/src/crawlers/python_crawler.rs) checked the same places for every project, in a fixed order: VIRTUAL_ENV, then ./.venv and ./venv, and Pipenv's own $WORKON_HOME venv only when all of those came up empty. Pipenv decides differently. I checked its venv-lookup code (Project.virtualenv_location / get_location_for_virtualenv, and VenvLocator in 2026.x) in the 2022.12.19, 2023.12.1 and 2026.8.0 wheels:

• VIRTUAL_ENV counts only when PIPENV_ACTIVE is absent and bool(PIPENV_IGNORE_VIRTUALENVS) is false (Agent-mode scan patches the activated VIRTUAL_ENV even when PIPENV_IGNORE_VIRTUALENVS or PIPENV_ACTIVE tells Pipenv to ignore it, leaving the Pipenv venv unpatched with exit 0 #384).
• venv/ is never used (Agent-mode scan skips Pipenv's out-of-tree venv when the project has a stray venv/ directory or a .venv with PIPENV_VENV_IN_PROJECT=0, and still exits 0 #334).
• PIPENV_VENV_IN_PROJECT=0 (or PIPENV_NO_VENV_IN_PROJECT=1), and on 2026.2+ the Pipfile's [pipenv] venv_in_project = false, make Pipenv ignore a ./.venv directory (Agent-mode scan skips Pipenv's out-of-tree venv when the project has a stray venv/ directory or a .venv with PIPENV_VENV_IN_PROJECT=0, and still exits 0 #334).
• On 2026.2+, an existing WORKON_HOME venv beats an auto-detected ./.venv (Agent-mode scan skips Pipenv's out-of-tree venv when the project has a stray venv/ directory or a .venv with PIPENV_VENV_IN_PROJECT=0, and still exits 0 #334).

Fix

• New pipenv_project_site_packages follows Pipenv's order. For a Pipenv project (a Pipfile or Pipfile.lock), find_local_venv_site_packages uses it right after a VIRTUAL_ENV that Pipenv would honour. The generic .venv / venv / Poetry checks run only when Pipenv's venv doesn't exist yet, so a project with a stray Pipfile and no Pipenv venv behaves as before. Even in that case, an opted-out VIRTUAL_ENV is never used.
• Environment variables are parsed the way Pipenv's get_from_env / env_to_bool does it, including the PIPENV_NO_* negations.
• One case where both venvs get patched: ./.venv is only auto-detected, nothing explicit is set, and a WORKON_HOME venv exists. Pipenv up to 2026.1 uses ./.venv, while 2026.2+ uses the WORKON_HOME venv. We can't tell the version without running Pipenv, so both are returned (WORKON_HOME first). Whichever one the installed Pipenv uses is patched, and both are this project's own venvs.
• The hosted and vendored stale-install warnings call the same function, so they now check the right venv too.
• Docs: README Pipenv section, and docs/testing/pipenv-compatibility.md.

Only Rust code changed. The npm, PyPI and gem wrappers don't discover venvs themselves, so they need no parallel change.

Tests (per issue)

• Agent-mode scan patches the activated VIRTUAL_ENV even when PIPENV_IGNORE_VIRTUALENVS or PIPENV_ACTIVE tells Pipenv to ignore it, leaving the Pipenv venv unpatched with exit 0 #384: crawlers::python_crawler::tests::pipenv_opt_outs_keep_virtual_env_from_hijacking_the_project covers PIPENV_IGNORE_VIRTUALENVS = 1 / true / non-boolean text, PIPENV_NO_IGNORE_VIRTUALENVS=0, and PIPENV_ACTIVE = 1 / empty. It also checks that an opted-out VIRTUAL_ENV isn't used even when Pipenv has no venv. Controls: falsy values, and non-Pipenv projects. At the CLI level, in_process_python_envs::pipenv_opt_outs_keep_activated_virtual_env_from_hijacking_scan runs scan and checks that the activated venv's package is never queried.
• Agent-mode scan skips Pipenv's out-of-tree venv when the project has a stray venv/ directory or a .venv with PIPENV_VENV_IN_PROJECT=0, and still exits 0 #334: pipenv_stray_venv_dir_does_not_shadow_the_workon_home_venv, pipenv_venv_in_project_settings_decide_about_dot_venv (env var 0 / false / Off, the NO_ form, Pipfile true and false, env var beating the Pipfile) and pipenv_auto_detected_dot_venv_and_workon_home_venv_are_both_returned. At the CLI level, in_process_python_envs::pipenv_stray_venv_dirs_do_not_shadow_the_pipenv_venv covers venv/, and .venv with PIPENV_VENV_IN_PROJECT=0.

Red on main (before the fix), green with it:

$ cargo test -p socket-patch-core --lib pipenv_        # before the fix
test ...pipenv_auto_detected_dot_venv_and_workon_home_venv_are_both_returned ... FAILED
test ...pipenv_opt_outs_keep_virtual_env_from_hijacking_the_project ... FAILED
test ...pipenv_stray_venv_dir_does_not_shadow_the_workon_home_venv ... FAILED
test ...pipenv_venv_in_project_settings_decide_about_dot_venv ... FAILED
test result: FAILED. 18 passed; 4 failed
$ cargo test -p socket-patch-core --lib pipenv_        # with the fix
test result: ok. 22 passed; 0 failed

Local runs:

• cargo clippy --workspace --all-features -- -D warnings: clean.
• cargo test --workspace --all-features --no-fail-fast: 10398 passed and 22 failed. All 22 failures come from this sandbox running as root, and none of them touch the Python crawler. 21 are write-failure tests that chmod a directory read-only, which root ignores (setup_silent_keeps_apply_phase_error_output, redirect_ledger_write_failure_*, *_write_failure_*, and so on). The last is stage_local_artifact_caps_oversized_artifact_before_buffering, which re-runs itself in a child process to measure memory use. CI runs as a regular user.
• cargo fmt --all -- --check already fails on main (60 files with the pinned 1.93.1 toolchain), and CI doesn't run it. The lines this PR touches are rustfmt-clean.
• Real-Pipenv e2e: SOCKET_PATCH_PIPENV_E2E_REQUIRED=1 SOCKET_PATCH_PIPENV_E2E_VERSIONS=2026.8.0 cargo test -p socket-patch-cli --all-features --test e2e_vex_build -- pipenv:: --ignored: 1 passed (pipenv_every_major_hosted_and_vendored_end_in_manifest_less_vex, 33s).

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes which site-packages paths are scanned for Pipenv projects (agent mode and stale-install checks that call the same helper); wrong discovery would patch or warn against the wrong environment, but scope is limited to Pipenv layout logic with heavy test coverage.

Overview
Fixes Pipenv venv discovery so agent-mode scan/patch targets the environment Pipenv would use, not a generic probe order that could patch the wrong venv or miss the real one (#334, #384).

For projects with a Pipfile or Pipfile.lock, find_local_venv_site_packages now branches early: it honors VIRTUAL_ENV only when Pipenv would (PIPENV_ACTIVE / PIPENV_IGNORE_VIRTUALENVS parsing mirrors Pipenv), then resolves venvs via new pipenv_project_site_packages (WORKON_HOME placement, in-project rules from env and Pipfile [pipenv] venv_in_project, never venv/). Generic .venv/venv/Poetry probing no longer runs for Pipenv projects, avoiding stray trees shadowing Pipenv’s venv or false positives when Pipenv has no venv yet. When both WORKON_HOME and auto-detected ./.venv could apply (Pipenv version ambiguity), both are returned so the active one is covered.

Adds injectable env for tests, FIFO-safe Pipfile reads, broad unit tests in python_crawler, end-to-end scan tests in in_process_python_envs, and updates docs/testing/pipenv-compatibility.md for agent-mode behavior.

^{Reviewed by Cursor Bugbot for commit cdcc2b3. Configure here.}

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Unguarded Pipfile read can hang
  • Replaced bare std::fs::read_to_string with read_regular_to_string_sync to prevent hanging when Pipfile is a FIFO or device.

Or push these changes by commenting:

@cursor push 2b7913a01e

Preview (2b7913a01e)

diff --git a/crates/socket-patch-core/src/crawlers/python_crawler.rs b/crates/socket-patch-core/src/crawlers/python_crawler.rs
--- a/crates/socket-patch-core/src/crawlers/python_crawler.rs
+++ b/crates/socket-patch-core/src/crawlers/python_crawler.rs
@@ -380,7 +380,7 @@
         Some(Err(text)) if !text.is_empty() => return Some(true),
         _ => {}
     }
-    let text = std::fs::read_to_string(cwd.join("Pipfile")).ok()?;
+    let text = read_regular_to_string_sync(&cwd.join("Pipfile")).ok()?;
     let doc = text.parse::<toml_edit::DocumentMut>().ok()?;
     let value = doc.get("pipenv")?.get("venv_in_project")?.as_value()?;
     // Python's `bool(value)` for the scalar shapes a Pipfile can hold.

_{You can send follow-ups to the cloud agent here.}

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

[burn-down agent] Ready for review on 38d67b0: mergeable, 0 commits behind main. CI: 332/332 completed checks green (6 workflow skips). Bugbot reviewed 38d67b0 and found no new issues. The one finding on 1cc51c9 ("Empty Pipenv result uses stray venvs") was real and is fixed in 38d67b0: a Pipenv project with no Pipenv venv yet no longer falls back to venv/ or a rejected .venv (regression test pipenv_without_its_venv_does_not_fall_back_to_stray_trees). Reviewer should look at the both-venvs return for an auto-detected .venv plus an existing WORKON_HOME venv in pipenv_project_site_packages.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issue.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit cdcc2b3. Configure here.}

[Mikola Lysenko (mikolalysenko)]

Burn-down agent check: ready for review at cdcc2b34.

• CI: 97/97 non-skipped checks green on cdcc2b3 (3 skipped)
• Bugbot reviewed cdcc2b3 and found 1 issue, which I refuted and resolved: "Empty Pipenv discovery scans global Python". The global fallback is pre-existing on main and this PR doesn't touch it. A stray venv/ now takes the same path as a Pipenv project with no venv. Removing that fallback is a separate product decision.
• Reviewers: the last push was a merge of main (Fix Poetry venv discovery to match Poetry (#327, #329) #330), which kept both Pipenv and Poetry venv discovery. Wenxin-Jiang's earlier approval was on 38d67b0, so it needs re-approval.

---

Generated by Claude Code