Skip to content

PDM PEP 582 detection misses python.use_venv = false as PDM 2.27+ writes it (a TOML string) and in the user config, so agent mode patches an activated or stray venv and leaves __pypackages__ unpatched #609

Description

[agent] Found by the scheduled PDM bug-hunt routine (ledger #312).

Summary

#540 taught Python env discovery that a PDM project with a base interpreter in .pdm-python installs into __pypackages__/<X.Y>/lib (PEP 582) when python.use_venv is off. The setting is read in two ways that miss what PDM actually writes:

  1. PDM ≥ 2.27.0 writes the value as a TOML string. pdm config -l python.use_venv false produces use_venv = "false" in pdm.toml (2.26.9 and older write a bare false). pdm_uses_venv reads it with .as_bool(), gets None, and falls back to "on". PDM itself parses both forms as False.
  2. The user config is never read. The default scope of pdm config python.use_venv false is the user config (~/.config/pdm/config.toml), which is how PDM's docs enable PEP 582. Discovery only looks at PDM_USE_VENV and the project's pdm.toml / .pdm.toml. This happens on every PDM version.

Either way discovery decides PDM uses a venv. It then takes the generic probes, so an activated VIRTUAL_ENV or a stray ./.venv (for example one an IDE or uv left behind) wins over __pypackages__. PDM ignores both, because the saved interpreter is a base Python and use_venv is off.

Impact

scan --mode agent / get patch a venv the project never runs and print status: success with exit 0. The copy that pdm run imports (__pypackages__) stays unpatched. vex does fail closed here (not_applied), so CI gated on VEX catches it. A plain scan --sync cron does not.

Repro (Linux, real PDM 2.29.2, mock patch API serving urllib3 1.26.18)

mkdir p && cd p
cat > pyproject.toml <<'EOF'
[project]
name = "p"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["urllib3==1.26.18"]
[tool.pdm]
distribution = false
EOF
pdm config -l python.use_venv false     # writes use_venv = "false" on PDM >= 2.27
# or: pdm config python.use_venv false  # user config, any PDM version
pdm use -f /usr/bin/python3.12          # "Using __pypackages__ because non-venv Python is used."
pdm install                             # -> __pypackages__/3.12/lib/urllib3
uv venv .venv && uv pip install -p .venv/bin/python urllib3==1.26.18   # stray venv PDM ignores
socket-patch scan --mode agent --json   # exit 0, status success
tail -1 .venv/lib/python3.12/site-packages/urllib3/_version.py    # patched
tail -1 __pypackages__/3.12/lib/urllib3/_version.py               # still original
pdm run python -c 'import urllib3; print(urllib3.__file__)'       # __pypackages__/3.12/lib/urllib3/__init__.py

Activating an unrelated venv (VIRTUAL_ENV=/some/other/venv) instead of the stray .venv gives the same result: that venv gets patched.

Expected vs actual

  • Expected (docs/testing/pdm-compatibility.md, "The installed env is the one PDM records"): "A base interpreter means PEP 582 (__pypackages__/<X.Y>/lib) only when python.use_venv is off (PDM_USE_VENV, or [python] use_venv in pdm.toml / .pdm.toml …) … Agent mode patches the env found". So with use_venv off, agent mode should patch __pypackages__.
  • Actual: use_venv = "false" in pdm.toml (PDM ≥ 2.27's own output) and use_venv in the user config are both treated as "on", and the activated or stray venv gets patched.

Matrix (Linux, main 045d7ec, each cell run twice)

PDM where use_venv=false is set value written agent patches result
2.29.2 project pdm.toml (pdm config -l) "false" (string) stray .venv / activated VIRTUAL_ENV fail
2.29.2 project pdm.toml, hand-written bool false __pypackages__ pass
2.29.2 user config (pdm config) "false" stray .venv fail
2.26.9 project pdm.toml (pdm config -l) false (bool) __pypackages__ pass
2.26.9 user config (pdm config) false stray .venv fail
2.29.2 PDM_USE_VENV=false env (control) — __pypackages__ pass

String-valued writes start at PDM 2.27.0: 2.12.4, 2.20.1, 2.24.2 and 2.26.9 write a bool, and 2.27.0, 2.28.2, 2.29.0 and 2.29.2 write a string. PDM 2.12.4 and 2.29.2 both read use_venv = "false" as False (pdm config python.use_venv). macOS and Windows weren't run. The logic is OS-independent, but the user-config path differs there (~/Library/Application Support/pdm/config.toml, %LOCALAPPDATA%\pdm\pdm\config.toml). The site config (/etc/xdg/pdm/config.toml, cf. #566) and PDM_CONFIG_FILE would be missed the same way.

First bad version

This is new code from #540 (b0a32db). Before that, PEP 582 wasn't crawled at all (documented), so it's a gap in the fix rather than a regression from a release.

Suspect code

  • crates/socket-patch-core/src/crawlers/python_crawler.rs:493-510 (pdm_uses_venv): .as_bool() drops string values, so it should parse them the way PDM's ensure_boolean does, like pdm_env_flag. It also never consults the user or site config (PDM_CONFIG_FILE, <user config dir>/pdm/config.toml), which pdm_global_site_packages (:1987) already locates.
  • :479-487 is the caller that returns None → generic probes (VIRTUAL_ENV at :367-384, ./.venv at :405-410) win over __pypackages__.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions