[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).
Summary
The Python crawler only recognises *.dist-info directories. When pip before 23.1 installs a package from an sdist and the wheel package isn't installed, it uses the legacy setup.py install path. That path writes <name>-<version>-pyX.Y.egg-info instead of .dist-info. This is the default state of a fresh python -m venv on CPython ≤ 3.11, which ships setuptools but no wheel, so it affects every pip release from 20 through 23.0.x (Ubuntu 22.04's pip 22.0.2 and Debian 12's pip 23.0.1 included).
socket-patch never sees these packages, which are really installed and importable:
- Agent mode (
scan --mode agent, get) reports the package as [skip] … (not installed; run your package manager's install first …) with errorCode: package_not_installed, then exits 0 / success. The installed file stays unpatched.
- Hosted mode (the default
scan) rewrites requirements.txt to the hosted wheel. pip then keeps the same-version egg-info install (Requirement already satisfied), so the patched bytes never land. The redirect_pypi_stale_install guard doesn't fire, because it uses the same crawler lookup (find_each_by_purl). The user gets no signal that the venv is still vulnerable.
- Global mode (
scan -g, --global-prefix) omits these packages from the report. On Debian/Ubuntu that also includes every apt-installed Python package (python3-six etc. ship six-1.16.0.egg-info). In this sandbox, scan -g sent 102 purls and left out six@1.16.0, setuptools@68.1.2, oauthlib, launchpadlib and other egg-info packages.
Impact
Silent false negatives. A vulnerable, patchable package is reported as "not installed" (agent) or quietly left stale (hosted), and the command still exits 0. The hosted case is the worst: the documented safety net is the stale-install warning, and it never fires.
Repro (Linux shown; the probe covers macOS and Windows)
python3.11 -m venv .venv
.venv/bin/pip install -q 'pip==23.0.1' # also 20.3.4, 22.3.1
.venv/bin/pip uninstall -y wheel || true # default venv state on py<=3.11 anyway
echo 'six==1.16.0' > requirements.txt
.venv/bin/pip install --no-binary six -r requirements.txt
ls .venv/lib/python3.11/site-packages | grep six # six-1.16.0-py3.11.egg-info, six.py
socket-patch scan --mode agent --yes --json # apply.patches[0] = skipped / package_not_installed, exit 0
socket-patch scan --yes --json # hosted: redirect.warnings == [] (no redirect_pypi_stale_install)
.venv/bin/pip install -r requirements.txt # "Requirement already satisfied" -> still unpatched
socket-patch scan -g --global-prefix .venv/lib/python3.11/site-packages --json # six not reported
The patch data came from a local mock of the authenticated patch API (the same batch / by-package / view / package shapes as tests/docker_e2e_pypi.rs) serving a marked six.py. pip, the venvs and the installs were all real.
Expected vs actual
- CLI_CONTRACT.md "Python stale-install guard": "after a hosted redirect,
scan / get use the Python crawler to inspect every matching installed package … A readable file that differs from the patch's afterHash emits redirect_pypi_stale_install". Actual: an installed egg-info copy isn't inspected, and no warning fires.
- docs/ecosystems.md lists PyPI agent mode as "✅ in place". Actual: pip's own legacy install layout is reported as "not installed" and isn't patched.
- The maintainer's
-g checklist (ledger Bug hunt ledger: pip / requirements.txt #309) says scan -g must find every globally installed package that has a patch. Actual: egg-info installs are missing.
OS × version matrix (probe run 36836217323, plus local Linux runs)
| OS |
Python |
pip 20.3.4 |
pip 22.3.1 |
pip 23.0.1 |
pip 23.1 (control, writes dist-info) |
| ubuntu-latest |
3.8.18 / 3.11.16 |
fail |
fail |
fail |
pass |
| macos-latest |
3.8.10 / 3.11.9 |
fail |
fail |
fail |
pass |
| windows-latest |
3.8.10 / 3.11.9 |
fail |
fail |
fail |
pass |
| Linux (local) |
3.10, 3.11 |
fail |
fail (21.3.1 too) |
fail |
pass (24.0 too) |
"fail" means all of: agent skipped/package_not_installed with the marker absent, hosted with no redirect_pypi_stale_install and the file still unpatched after pip install -r, and -g not reporting six. Each pip 23.1 cell gives agent added with the file patched, the stale warning present, and -g reporting six.
First bad version
Not a regression. Released v4.0.0 (PyPI socket-patch==4.0.0) behaves the same way: [skip] pkg:pypi/six@1.16.0 (not installed …). crates/socket-patch-cli/tests/in_process_python_envs.rs:455 (pypi_egg_info_layout_handled) pins the gap as the "current contract" and asks for the assertion to be flipped once egg-info support lands. No user-facing doc mentions the limitation.
Suspect code
crates/socket-patch-core/src/crawlers/python_crawler.rs:1576-1584: list_dist_info_packages_sync keeps only entries ending in .dist-info. It feeds find_each_by_purl (:1529), which the hosted stale-install probe uses (crates/socket-patch-cli/src/commands/scan/hosted/python.rs:76), and the crawl / apply lookups.- Egg-info metadata lives in
<dir>.egg-info/PKG-INFO, or in a bare .egg-info file (distutils / some distro packages, e.g. PyGObject-3.48.2.egg-info). It has the same Name: / Version: headers as METADATA. Filenames can carry a -pyX.Y suffix.
Probe
[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).
Summary
The Python crawler only recognises
*.dist-infodirectories. When pip before 23.1 installs a package from an sdist and thewheelpackage isn't installed, it uses the legacysetup.py installpath. That path writes<name>-<version>-pyX.Y.egg-infoinstead of.dist-info. This is the default state of a freshpython -m venvon CPython ≤ 3.11, which ships setuptools but no wheel, so it affects every pip release from 20 through 23.0.x (Ubuntu 22.04's pip 22.0.2 and Debian 12's pip 23.0.1 included).socket-patch never sees these packages, which are really installed and importable:
scan --mode agent,get) reports the package as[skip] … (not installed; run your package manager's install first …)witherrorCode: package_not_installed, then exits 0 /success. The installed file stays unpatched.scan) rewritesrequirements.txtto the hosted wheel. pip then keeps the same-version egg-info install (Requirement already satisfied), so the patched bytes never land. Theredirect_pypi_stale_installguard doesn't fire, because it uses the same crawler lookup (find_each_by_purl). The user gets no signal that the venv is still vulnerable.scan -g,--global-prefix) omits these packages from the report. On Debian/Ubuntu that also includes every apt-installed Python package (python3-sixetc. shipsix-1.16.0.egg-info). In this sandbox,scan -gsent 102 purls and left outsix@1.16.0,setuptools@68.1.2,oauthlib,launchpadliband other egg-info packages.Impact
Silent false negatives. A vulnerable, patchable package is reported as "not installed" (agent) or quietly left stale (hosted), and the command still exits 0. The hosted case is the worst: the documented safety net is the stale-install warning, and it never fires.
Repro (Linux shown; the probe covers macOS and Windows)
The patch data came from a local mock of the authenticated patch API (the same
batch/by-package/view/packageshapes astests/docker_e2e_pypi.rs) serving a markedsix.py. pip, the venvs and the installs were all real.Expected vs actual
scan/getuse the Python crawler to inspect every matching installed package … A readable file that differs from the patch'safterHashemitsredirect_pypi_stale_install". Actual: an installed egg-info copy isn't inspected, and no warning fires.-gchecklist (ledger Bug hunt ledger: pip / requirements.txt #309) saysscan -gmust find every globally installed package that has a patch. Actual: egg-info installs are missing.OS × version matrix (probe run 36836217323, plus local Linux runs)
"fail" means all of: agent
skipped/package_not_installedwith the marker absent, hosted with noredirect_pypi_stale_installand the file still unpatched afterpip install -r, and-gnot reporting six. Each pip 23.1 cell gives agentaddedwith the file patched, the stale warning present, and-greporting six.First bad version
Not a regression. Released v4.0.0 (PyPI
socket-patch==4.0.0) behaves the same way:[skip] pkg:pypi/six@1.16.0 (not installed …).crates/socket-patch-cli/tests/in_process_python_envs.rs:455(pypi_egg_info_layout_handled) pins the gap as the "current contract" and asks for the assertion to be flipped once egg-info support lands. No user-facing doc mentions the limitation.Suspect code
crates/socket-patch-core/src/crawlers/python_crawler.rs:1576-1584:list_dist_info_packages_synckeeps only entries ending in.dist-info. It feedsfind_each_by_purl(:1529), which the hosted stale-install probe uses (crates/socket-patch-cli/src/commands/scan/hosted/python.rs:76), and the crawl / apply lookups.<dir>.egg-info/PKG-INFO, or in a bare.egg-infofile (distutils / some distro packages, e.g.PyGObject-3.48.2.egg-info). It has the sameName:/Version:headers asMETADATA. Filenames can carry a-pyX.Ysuffix.Probe