Skip to content

Commit e146e03

Browse files
mikolalysenkoclaude
mikolalysenko
and
claude
authored
Support hosted and vendored PDM patches (#243)
* Support hosted and vendored PDM patches on a shared pdm.lock rewriter Adds hosted PDM redirects and refactors the vendored PDM backend onto one validated `pdm.lock` rewriter (`utils/pdm_lock.rs`). Hosted `scan --mode hosted` rewrites the target `[[package]]` to the hosted wheel URL + patched SHA-256 (`redirect_pdm_lock_package`); vendored wires the same unit to a committed wheel. Both support the legacy `[metadata.files]` table and `extras` entries, preserve line endings and non-canonical spacing, are idempotent, and leave `pyproject.toml` / `content_hash` untouched. Supported lock formats are `2` (PDM 0.12-1.4) and `4.3`-`4.5.1` (PDM 2.8+); `3.1` / `4.0`-`4.2` (PDM 1.8-2.7) and unknown formats are refused before any write. Hardening beyond the original change: - Hosted lock-only checkouts are discovered and redirected: the lock inventory now reads `pdm.lock` (discovery-only, mirroring poetry). Vendored keeps refusing a lock-only checkout since PDM installs into `__pypackages__`, which the crawler does not probe. - A re-scan after an external `pdm lock` rebases the ledger's edits onto the relocked text. PDM un-patches AND may reflow line endings (CRLF -> LF) on relock, so for the pdm edit kind the rebase adopts the fresh `original` (the relocked registry fragment); poetry's relock keeps the patch and keeps its older original, so its path is unchanged. - When several Python lockfiles coexist, `uv.lock` and `poetry.lock` drive hosted PyPI redirects ahead of `pdm.lock`; a leftover `pdm.lock` neither blocks a live redirect nor is falsely attested. The same predicate gates the redirect dispatch and the hosted confirmation filter. Adds byte-exact `pdm-native` fixtures and CLI-level hosted regression tests (lock-only redirect + attest + rollback, legacy `[metadata.files]`, and relock-reflow convergence). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs, matrix harness, and CI for PDM patch compatibility Documents hosted PDM support (ecosystems.md, hosted-production-e2e.md, CLI_CONTRACT.md new codes, CHANGELOG) and adds the per-release matrix: docs/testing/pdm-compatibility.md with a generated table, scripts/backtest-pdm.py (real PDM releases via uv, hosted/vendored/agent, installed-byte verification, tamper rejection, relock convergence, rollback), and .github/workflows/pdm-compatibility.yml (Linux/Windows/macOS). Measured 0.12-2.29 across 17 shapes x 3 modes: 523 pass, 268 refused as expected (identity-losing 3.1/4.0-4.2, forked two-version locks, non-default lock filenames), 46 unsupported (`__pypackages__` agent/vendored, a documented layout limitation), 0 failures. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Harden PDM patches: 14 correctness fixes from an adversarial review A 12-finder review (42 confirmed findings) plus a real-PDM verification pass surfaced correctness gaps beyond the initial three. Fixes, each verified against real PDM binaries: Confirmation / attestation (false-success class): - Refuse a lock whose `strategy` lacks `inherit_metadata` (lock_version 4.4.1/4.5.x) — PDM 2.12-2.23 re-resolves such a lock and cannot match a rewritten `url`/`path` source (CandidateNotFound); read the 4.3 boolean strategy spelling (`cross_platform`/`static_urls`) that PDM 2.8-2.9 write. - A `pdm.lock` refusal now vetoes the sibling pypi rewriters only for a package the lock CONTAINS but refuses, never one it simply lacks (which a `requirements.txt`/pylock legitimately installs) — the old veto silently dropped those redirects. - An inactive `pdm.lock` (a `uv.lock`/`poetry.lock` drives) no longer feeds the substring confirmation probe, so an untouched uv/poetry project cannot report a bogus `redirected: 1`. - Emit `redirect_pdm_stale_install_risk` for PDM < 2.11 locks (2/4.3/4.4), which do not reinstall over an already-installed same-version package. - A marker/multi-target lock holding one package at several versions is refused with an accurate, ordering-independent message (hosted and vendored). Vendored integrity: - Refuse a symlinked `pdm.lock` before wire/revert (the rename-over would replace the link and strand its target) — mirrors the uv/hosted guards. - Require a real sha256 in the pre-flight hash guard so a non-sha256 lock is refused before the wheel is built. - Refuse a `pdm add`/partial-relock unit that kept the patched hash but lost the vendored `path`, so a stale patched hash is never recorded as the registry original. - Supersede a prior Socket-hosted `url` (rotated token / new uuid, same origin and wheel leaf) in place instead of refusing it as foreign. - Fix the version-mismatch refusal to print the plain version, not `Some("…")`. Fragments / keys: - Give the recorded package fragment a trailing boundary so a lock_version-2 pristine fragment is not a prefix of the rewritten one (replay convergence). - Key legacy `[metadata.files]` by PDM's `safe_name().lower()` (dots preserved), so dotted-name packages in format-2 locks are found. Docs: PDM 2.8.0 writes an indistinguishable `4.3` lock but shares the identity-loss bug, so the supported floor is 2.8.1; the docker vendored e2e awk counter now matches the inline `files` shape. Adds unit tests for the strategy gate, multi-version fork, URL supersede, dotted-name key, absent-package veto skip, stale advisory, symlink guard and the non-sha256 pre-flight refusal. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 07a6b88 commit e146e03

35 files changed

Lines changed: 33000 additions & 369 deletions

‎.gitattributes‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,5 @@
33
# golden.test.ts). EOL conversion on checkout (Windows CRLF) breaks the
44
# byte-for-byte comparison, so they are checked out exactly as committed.
55
crates/socket-patch-core/tests/fixtures/redirect/** -text
6+
7+
crates/socket-patch-core/tests/fixtures/pdm-native/*.lock -text

‎.github/workflows/pdm-compatibility.yml‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
name: PDM patch compatibility
2+
3+
# Native PDM installer matrix: builds the CLI once per OS, bootstraps pinned
4+
# PDM releases with uv, and runs `scripts/backtest-pdm.py` — hosted, vendored
5+
# and agent mode against the public urllib3 free patch, verifying the
6+
# INSTALLED bytes, lock/manifest stability, hash rejection and rollback. No
7+
# Socket API token is needed. See docs/testing/pdm-compatibility.md.
8+
9+
on:
10+
pull_request:
11+
paths:
12+
- '.github/workflows/pdm-compatibility.yml'
13+
- 'scripts/backtest-pdm.py'
14+
- 'crates/socket-patch-core/src/utils/pdm_lock.rs'
15+
- 'crates/socket-patch-core/src/patch/redirect/**'
16+
- 'crates/socket-patch-core/src/vendor/pypi*.rs'
17+
- 'crates/socket-patch-core/src/vendor/common.rs'
18+
- 'crates/socket-patch-core/src/vendor/lock_inventory.rs'
19+
- 'crates/socket-patch-cli/src/commands/scan/**'
20+
- 'crates/socket-patch-cli/src/commands/rollback.rs'
21+
- 'crates/socket-patch-core/tests/fixtures/pdm-native/**'
22+
push:
23+
branches: [main]
24+
paths:
25+
- 'scripts/backtest-pdm.py'
26+
- 'crates/socket-patch-core/src/utils/pdm_lock.rs'
27+
- 'crates/socket-patch-core/src/patch/redirect/pdm.rs'
28+
- 'crates/socket-patch-core/src/vendor/pypi_pdm.rs'
29+
workflow_dispatch:
30+
inputs:
31+
versions:
32+
description: 'Space-separated PDM versions (empty = the required matrix below)'
33+
required: false
34+
default: ''
35+
shapes:
36+
description: 'Space-separated shapes (empty = every shape)'
37+
required: false
38+
default: ''
39+
40+
permissions:
41+
contents: read
42+
43+
concurrency:
44+
group: pdm-compat-${{ github.event.pull_request.number || github.ref }}
45+
cancel-in-progress: true
46+
47+
env:
48+
SOCKET_NO_CONFIG: '1'
49+
SOCKET_NO_UPDATE_CHECK: '1'
50+
PDM_CHECK_UPDATE: 'false'
51+
52+
jobs:
53+
build:
54+
strategy:
55+
fail-fast: false
56+
matrix:
57+
os: [ubuntu-latest, windows-latest, macos-latest]
58+
runs-on: ${{ matrix.os }}
59+
timeout-minutes: 30
60+
steps:
61+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
62+
with:
63+
persist-credentials: false
64+
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
65+
with:
66+
key: pdm-compat
67+
- run: cargo build --locked -p socket-patch-cli
68+
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
69+
with:
70+
name: pdm-cli-${{ matrix.os }}
71+
path: |
72+
target/debug/socket-patch
73+
target/debug/socket-patch.exe
74+
if-no-files-found: error
75+
retention-days: 7
76+
77+
native:
78+
needs: build
79+
strategy:
80+
fail-fast: false
81+
matrix:
82+
# Every stable PDM major family (0.x, 1.x, 2.x) and each 2.x lock-format
83+
# boundary, on Linux and Windows; macOS samples the ends of the range.
84+
os: [ubuntu-latest, windows-latest]
85+
pdm: ['0.12.3', '1.15.5', '2.0.3', '2.1.5', '2.3.4', '2.6.1', '2.7.4', '2.8.2', '2.9.3', '2.10.4', '2.11.2', '2.17.3', '2.20.1', '2.25.9', '2.29.2']
86+
include:
87+
- { os: macos-latest, pdm: '0.12.3' }
88+
- { os: macos-latest, pdm: '2.8.2' }
89+
- { os: macos-latest, pdm: '2.29.2' }
90+
runs-on: ${{ matrix.os }}
91+
timeout-minutes: 45
92+
steps:
93+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
94+
with:
95+
persist-credentials: false
96+
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
97+
with:
98+
name: pdm-cli-${{ matrix.os }}
99+
path: native-cli
100+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
101+
with:
102+
python-version: '3.12'
103+
# uv bootstraps every pinned PDM release (and its interpreter) itself;
104+
# pinning uv keeps the bootstrap reproducible across runner images.
105+
- run: python -m pip install uv==0.11.19
106+
- name: Exercise the native PDM installers
107+
shell: bash
108+
env:
109+
PDM_VERSION: ${{ matrix.pdm }}
110+
VERSIONS_OVERRIDE: ${{ github.event.inputs.versions }}
111+
SHAPES_OVERRIDE: ${{ github.event.inputs.shapes }}
112+
run: |
113+
chmod +x native-cli/socket-patch* || true
114+
cli="native-cli/socket-patch"
115+
if [ "$RUNNER_OS" = "Windows" ]; then cli="native-cli/socket-patch.exe"; fi
116+
versions="$PDM_VERSION"
117+
if [ -n "$VERSIONS_OVERRIDE" ]; then versions="$VERSIONS_OVERRIDE"; fi
118+
shapes_arg=()
119+
if [ -n "$SHAPES_OVERRIDE" ]; then shapes_arg=(--shapes $SHAPES_OVERRIDE); fi
120+
python3 scripts/backtest-pdm.py \
121+
--socket-patch "$cli" \
122+
--socket-patch-revision "$GITHUB_SHA" \
123+
--output native-pdm \
124+
--versions $versions \
125+
--modes hosted vendored agent \
126+
"${shapes_arg[@]}" \
127+
--jobs 3
128+
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
129+
if: always()
130+
with:
131+
name: pdm-results-${{ matrix.os }}-${{ matrix.pdm }}
132+
path: |
133+
native-pdm/summary.json
134+
native-pdm/summary.md
135+
native-pdm/*.log
136+
native-pdm/original/**/pdm.lock
137+
native-pdm/original/**/pyproject.toml
138+
native-pdm/captures/**/result.json
139+
native-pdm/captures/**/cli-output.json
140+
native-pdm/captures/**/pdm.lock
141+
native-pdm/captures/**/*.log
142+
retention-days: 14

‎CHANGELOG.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,33 @@ into the new version's section — see docs/releasing.md.
8888
the verified remedy (`pipenv run pip uninstall -y <pkg> && pipenv sync`, or
8989
a clean `pipenv --rm && pipenv sync`), and the stale purl is excluded from
9090
the same-run `--vex`.
91+
- **PDM projects take hosted patches, and hosted and vendored patches share
92+
one validated `pdm.lock` rewriter.** `scan --mode hosted` rewrites `pdm.lock`
93+
to point the target `[[package]]` at the hosted wheel URL with the patched
94+
SHA-256 (`redirect_pdm_lock_package`), and `scan --mode vendored` wires the
95+
same unit to a committed wheel through the shared rewriter
96+
(`utils/pdm_lock.rs`). Both preserve line endings and non-canonical spacing,
97+
support the legacy `[metadata.files]` table and separate `extras` entries,
98+
are idempotent, and leave `pyproject.toml` and `content_hash` untouched.
99+
Supported lock formats are `2` (PDM 0.12–1.4) and `4.3`–`4.5.1` (PDM 2.8.1+;
100+
PDM 2.8.0 writes the same `4.3` lock but still loses candidate identity, so
101+
upgrade to ≥ 2.8.1);
102+
the identity-losing `3.1` / `4.0`–`4.2` formats (PDM 1.8–2.7) and unknown
103+
future formats are refused before any write (`redirect_pdm_refused` /
104+
`pypi_pdm_lock_version_unsupported`), leaving the registry lock installable.
105+
A `pdm.lock` written by PDM 0.x/1.x (lock format `2`) warns
106+
`redirect_pdm_legacy_sync_required`: those releases have an upstream
107+
freshness bug, so `pdm install` can regenerate the lock — use `pdm sync`.
108+
Verified end-to-end against real PDM 0.12–2.29 across hosted, vendored and
109+
agent mode on Linux, Windows and macOS; see
110+
`docs/testing/pdm-compatibility.md` and `scripts/backtest-pdm.py`. When
111+
several Python lockfiles coexist, `uv.lock` and `poetry.lock` drive hosted
112+
PyPI redirects ahead of `pdm.lock`, so a leftover `pdm.lock` beside them
113+
neither blocks a live redirect nor is falsely attested. A hosted lock-only
114+
`pdm.lock` checkout is discovered and redirected (the lock inventory now
115+
reads `pdm.lock`), and a re-scan after an external `pdm lock` rebases the
116+
ledger's recorded edits onto the relocked text so `rollback` stays
117+
byte-invertible even when PDM reflows the lock's line endings.
91118
- **Poetry projects take hosted patches, and vendored patches now cover every
92119
`poetry.lock` generation.** `scan --mode hosted` rewrites `poetry.lock` to a
93120
`[package.source] type = "url"` pointing at the Socket-hosted, SHA-256-pinned

0 commit comments

Comments
 (0)