Skip to content

Latest commit

 

History

History
210 lines (189 loc) · 15.2 KB

File metadata and controls

210 lines (189 loc) · 15.2 KB

Poetry compatibility and production backtests

socket-patch supports hosted and vendored Python patches in every poetry.lock generation Poetry has written, and agent mode (in-place patching of the project's virtualenv) on every Poetry release. The tests use real Poetry releases bootstrapped with uv, real PyPI artifacts, and the public Socket patch service. Successful rewriting alone is not an installation result: the backtest reinstalls from the rewritten lock and compares the installed bytes with the published patch.

This supplements the hosted and vendored production suites and mirrors the uv matrix. See the ecosystem matrix for other package managers.

Formats and rewrite behavior

Lock generation (writer) Hosted (scan --mode hosted) Vendored (scan --mode vendored)
[metadata.hashes], no lock-version (Poetry 0.12) Refused (redirect_poetry_lock_unsupported): the installer ignores [package.source] type = "url" and installs the registry artifact, so a rewrite would attest a patch that never lands. [package.source] type = "file" + reference = "" (read unconditionally by 0.12) and the wheel's SHA-256 in [metadata.hashes].
lock-version = "1.0" (Poetry 1.0) type = "url" + reference = ""; the URL carries #sha256=<hex>& because Poetry 1.0 appends #egg=<name> unconditionally (see the pip caveat below). pip verifies the fragment; the patched hash is also written to [metadata.files] (consistency) and to the package's own files (what Poetry ≥ 1.2 verifies when it consumes a 1.0 lock; 1.0 ignores the extra key). type = "file" + reference = ""; [metadata.files] entry replaced.
lock-version = "1.1" (Poetry 1.1, 1.2) type = "url"; the patched hash is written to BOTH [metadata.files] (what Poetry 1.1 verifies) and the package's own files (what Poetry 1.2 verifies — it drops URL hashes from [metadata.files]). type = "file"; [metadata.files] entry replaced.
lock-version = "2.0" / "2.1" / any "2.<n>" (Poetry 1.3+) type = "url"; files = [{file, hash}] replaced with the single patched wheel. type = "file"; files replaced. LF 2.x locks keep Poetry's own multi-line files formatting; CRLF locks and legacy formats go through the shared toml_edit rewriter, which writes a single-line inline array. Both are valid TOML and byte-stable under poetry check --lock.

Both modes retain the package version, dependencies, groups, markers, extras and the pyproject content-hash; no pyproject edit is required. A repeated scan leaves the lock unchanged. Rollback restores the recorded original fragments — one per patch (plus the integrity-table entry on legacy formats), so either of two patches can be rolled back first and unrelated edits survive. Refused before any write: a [[package]] listed at several versions (marker fork), a user-authored [package.source] on another origin (an earlier Socket URL for the same wheel is superseded in place, e.g. after a grant-token rotation), an unsupported lock-version, a malformed [metadata.files] / [metadata.hashes] value, and a wheel whose filename does not match the locked package.

Installer boundaries (measured)

Poetry Hosted Vendored Verifies the lock hash on install Replaces an already-installed same-version package
0.12 refused (URL sources ignored) supported no no
1.0 supported (#sha256=…& fragment; pip ≤ 22.2 or ≥ 23.1) supported hosted: yes (pip fragment); vendored: no no
1.1 – 1.3 supported supported hosted: yes; vendored: no no
1.4 – 1.8 supported supported yes / yes yes
2.0 – 2.4 supported supported yes / yes yes

Two consequences for Poetry releases before 1.4:

  • A warm virtualenv keeps the upstream package after the lock is rewritten: poetry install compares installed packages by name and version only and prints "No dependencies to install or update". Recreate the virtualenv (or pip uninstall the package) before installing, or upgrade Poetry. Fresh installs pick up the patched wheel on every release. The CLI flags this as redirect_poetry_stale_install_risk (hosted) and pypi_poetry_integrity_unverified (vendored), keyed on the lock's writer: formats 0 / 1.0 / 1.1 are only written by pre-1.4 releases, and a lock 2.0 whose header lacks a @generated by Poetry X.Y.Z version was written by 1.3 (1.4+ stamp their version). A 2.0 lock stamped 1.4–1.8 is not flagged.
  • Poetry 1.0 installs URL sources through pip, which reads the hash from the URL fragment. Poetry appends #egg=<name> to every URL, so the rewrite ends the fragment with & to keep sha256=<hex> intact. Measured across pip releases: pip 20.3 – 22.2 and 23.1+ install and verify the patched wheel with that spelling; pip 22.3 and 23.0 (the pip Python 3.8's ensurepip seeds) treat the trailing &#egg=… as part of the digest and refuse the install (fail-closed, nothing installed) — no fragment spelling satisfies them, so upgrade or downgrade pip inside the virtualenv. Without the &, every pip ≥ 22.3 refuses. The hosted advisory names this window for lock 1.0.
  • Local (vendored) wheel hashes are not verified; the committed wheel bytes are the protection — review them. Hosted URL hashes are verified on every release from 1.0 on by the default installer (Poetry's deprecated pip backend, experimental.new-installer = false, verifies nothing).

Hosted mode also checks the installed package's bytes, independently of the lock's writer version. redirect_pypi_stale_install means readable files in a discovered interpreter still differ from the patched hashes. This read-only check repeats on re-scans and excludes the package from same-run VEX, even with --vex-no-verify or a healthy copy in another interpreter. If no patches remain to attest, --vex fails with no_applicable_patches. Reinstall in the affected interpreter, then verify with socket-patch vex. A lock-only checkout has no installed bytes to judge and keeps the existing lock-based attestation behavior.

Other measured details:

  • poetry lock --no-update (1.1–1.8) and bare poetry lock (2.x) keep the patch source. Bare poetry lock on 0.12 / 1.0 (no --no-update), poetry lock --regenerate (2.x), poetry update (with or without --lock) and poetry update <patched-package> — even when the version does not change — drop the source and restore the registry hashes on every release. metadata.content-hash is unchanged by that, so poetry check --lock / poetry lock --check cannot detect the loss: re-run socket-patch scan --mode … after any of them, or gate CI on socket-patch vex.
  • poetry lock --no-update on Poetry 1.1 / 1.2 keeps the hosted source but rewrites the entry in its own lock-1.1 shape, dropping the package-level files the rewrite added for Poetry ≥ 1.2's hash check; a lock relocked by 1.1 and then installed by 1.2+ installs the hosted wheel unverified. Re-run socket-patch scan --mode hosted after relocking on those releases: the re-scan restores the entry and rebases the ledger's recorded edits onto the relocked text (pristine → current, never an appended chain), so rollback still lands on the pristine lock afterwards. A relocked-but-not-rescanned lock is refused by rollback (its recorded fragments match nothing) rather than reported as already reverted.
  • Poetry 0.12 and 1.0 resolve a relative type = "file" path against the shell's working directory, not the project root; run poetry install from the project root on those releases.
  • Poetry ≤ 1.1 stores its HTTP cache under the user cache directory regardless of POETRY_CACHE_DIR, behind one lockfile that wedges parallel runs and stays wedged after a SIGKILL; the harness gives each legacy case its own HOME.

Mode notes

  • Agent mode patches the interpreter the crawler finds: VIRTUAL_ENV, ./.venv, ./venv, then — for a Poetry project — the virtualenv(s) Poetry placed under its virtualenvs.path (<name>-<hash>-py<X.Y>; every interpreter minor), reproducing Poetry's own placement from POETRY_*, the project's poetry.toml, the user config.toml and the platform default cache dir without running Poetry; else — for a project directory — the global interpreter's site-packages. So a bare socket-patch scan --mode agent / rollback in a default-configured Poetry checkout works; poetry run socket-patch …, VIRTUAL_ENV=$(poetry env info -p) and --global-prefix <site-packages> keep working. virtualenvs.create = false (containers) means Poetry installed into the system interpreter, which the project-marker global fallback covers. Patched bytes survive poetry install, poetry sync and poetry install --sync on every release (same version → no reinstall).
  • Vendored mode works lock-only for locks that list a pure-Python (-none-any.whl) wheel for the package — lock 2.x files, lock 1.0/1.1 [metadata.files]: the inventory carries that sha256 and the fetcher resolves the file through PyPI's JSON API by digest, verifying the bytes again, exactly like uv's lock-only path. Poetry 0.12's bare [metadata.hashes] names no wheel, and a package that ships only platform wheels has no platform-independent choice: those still need the package installed in the discovered virtualenv (vendor_fetch_unverifiable + package_not_installed). Fresh clones of the committed .socket/vendor/ tree and rewired lock install from the committed wheel with no CLI at all (the matrix's fresh-clone leg).
  • Hosted mode works lock-only (redirected: 1 with no virtualenv).

Running the matrix

cargo build -p socket-patch-cli
cp target/debug/socket-patch /tmp/socket-patch-under-test   # rebuilds must not swap it mid-run
python3 scripts/backtest-poetry.py \
  --cli /tmp/socket-patch-under-test \
  --cli-revision "$(git rev-parse --short HEAD)" \
  --output /tmp/socket-patch-poetry-backtest \
  --modes hosted vendored agent agent-oot setup \
  --shapes direct populated crlf pep621
python3 scripts/backtest-poetry.py --render-doc-table /tmp/socket-patch-poetry-backtest/summary.json

The harness bootstraps every release in VERSIONS with uv (Python 3.8.20 for 0.12–1.1, 3.12.13 for 1.2+; Poetry 1.2.2 needs cleo==1.0.0a5), generates a native lock for a one-dependency project (urllib3 = "1.26.18", whose public free-tier patch needs no token), and for each mode checks: exactly one patch applied, pyproject untouched, idempotent re-scan, poetry install into an emptied virtualenv installs bytes matching the patch record's afterHash, poetry check --lock, a fresh clone of the committed state installs the patch, socket-patch vex attests it, a corrupted hash is rejected where the installer verifies, Poetry's own relock keeps the source, and rollback restores every byte and clears the ledgers. Shapes: direct (the committed native fixture), populated (legacy locks with real upstream hashes filled in — today's PyPI JSON API leaves old Poetry's [metadata.files] empty), crlf, and pep621 (2.x [project] tables with package-mode = false). Modes agent-oot (Poetry's default out-of-tree virtualenv via poetry run) and setup (socket-patch setup on a Poetry project, then poetry lock) are informational.

Rust coverage of the rewriters: cargo test -p socket-patch-core --lib utils::poetry_lock vendor::pypi_poetry and cargo test -p socket-patch-core --test poetry_hosted. The committed native locks under crates/socket-patch-core/tests/fixtures/poetry/<version>/ are the harness's original/ inputs (same pyproject, same content-hash).

Results

Poetry hosted vendored agent (in-project venv) agent (poetry run, out-of-tree venv) tamper rejected (hosted / vendored) warm venv re-installed (hosted / vendored) relock keeps patch (hosted / vendored) lock-only vendored
0.12.17 refused (0.x ignores URL sources) pass (crlf,direct,populated) pass (direct) n/a n/a / no n/a / false n/a / false refused
1.0.10 pass (crlf,direct,populated) pass (crlf,direct,populated) pass (direct) pass (direct) yes / no false / false false / false refused (crlf), refused (direct), yes (populated)
1.1.15 pass (crlf,direct,populated) pass (crlf,direct,populated) pass (direct) pass (direct) yes / no false / false true / true refused (crlf), refused (direct), yes (populated)
1.2.2 pass (crlf,direct) pass (crlf,direct) pass (direct) pass (direct) yes / no false / false true / true yes
1.3.2 pass (crlf,direct) pass (crlf,direct) pass (direct) pass (direct) yes / no false / false true / true yes
1.4.2 pass (crlf,direct) pass (crlf,direct) pass (direct) pass (direct) yes / yes true / true true / true yes
1.5.1 pass (crlf,direct) pass (crlf,direct) pass (direct) pass (direct) yes / yes true / true true / true yes
1.6.1 pass (crlf,direct) pass (crlf,direct) pass (direct) pass (direct) yes / yes true / true true / true yes
1.7.1 pass (crlf,direct) pass (crlf,direct) pass (direct) pass (direct) yes / yes true / true true / true yes
1.8.5 pass (crlf,direct) pass (crlf,direct) pass (direct) pass (direct) yes / yes true / true true / true yes
2.0.1 pass (crlf,direct,pep621) pass (crlf,direct,pep621) pass (direct) pass (direct) yes / yes true / true true / true yes
2.1.4 pass (crlf,direct,pep621) pass (crlf,direct,pep621) pass (direct) pass (direct) yes / yes true / true true / true yes
2.2.1 pass (crlf,direct,pep621) pass (crlf,direct,pep621) pass (direct) pass (direct) yes / yes true / true true / true yes
2.3.4 pass (crlf,direct,pep621) pass (crlf,direct,pep621) pass (direct) pass (direct) yes / yes true / true true / true yes
2.4.3 pass (crlf,direct,pep621) pass (crlf,direct,pep621) pass (direct) pass (direct) yes / yes true / true true / true yes

Captured 2026-09-17 on macOS arm64 against the fix-branch head; 108 cases, all passing (the pass/refused cells are the asserted outcomes; the tamper / warm venv / relock / lock-only vendored columns are the measured installer facts the sections above describe). The machine-readable results carry every check, the CLI envelopes' relevant fields and the per-step exit codes. poetry lock on 0.12 / 1.0 is bare (no --no-update), hence relock keeps patch = false there; lock-only vendored = refused on 0.12 and on the unpopulated 1.0/1.1 fixtures (urllib3 = []) because those locks name no wheel hash. The companion SBOM annotation work and its own capture set live in SocketDev/depscan (tools/pipeline/poetry-patch-backtest.py).