Skip to content

Fix vendored uv/Hatch re-vendor to a newer patch (#742, #650) - #943

Open
Mikola Lysenko (mikolalysenko) wants to merge 7 commits into
mainfrom
agent/fix-pypi-vendored-revendor
Open

Mikola Lysenko (mikolalysenko) wants to merge 7 commits into
mainfrom
agent/fix-pypi-vendored-revendor

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Fixes #742
Fixes #650

Summary

A uv project, a uv PEP 723 script lock (or pylock) and a Hatch project that
were vendored with one patch now move to a newer patch for the same package on
the next vendor / scan --mode vendored / get --mode vendored. Before
this, the run failed with exit 1 (pypi_uv_source_already_exists,
pypi_lock_source_already_exists or pypi_hatch_unsupported) and told the
user to revert wiring socket-patch had written itself. The project kept
installing the old patch.

#743 fixed the hosted half of both issues. This PR fixes the vendored
half, which is what keeps both issues open. #766 did the same for
requirements.txt, and #825 does it for Pipenv.

Root cause

The three vendored backends' pre-flight guards (check_target_guards in
vendor/pypi_uv.rs, load_python_locks in vendor/pypi_lock.rs, and
load → hatch::replacement in vendor/pypi_hatch.rs) accept only wiring
for the current patch uuid. They treat socket-patch's own
.socket/vendor/pypi/<older uuid>/ source as a foreign one. Nothing ever
unwound the older uuid's wiring, even though the CLI contract says "a package
the ledger holds at an older patch uuid is still re-vendored automatically".

Fix

All in crates/socket-patch-core/src/vendor/pypi.rs, shared by the three
flavors:

  • supersede_or_refuse: when one of those guards refuses, check the vendor
    ledger. If it holds exactly one entry for the same name and version under
    another uuid, with this flavor and recorded wiring, and a project file still
    references that uuid dir, the plan becomes WiringPlan::Supersede.
    Otherwise the refusal stands unchanged.

    • Hatch reports its uv-installer and Hatch >=1.2 guards under the same
      pypi_hatch_unsupported code. For Hatch, pypi_hatch::preflight checks
      those guards before anything is unwound, so a refusal unrelated to the
      old wiring never touches the project.
  • unwire_superseded, at wiring time after the new wheel is built:

    1. snapshots every file the old entry may write, through the
      group-commit-aware utils::fs readers. Wiring paths from the tamper-able
      ledger must be plain relative paths outside .socket/.
    2. replays the old entry's own revert with keep_artifact. The CLI already
      sweeps the old uuid dir (vendor_stale_artifact_removed) once the new
      ledger entry lands.
    3. requires that revert to succeed with no drift and no remaining reference
      to the old uuid dir.
    4. re-plans the flavor fresh over the restored pre-vendor files
      (fresh_pyproject_plan) and wires the new wheel through the normal path.

    The new entry therefore records the user's real pre-vendor originals, and
    vendor --revert restores them byte for byte.

  • Any failure (drifted wiring, a fresh guard refusing, a wiring error) puts
    every snapshotted file back and sweeps the new wheel, so the tree is left as
    it was. In a vendor run those writes sit inside the group commit. Any file
    that can't be restored is named in the reported error, never dropped. With
    no ledger entry for the old uuid it still refuses before writing, because
    there would be no recorded original to restore.

docs/testing/hatch.md now notes the vendored re-vendor. CHANGELOG.md is
untouched (release-time only).

Also ported: 13f6eee is #878's Gradle digest routing. Main has been red on
production_digests_go_through_the_helpers since #865. The port becomes a
no-op once #878 lands.

Tests (red → green)

Issue Test Before fix After
#742 (uv project, script lock), #650 (Hatch) vendor::pypi::tests::pyproject_flavors_revendor_to_a_superseding_uuid (each flavor: re-vendor to uuid B, originals carried, no uuid A left, revert byte-exact) FAIL (refused) pass
#742, #650 tests/mode_migration_pypi.rs::pyproject_flavors_vendored_revendor_superseding_patch (real CLI: vendor A → vendor B → in-sync re-run → vendor --revert; vendor_stale_artifact_removed, ledger on B only) FAIL pypi_uv_source_already_exists (exit 1) pass
guard pyproject_flavors_superseding_uuid_with_drifted_wiring_refuses (hand-edited wiring: refuses, files byte-identical, no uuid B dir) pass (refused) pass
guard pyproject_flavors_superseding_uuid_without_ledger_refuses, uv_stale_uuid_vendor_refuses_through_orchestrator (no ledger entry: still refuses before writing) pass pass
Bugbot hatch_unrelated_guard_refuses_superseding_patch_before_unwinding (CLI, installer guard set: files, ledger, artifact A unchanged, no uuid B dir) n/a pass
Bugbot restore_snapshot_reports_unrestored_files_and_restores_the_rest n/a pass

Local checks:

  • cargo clippy --workspace --all-features -- -D warnings: clean.
  • The changed files are rustfmt-clean.
  • cargo test --workspace --all-features --no-fail-fast: 10822 passed, 12
    failed. All 12 are chmod 0o555 permission-denial tests that can't fail when
    run as root, which this sandbox is (uid 0). The same 12 are noted on Fix uv/Hatch hosted re-pin to a newer patch (#742, #650) #743,
    and CI runs them as non-root.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SECNaPEKiAJVRwYMMVAcLx


Note

Medium Risk
Changes core PyPI vendor wiring, ledger-driven revert, and atomic file snapshots; mistakes could corrupt lockfiles or leave half-unwound state, though failure paths restore snapshots and add broad tests.

Overview
Fixes vendored re-vendor when the manifest offers a newer patch UUID for the same PyPI release on uv projects, PEP 723 script locks, and Hatch—cases that previously failed with pypi_uv_source_already_exists, pypi_lock_source_already_exists, or pypi_hatch_unsupported because guards treated socket-patch’s own older .socket/vendor/pypi/<uuid>/ wiring as a foreign source.

In vendor/pypi.rs, guard refusals for those codes now consult the vendor ledger: if exactly one older entry still wires the same package/version, the plan becomes WiringPlan::Supersede. At wiring time, unwire_superseded snapshots affected project files, replays the old entry’s revert (keeping the old artifact until the new ledger lands), re-plans fresh wiring on the restored pre-vendor files, then wires the new wheel so vendor --revert still restores user originals. Failures restore snapshots and sweep the new wheel; Hatch runs pypi_hatch::preflight before unwinding so unrelated installer/version refusals leave the tree untouched.

Hatch splits installer checks into require_pip_installer() plus a shared preflight. CLI and core tests cover happy-path re-vendor, drift/ledgerless refusal, snapshot restore errors, and the Hatch uv-installer guard. Docs note vendored supersede behavior alongside hosted re-pin.

Reviewed by Cursor Bugbot for commit 3e28db7. Configure here.


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
A uv project, a uv script lock (or pylock) and a Hatch project vendored
with one patch could not move to a newer patch for the same package:
scan --mode vendored, vendor and get --mode vendored failed with
pypi_uv_source_already_exists, pypi_lock_source_already_exists or
pypi_hatch_unsupported, and the project kept installing the old patch.
The guards treated socket-patch's own wiring as a user source.

When that wiring belongs to an older patch uuid of the same release
and the vendor ledger still records it, vendor now replays the older
entry's revert, keeping its wheel, and wires the new wheel fresh over
the restored files. The new entry records the user's real pre-vendor
originals, so vendor --revert still restores them byte for byte. If
the old wiring was edited since vendoring, or the restored files
refuse the new wiring, every file is put back and the run fails as
before. Without a ledger entry it still refuses before writing.
(#742, #650)

Assisted-by: Claude Code:claude-opus-5-5
Drive the real binary through vendor with patch A, then vendor with
patch B for a uv project, a uv script lock and a Hatch project. Each
must move to patch B, remove patch A's wheel, settle on a re-run and
restore the user's files on vendor --revert. (#742, #650)

Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Claude Code:claude-opus-5-5
Main is red since #865: its production_digests_go_through_the_helpers
guard flags the inline digests #646 added in gradle_cache.rs,
jvm_jar.rs and sidecars/maven.rs. This is the same change as #878 and
becomes a no-op once that lands.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[agent] coverage failed on dc2fadc in utils::digest::tests::production_digests_go_through_the_helpers. That failure isn't from this PR: main has been red on it since #865, whose guard flags the inline digests #646 added in gradle_cache.rs, jvm_jar.rs and sidecars/maven.rs. I ported #878's fix in 13f6eee. It becomes a no-op once #878 lands.


Generated by Claude Code

@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 6, 2026 15:13
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread crates/socket-patch-core/src/vendor/pypi.rs
Comment thread crates/socket-patch-core/src/vendor/pypi.rs
Hatch reports its uv-installer and Hatch-version guards under the same
pypi_hatch_unsupported code as a foreign direct reference, so a
superseding patch could unwind patch A's wiring only for the fresh plan
to refuse on a guard unrelated to it. Run those guards (pypi_hatch::
preflight) once a superseded entry is found, before anything is
touched.

restore_snapshot now attempts every file and names any it could not
write back in the reported failure, instead of dropping the error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SECNaPEKiAJVRwYMMVAcLx
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit efc6d91. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 6, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[burn-down agent] Ready for review at head efc6d91.


Generated by Claude Code

@mikolalysenko Mikola Lysenko (mikolalysenko) removed the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 7, 2026
Main's #825 (#769) changed WiringPlan::Pipenv to carry the superseded
older-uuid ledger entry for its in-place Pipfile.lock re-wire, while this
branch added WiringPlan::Supersede for uv, script-lock and Hatch
projects (#742, #650). Keep both variants: Pipenv keeps its own
re-wire path and the new Supersede plan stays limited to the
pyproject-family refusals it was written for.

Co-Authored-By: Claude <noreply@anthropic.com>
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment