Cleanup: no .socket residue, locks that never outlive a command, manifest-free vendored mode - #247
Conversation
- manifest/operations.rs: parse the manifest straight from the string
(serde_json::from_str) instead of building an intermediate Value and
cloning it into PatchManifest; classify errors with is_data() so the
two historical message prefixes ("Invalid manifest" / "Failed to parse
manifest JSON") are preserved. read_manifest now goes through the
shared FIFO-guarded utils::fs::read_regular_to_string helper.
- utils/fs.rs: entry_is_dir uses the DirEntry's cached file type and
only stats the resolved path for symlinks, removing one stat per
entry on every crawler directory walk while keeping the documented
follow-symlinks contract.
- utils/socket_cli_config.rs: delete the private read_regular_file twin
in favour of utils::fs::read_regular_to_bytes_sync.
Full workspace suite green (239 suites, 7317 passed, 0 failed).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
acquire() now owns the whole lock lifecycle: it creates a missing .socket/ itself (idempotently, inside the retry loop), opens-or-creates apply.lock, takes the OS lock, and verifies via a same_file::Handle that the handle it locked is still the file the path names (dev+ino / volume+index); a mismatch or missing file is an orphan left by a releaser and is retried, never honored. LockGuard::drop unlinks apply.lock WHILE still holding the lock, closes the handle, then best-effort remove_dir()s an otherwise-empty .socket/ (gated to a dir literally named .socket so --manifest-path user dirs are never deleted). A failed acquire prunes the empty dir it may have created. Transient outcomes of racing a releaser's cleanup are retried on their own bounds instead of surfacing as lock_io: open/mkdir failures while the parent is gone (ENOENT, and macOS EINVAL on O_CREAT in a just-rmdir'd dir), std create_dir_all's AlreadyExists-for-a-vanished-dir TOCTOU, and Windows delete-pending codes 5/32/303 (5 ms x 40 grace, independent of --lock-timeout). Waiters drop the File before every backoff sleep so they never prolong a delete-pending window. Only fs2 contention consumes the deadline. lock_cli: acquire_or_emit no longer fails on a missing .socket/; new pub(crate) lock_failure() is the single LockError -> (errorCode, message) rendering for other lock sites to adopt. Tests pin the new protocol: file present while held / gone after drop, empty .socket/ pruned, non-empty left alone, non-.socket dirs kept, squatting file -> Io, orphaned-inode holder never blocks, two-thread acquire/release hammer never double-holds and leaves no residue, waiter-vs-fresh acquire (Ok, Ok) is now a hard failure. e2e_safety_lock asserts apply.lock is gone after every apply run and the manifest survives; apply_invariants' snapshot no longer excludes apply.lock. Adds same-file = "=1.0.6" as a direct core dependency (already in Cargo.lock via walkdir). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…iene (D4) - utils/socket_dir.rs (new): prune_empty_dirs / remove_file_and_prune / remove_tree_and_prune — one "delete, then rmdir the now-empty parents up to but excluding <stop_dir>" implementation, confined to stop_dir's subtree, non-recursive and best-effort; write_json_ledger — pretty JSON + trailing newline + create_dir_all + atomic write, skipping a byte-identical ledger already on disk. - redirect/state.rs: persist_redirect_state now prunes an emptied .socket/vendor/ (the observed residue after a hosted rollback); the unlink error still propagates first. save_redirect_state goes through write_json_ledger (idempotent hosted re-runs no longer churn the committed ledger). CorruptRedirectState gains `unreadable`: an I/O failure or a directory/FIFO squatting the path is reported as "cannot be read" and is never quarantined — quarantine stays for malformed JSON. read_ledger_bytes twin deleted in favour of utils::fs::read_regular_to_bytes. - vendor/state.rs: save_state uses the shared helpers (delete + prune, eco-husk backstop kept), read_state_bytes twin deleted, detached/record docs describe the manifest-free vendored posture, PatchRecord import used. - manifest/cleanup_blobs.rs: cleanup_dir drops the redundant pre-stat (read_dir NotFound is the missing-dir signal), keeps sweeping past a per-file unlink failure (counting only removed files, returning the first error after the pass), closes the ReadDir handle and removes the emptied store directory on a wet run — no more empty .socket/blobs|diffs| packages husks. "No blobs directory found" wording corrected to "No blobs to clean up." (also true for an existing empty dir). - utils/fs.rs: open_regular_file and both async readers are one spawn_blocking hop over the single sync guard; atomic_write_bytes_as folds its five identical stage-cleanup arms into commit_stage; module doc for entry_is_dir updated. - constants.rs: SOCKET_DIR const; stale "currently 3.x" doc dropped. - socket_cli_config.rs: env_flag -> pub env_truthy (doc lists the real vocabulary) so the CLI's byte-identical copy can be deleted. - operations.rs: tests pin the "Invalid manifest" / "Failed to parse manifest JSON" prefix split and the duplicate-struct-field rejection. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ng, nuget/cargo-lock fixes Hosted-redirect reverts (per-purl takeover + whole-ledger replay): - New redirect/staged.rs shared by takeover.rs and replay.rs: FIFO-safe read_rel (read_regular_to_string), staged_read, and one flush_staged with the not-a-regular-file guard on every path (text, binary, delete), atomic_write_bytes_preserving_mode for text AND bun.lockb (replay's plain writer reset the mode), and the emptied-parent prune scoped to sub-directories only. takeover.rs read_rel/write_rel were raw read_to_string/fs::write (FIFO wedge, write-through-symlink, torn lockfile on ENOSPC); replay.rs text flush was a truncating fs::write. - takeover.rs: find_record_key/drop_claimed dedupe the per-purl ledger bookkeeping; package-lock.json is read once (attribution text reused by the JSON replay); registry-uuid regex hoisted to a LazyLock static; per-edit FileEdit clones become borrows; dead CargoRedirectRevert alias removed; dry_run docstrings now describe the in-memory ledger claim. - replay.rs: HatchDocument arm no longer stages a byte-identical write / editedFiles credit when the file is already at its original. Rewriters (mod.rs): - registry_override_of_kind unifies the five override-kind gatings: absent AND foreign-kind overrides both warn the ecosystem's missing-override code (nuget/gem/golang previously skipped silently). - composer/gem/maven grants with no composer.lock / Gemfile(.lock) / pom.xml now warn redirect_composer_no_lockfile / redirect_gem_no_gemfile / redirect_maven_no_pom (once per run; a Gradle-only project keeps its snippet path) instead of vanishing. - nuget: a present-but-corrupt packages.lock.json warns redirect_nuget_lock_unparseable and skips (was silently treated as absent); the re-run probe reads the parsed <packageSources> keys (hand-normalized spellings no longer duplicate the source); a nuget.config authored from scratch records action "added". - plan_cargo_lock: same-name+version twin blocks are line-anchored and disambiguated — exactly one twin at the target index is ours (re-run beside a crates.io copy is a no-op), otherwise the dep is skipped with redirect_cargo_lock_pkg_ambiguous instead of repointing the first hit. - Fixed regexes hoisted to LazyLock statics (cargo toml/lock, composer, nuget, maven dependency block); maven tag scan is a plain find; yarn berry probe is is_berry_lock(); maven pom edited in place via as_mut. - Dispatcher: pub fn pdm_drives(files); withhold() borrows the override set unless a pdm/pipenv veto applies (two unconditional clones gone); requirements::rewrite called directly (pass-through wrapper deleted); rewrite_hatch overlays only pyproject.toml/hatch.toml. - serialize_json: expect() instead of swallowing an Err into "\n". Tests: write_failure_at_flush now makes the parent dir read-only (rename ignores the target's mode); new takeover FIFO/symlink/mode twins, replay mode + hatch-already-original tests, bun.lockb mode assertion, nuget unparseable/hand-normalized/added tests, cargo twin-block tests, composer/gem/maven no-file tests; foreign-kind and gem override tests pin the warn-everywhere policy. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…pnpm fan-out apply.rs - Validate manifest-supplied strings before they become paths: afterHash must be a 64-hex blob hash (new pub is_valid_blob_hash) and uuid a plain path segment; the disk blob is read through read_blob_entry (lstat the ENTRY, refuse symlinks/FIFOs, FIFO-safe open) so a poisoned manifest or a planted blobs/ symlink can no longer read out of tree or leak a hash. - Archive/diff strategies are byte resolvers feeding ONE apply_file_patch_at write site: write I/O errors surface as themselves instead of falling through and misreporting "Failed to read blob". - Borrow the in-memory vendor blob overlay (Cow) instead of cloning per file. - Drop the break_hardlink_if_needed pass: rename-over in atomic_write_bytes already isolates shared inodes. Stat the parent once and hold a single DirWriteGuard (from_metadata) instead of two acquires + a mkdir wash. - verify_file_patch: the opener's NotFound|NotADirectory is the existence probe (is_missing_path); no separate metadata stat per verified file. - A post-rename chown failure is applied-with-warning (advisory on `error` alongside success), with the mode still restored last. - apply_file_patch is now a single-copy wrapper kept for the cargo checksum sidecar; the pnpm fan-out lives at package level only. rollback.rs - rollback_package_patch = rollback_package_patch_at + pnpm peer-variant fan-out mirroring apply: patch-added files are deleted in every store copy, an already-original primary still heals a patched twin, copy failures aggregate as "pnpm store copy … failed to roll back". - Restore writes through apply_file_patch_at (no per-file store scan) and reads the before blob through the shared read_blob_entry. - verify_file_rollback drops its metadata pre-probe the same way. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ing and shared helpers
Vendor backends (crates/socket-patch-core/src/vendor/*, state.rs untouched):
- gem revert: liveness-aware record restore (`RecordRevert::{Done, Drifted,
FileMissing}`) + the family-wide drift-keep gate (`keep_artifact` on any
`vendor_lock_entry_drifted`); converged files (regenerated pre-vendor
Gemfile/lock, hand-deleted Added block, registry checksum line back) are
silent; a missing Gemfile/Gemfile.lock reports `vendor_lockfile_missing`
instead of drift so the artifact is still removed.
- gem vendor: the pure Gemfile.lock edit is computed before any download /
copy / write (a lock-shape failure no longer builds and unwinds); the
copy hash + stub probe run only once the lock is known wired.
- npm / yarn-classic / yarn-berry / pnpm-v9 reverts honor `keep_artifact`
in the unwired-revert guard (bun + pnpm-legacy parity; doc/code mismatch).
- pypi flavors: shared `refuse_symlinked` (poetry / pipenv / requirements
gain `pypi_<flavor>_symlink_unsupported`; pdm / uv / pylock reuse
`utils::fs::first_symlink`) and `ensure_unchanged` pre-write snapshot
re-verification (`pypi_{poetry,pdm,pipenv,uv}_changed`) mirroring the
pylock flavor; `wire_pipenv` takes the orchestrator's version instead of
re-parsing the wheel filename.
- golang service leg stages the module zip at `<copy>.socket-stage` and
swaps it in only once verified (cargo / composer / gem shape): a failed
re-download keeps a wired copy + directive; a missing copy keeps the
dangling-directive teardown. go-patches takeover prune loop simplified.
- D4 residue: one shared `common::prune_empty_vendor_levels` (uuid -> eco ->
`.socket/vendor`, never `.socket`) replaces the cargo / gem / composer
copies and runs after every wet non-keep revert (all 9 ecosystems) and on
the golang / maven / nuget / pypi / gem failure legs.
- cargo_config: the never-shipped `.socket/cargo-patches` legacy takeover is
removed (writer ed435b4 and its removal 5356bb6 both first appear in
v4.0.0; no tagged tree carries the writer); such entries now refuse as
user-authored.
- Guarded readers: 20 private `read_regular*` twins consolidated onto
`utils::fs::{read_regular_to_string, read_regular_to_bytes,
read_regular_to_string_sync}`; bun revert reads bun.lock through the
guard (FIFO wedge); `verify::file_sha256_hex` opens once via
`open_regular_file`; harvest's tarball decode runs on `spawn_blocking`.
- Waste: `revert_lock_fragment_splice` writes only when a fragment changed;
`zip_bytes_match_after_hashes` + `read_zip_artifact` (256 MiB cap) let the
maven / nuget hot paths read the committed archive once; maven pom fetch
streams through `utils::http::read_capped`; bun's prior-artifact hash is
gated on a digest-less own tuple; `NpmStagedPack::uuid_dir_preexisted`
replaces six per-backend stats; pypi dist lookup checks stem-matching
dist-infos first; pipenv stale-install probe hoisted; `MetaSlot::Uv`
no longer wraps an always-Some option.
- Stage/swap helpers (`stage_dir_for`, `backup_dir_for`,
`swap_stage_into_place`) hoisted from cargo / composer / gem into common.
Tests: new preserve-state empty-wiring twins (npm, yarn-classic, yarn-berry,
pnpm), bun revert FIFO, poetry/pipenv/requirements symlink refusals,
changed-during-vendoring refusals (poetry/pdm/pipenv/uv), golang staged
service rebuild (stale copy kept / missing copy torn down), converged-splice
inode pin; gem drift tests re-pinned to the liveness contract; cargo legacy
tests inverted to refusals.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…te hygiene - api/client: build the header-free proxy client once per ApiClient instead of per blob/diff/tarball download; one `patches_path` builder for the four JSON routes, with `fetch_registry_references_for_org` honoring a per-call org override (one-arg wrapper kept); `get_api_client_with_overrides` builds the client once and fills the auto-resolved slug in place. - telemetry: drop the VITEST kill-switch relic (cleanup ruling: no in-repo harness sets it); add a 2 s connect timeout so a blackholed endpoint no longer stalls every command for the full 5 s request budget. - setup: replace four private FIFO-guarded readers with utils::fs::read_regular_to_string; gem `--remove` prunes an emptied `.socket/`; pyproject.toml edits preserve CRLF (toml_edit re-emits LF); `add_plugin_directive_with` lets the CLI probe bundler once per run. - update/swap: only a contention errno maps to `update_in_progress`; other flock failures surface their real cause. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…den the retry bound The full-suite run tripped the new acquire/release hammer test with "failed to open lock file: Invalid argument (os error 22)": macOS reports an O_CREAT open inside a directory that a releaser rmdir'd a moment ago as EINVAL, and open_failure only treated that as the benign "vanished" race when the parent was STILL missing at classification time. Under load the competitor recreates the directory in between, so the race was misreported as a hard I/O fault. - open_failure: a Unix EINVAL from the open is always Vanished (we never pass invalid flags, so it has no other cause on this path). - VANISHED_LIMIT 16 -> 256 with a yield_now between attempts: each vanished outcome means a competitor completed a whole cycle, so the bound only guards against a pathological non-cooperating actor. Verified: apply_lock module 20/20; hammer test 25/25 in a loop; clippy. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dger/inventory loads
D3: a mode-less human `scan` (no --mode/--apply/--sync/--vendor/--redirect,
no --prune) with a non-TTY stdin and no --yes now stops before the download
with exit 0 and the get-hint — never downloads, never creates .socket/. The
gate is a scan-side pre-check before confirm(); confirm() keeps its non-TTY
auto-accept so every explicit-intent flag (and every other command) still
proceeds unattended. --prune counts as intent.
Hosted human arm: no longer returns straight into the redirect engine — it
shares the results table and update detection with the other modes and
confirms ("Redirect N package(s) to the hosted patch server?", default yes,
skipped by --yes / --dry-run) before handing the selection to
run_redirect_selected, the same entry `get --mode hosted` uses.
D2 follow-through in scan: the four vendored-mode fall-throughs that let an
empty discovery/selection reach the vendor step (re-vendor-from-manifest)
are plain early returns; `--detached` is hidden (still requires vendored
mode, still a no-op); update detection folds the vendor ledger's embedded
records too (manifest > redirect ledger > vendor ledger), so manifest-free
vendored projects keep their updates[] signal; the corrupt-ledger supplement
fallback also recovers purls from the committed artifact leaves.
Fewer redundant reads/copies: .socket/vendor/state.json is loaded once per
run (supplement + prune/skip key set + updates); the lockfile inventory is
parsed once (LockfileSupplement::entries feeds the hosted-wiring probe); the
whole-manifest clones, the per-batch purl Vec and the third purl set are
gone. Shared helpers replace the duplicated JSON-vs-human blocks: the
detail-fetch loop (fetch_patch_details), the agent skip partition
(partition_agent_selection + lockfile_only_contains), the \r-vs-plain
progress pairs, the get-hint. The feature-gate relic install hint is one
literal.
Tests: discovery.rs merge/supplement/key-set units re-pinned + new vendored
fold and leaf-recovery cases; mod.rs probe tests take the inventory
explicitly; covgap_commands_scan_mod.rs gains the non-TTY report-only /
explicit-intent / --prune / empty-discovery / hosted table+confirm tests and
a PTY hosted-decline twin; cli_parse_scan.rs pins --detached hidden.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dering, residue-free reverts, qualified-key repair resolver vendor.rs - Standalone `vendor` without a manifest stays a lock-free exit-0 no-op, but names the MANIFEST: "No manifest found, nothing to vendor." / "No manifest to vendor from; N vendored entr(y|ies) tracked in the ledger — `socket-patch repair` verifies them." (the old "No .socket folder found" was false on every hosted-only / vendored-mode project). - API client + VendorServiceConfig are built only for the vendoring arm, after the no-manifest no-op and before the lock; `--revert` never touches the API. - The lock is released right after note_vendor_supersedes_redirect (which may persist the redirect ledger) and before JSON print + telemetry; the `--revert`-without-.socket skip stays (a no-op revert must never create .socket/), comment rewritten for D1. - vendor_records: one load_state for the run (fails vendor_state_unreadable before the crawler walk and any registry traffic); lockfile inventory built lazily and shared by the fetch rung and the --offline detail. - reconcile_dropped: per-purl save mirroring --revert (no write when nothing was reverted; a failed save is a counted vendor_state_write_failed event). - run_vendor_gc: pass (b) applies to detached entries too (D2 — the probe asks the lockfile, not the manifest; (a) keeps exempting them); acquire distinguishes Held (unchanged skip marker) from Io (distinct lock_io marker + human warning) and honors --lock-timeout; ledger written only when a pass removed something; failed ledger/manifest rewrites are surfaced as pass-level markers instead of `let _ =`. - Orphan sweep and stale-uuid removal use core's remove_tree_and_prune so no empty .socket/vendor/<eco>/ or .socket/vendor/ husk survives a revert. - Docs rewritten to the D2 posture (vendored modes are always detached; this command is the one manifest-driven writer). repair_vendor.rs - Installed copies resolved with find_packages_for_rollback: qualified ledger keys (gem ?platform=, pypi ?artifact_id=, maven ?classifier=) never matched the base-keyed map, so installed packages read as absent. - repair_vendored_artifacts_with_references(...) added; the old signature is a wrapper that scans, so repair.rs can pass its hoisted scan (handoff). - A crashed set-aside's `<uuid>.pre-rebuild` leftover is put back when the live dir is absent (the only copy the wiring points at), wet runs only. ecosystem_dispatch.rs (one-line ownership exception): #[allow(dead_code)] on find_packages_for_purls — no production caller remains; CI's lib clippy would otherwise fail. Integration pass: delete or cfg(test). Tests: gc detached test flipped to assert (b) reclaims; new lock_io marker, no_manifest_message, orphan-sweep prune, reconcile save-failure, ledger-aware wording, qualified-key repair and pre-rebuild leftover tests; gem scan --vendor tests made manifest-agnostic (no manifest reads, revert instead of manifest-edit reconcile) ahead of the D2 scan flip. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… collapse Bun vendored-detached leg Update every pinned description to the round-1/round-2 behavior (decisions D1-D4): - Lock lifecycle: apply.lock never outlives a command (acquire mkdirs .socket/, drop unlinks while held and prunes an empty .socket/); repair loses its lock-cleanup role; hosted scan/get take the lock around the first wet write; --lock-timeout / lock_held rows name every lock-taking subcommand. README drops the gitignore-apply.lock recipes and the repair housekeeping claims. - Vendored mode is manifest-free: scan/get --mode vendored write only .socket/vendor/**, every ledger entry carries detached:true + record, --detached is a hidden compatibility no-op (still requires vendored mode), the JSON download vocabulary is the detached one (downloaded/skipped/failed, detached:true), "whole manifest is vendored" is retired, legacy manifest records migrate into the ledger, list reads the vendor ledger ((vendored) marker, exit 0), --prune's lockfile-unused leg covers every ledger entry, standalone vendor without a manifest is a clean no-op that names the manifest. - Normal non-TTY scan without --yes/intent flag is report-only; human scan --mode hosted prints the table and confirms once. - Reversal residue rule: emptied ledgers, vendor husks and blob/diff/package stores are pruned, .socket/ goes with the lock; the zero-patch manifest, setup-owned files and .corrupt quarantines survive. Setup property 5/8 name the real write set (.socket/.gitignore, gem-plugin-stamp location, composer.json) and setup --remove's .socket/ prune; the composer feature-gate sentence is corrected. - Round-1 handoffs: redirect warning codes (composer/gem/maven/nuget/cargo), nuget "added" edit action, unreadable-vs-malformed ledger split, gem drift-keep parity + vendor_lockfile_missing, golang staged service leg, pypi symlink/changed refusal rows, pypi_pipenv_invalid_wheel retired. - CHANGELOG [Unreleased]: Changed (BREAKING) + Fixed bullets naming every user-visible change; Bun preflight bullet and matrix wording updated. - Bun backtest harness + workflow + docs/testing: vendored and vendored-detached have the same footprint, so MODES collapses to hosted/vendored, ledger_record reads state.json for vendored, --detached is no longer passed, and every cell asserts no .socket/manifest.json. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…T gate, drop cow.rs - telemetry: REVERSE the round-1 ruling — restore the `VITEST=true` kill-switch (+ its two integration tests). socket-cli's vitest suite (packages/cli/test/integration/cli/cmd-patch*.test.mts) spawns this binary with inherited env and no SOCKET_TELEMETRY_DISABLED; the gate is load-bearing, and its doc comment now names that dependency. - patch: delete `patch/cow.rs` (no production caller — the rename-over write in `utils::fs::atomic_write_bytes` is the documented CoW defense, its doc now carries the guarantee), the `pub mod cow`, and the cow_* tests in the CLI's e2e_safety_internals.rs; `pkg_managers.rs` / `sidecars/cargo.rs` / `package.rs` comments repointed. The cargo sidecar writes through `apply_file_patch_at`; the callerless `apply_file_patch` wrapper is gone. `apply_lock::is_lock_contended` is pub(crate) and update/swap.rs uses it instead of its inline errno compare. - utils::fs: the last private guarded-read twins are gone (package_json/find.rs read_project_file_to_string, sidecars/cargo.rs read_regular_file, update/state.rs read_state_bytes) and the four crawler inline open+read expansions collapse onto read_regular_to_string; `open_regular_file_sync` is pub(crate) so vendor/verify.rs's wheel audit uses it instead of its own O_NONBLOCK open. - vendor: `common::prune_empty_vendor_levels` is now a thin wrapper over `utils::socket_dir::prune_empty_dirs` (stop dir = `.socket/`), npm_common's inline eco/vendor rmdir pair goes through it, and every per-unit revert (npm/yarn/pnpm/bun/bun-binary/maven/nuget/composer/gem/pypi) removes its uuid dir with `remove_tree_and_prune`. One `state::write_marker_or_warn` emits the single `vendor_marker_write_failed` code at all 17 marker sites (cargo/golang/pypi's `marker_write_failed` retired); pypi's fresh path no longer fails a fully-wired vendor on a marker write error (test inverted). vendor/mod.rs: D2 wording + re-export placement. - redirect: `RedirectState::record_keys_for`, `utils::purl::canonical_purl` and the shared `purl_name_version` / `parse_name_version` replace the repeated canon closures and takeover.rs's `parse_npm_purl`; the edit anchor probe reads `Value::String` payloads in place. `HATCH_FILES` is the one list for the hatch planner and the redirect overlay. - api/blob_fetcher: the cache dir is created inside `write_cache_entry_atomic` on the first verified download — the three up-front create_dir_all blocks and `all_failed_result` are gone, so a fetch that lands nothing leaves no `.socket/blobs/` (covgap tests rewritten; new all-404 residue test). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
scan --mode vendored now takes the detached path unconditionally:
records are fetched in memory (download_patch_records), the vendor
engine gets them as detached records with embedded copies, and
.socket/manifest.json is never written or read as a record source.
The manifest-mode branches in both arms and in run_scan_vendor_step
are gone (download_and_apply, the "whole manifest is vendored"
re-vendor, reconcile_dropped, invalid_manifest, the Option<Envelope>
error payload); --detached stays an accepted no-op and
download.detached: true stays on the JSON sub-object.
- run_scan_vendor_step: empty selection is a no-op BEFORE the lock
(nothing to vendor => no .socket/ created); the pre-lock
create_dir_all + socket_dir_unwritable code is gone (acquire owns the
dir, a file squatting on .socket is lock_io); lock failures render
through lock_cli::lock_failure (--lock-timeout wait clause); the
guard now covers the ledger migration and the redirect-ledger
reconcile in note_vendor_supersedes_redirect (was an unlocked RMW).
- Legacy manifest-mode projects are migrated on their next vendored
run (migrate_legacy_manifest_records): a same-uuid legacy entry is
upgraded in place (detached + record), every manifest record the
ledger owns is dropped, an emptied manifest stays {"patches":{}} (D4);
reported as run-level warnings vendor_manifest_record_migrated /
vendor_manifest_migration_failed, never as a run error.
- GC runs AFTER the vendor step in both arms (the step never reads the
manifest; the sweep reclaims what the run orphaned).
- gc.rs: run_apply_gc gates on manifest existence BEFORE acquiring
(a bare project's scan --prune never creates .socket/), honors
--lock-timeout, and distinguishes Held from Io; both are recorded as
an additive `skipped: {code, message}` on the apply shape instead of
a silent all-zero pass. absorb_vendor_gc carries
VendorGcSummary.failed as `failedVendoredEntries` (apply shape) plus
a human line; the vendored half's lock marker feeds `skipped`. The
dead `skipped: bool` field and its two arms are deleted.
- boxed_scan_vendor_step keeps its signature for get.rs: Some(records)
runs the manifest-free step, None the pre-D2 manifest-mode shim
(legacy_manifest_vendor_step) until get passes its records.
Tests: owned suites re-pinned to the new footprint (no manifest after
vendored runs, detached entries with records, apply.lock gone, empty
.socket/ gone, download vocabulary downloaded/skipped/failed); new
migration unit tests + e2e; GC lock-skip / lock-io / --lock-timeout /
pristine-project unit tests; covgap error tests re-fixtured (real work
+ external lock / dir squat / contentless view).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…idate vector
Hosted engine (`scan --mode hosted`, `get --mode hosted`) fixes per the
.socket hygiene decisions and the cli-scan-hosted triage:
- D1: acquire `<manifest dir>/apply.lock` before the redirect-ledger load,
only on a WET run with at least one granted reference — the only runs
that can write (takeover pre-reverts, ledger merge, lockfile writes).
Dry runs and zero-grant runs never lock, so previews create no
`.socket/`; a granted run that writes nothing leaves none either (the
guard's drop unlinks + prunes). Contention renders through
`lock_cli::lock_failure` into the hosted envelope (top-level
`errorCode: lock_held|lock_io`) plus `Error (<code>): …` on stderr and
the `--lock-timeout` hint for a live holder.
- P1 ordering bug: a symlinked wiring file was detached by the wet
vendored→hosted takeover revert BEFORE the fail-closed symlink guard
ran. A pre-check over each takeover entry's recorded wiring files now
refuses first (same code, dry-run parity); the general guard and the
bun.lockb check share one `refuse_symlinked_file` helper.
- Vendor ledger loaded once for the takeover and retained in memory per
reverted purl (no per-purl reload under the lock); the bun lock
preflight only reads when an npm purl actually has a vendored entry.
- Redirect ledger held as ONE in-memory value: no whole-records clone for
the stale-install probes, no second `RedirectState` construction.
- The `(purl, uuid, url, index, suffixed, go_module)` 6-tuple and the
parallel `overrides` vector collapse into `Vec<Candidate { purl, dep }>`;
the rewriters' slice is materialized once after the last filter, so the
two hand-synced retains are gone.
- Candidate-file reads are skipped when no candidate survived (the rewrite
is provably empty); everything after the rewrite still runs.
- `pdm_drives` (core export) replaces both hand-rolled pdm predicates.
- `redirect_json_block` / `prune_ignored_warning` helpers are the one
spelling of the hosted `redirect` block and prune warning (mod.rs's
zero-discovery arm to adopt them).
Tests: new `hosted_lock_held_refuses_before_any_write` (wet JSON + human
refusal, dry-run and all-skipped runs never contend, no `.socket/` left)
and `takeover_refuses_symlinked_wiring_file_before_reverting` (link,
target, vendored ledger and artifact byte-identical, wet + dry-run +
human) in covgap_commands_scan_hosted.rs; the envelope unit test now
builds its block through the shared helper.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… agent RMW
D2 for `get --mode vendored`: both identifier paths run the detached
download phase (`download_patch_records_with`: records fetched into
memory, no manifest, no blobs — the uuid path hands its already-fetched
view in as `prefetched`, so the proxy-fallback client is never re-hit)
into scan's detached vendor step. `save_patch_record`'s vendored posture,
the `Patch record saved to` block and the whole-manifest `[note]` are
gone; the JSON envelope uses the detached vocabulary (`downloaded` /
`skipped` / `failed`, `detached: true`) with `oldUuid` derived from the
vendor ledger, and the human `[fetch]` line carries `(replacing <uuid>)`.
Both download engines share one fetch loop (`fetch_selected_patches` over
a `RecordStore::{Ledger, Manifest}`), which also serves the views the
release-variant narrowing already fetched instead of fetching them again.
Agent engine hygiene: the manifest read-modify-write and blob writes run
under the apply lock (released before the nested apply, which takes its
own), `.socket/blobs` is created lazily at the first persisted blob, an
unchanged manifest is never rewritten (all-skipped/all-failed runs leave
no `.socket/` behind), and a same-uuid re-get writes nothing. The nested
apply inherits every caller flag (`--lock-timeout` and `--verbose` were
dropped). `download_and_apply_patches_with` / `download_patch_records_with`
take the run's client (+ lock/verbosity flags); the two-arg functions stay
as wrappers for scan and the integration tests.
Also: `is_valid_blob_hash` re-exported from core (local copy deleted),
`effective_org_slug` plumbing removed, identifier regexes compiled once
(UUID via `looks_like_uuid`), the install-hint literal, the unreachable
empty-selection branch and the three `DownloadParams` literals collapsed.
Tests re-pinned to the new behaviour: ledger assertions in the nine
e2e_vendor build suites, in_process_get_modes and get_modes_e2e; lock-first
engine failures, no-residue all-failed runs and the detached vendor step's
hands-off posture toward legacy manifest/ledger state in
covgap_commands_get; new unit tests for the nested-apply arg builders, the
prefetched-view engine call and the ledger-derived `oldUuid`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… ledger-only remove fixes repair: drop the post-drop apply.lock unlink and its pre-lock mkdir (the lock guard now creates .socket/ on acquire and removes apply.lock plus an emptied .socket/ on drop, dry-run included); scan the lockfile vendor references once per run and reuse the manifest-missing gate's result; pass run()'s API client into repair_inner (one token notice, not two); print the blob-cleanup status through core's format_cleanup_result. rollback: thread the manifest and the vendor-ownership key set (loaded once under the lock) into rollback_patches_inner instead of re-reading both; the "No patches found in manifest" line prints only for an unscoped run with no work in ANY leg; never mkdir .socket/blobs (the download creates it on demand; a file squatting on the path is still refused); precompile the path-scope globs once; collapse is_local_redirect / exclude_local_redirects into before_blob_gate_manifest; one manifest clone and an inlined `vendored: []`; surface the engine's ownership-not-restored advisory as a run warning; reword the vendored prompt clause and the pre-lock comment for the new lock semantics. remove: existence probes before the lock (no pre-lock ledger parses); the manifest-less path is no longer gated on entry.detached — any ledger entry without a manifest record is removable through the ledger; the ledger-only path honors --preserve-state and drift-keeps exactly like the manifest path (both now share one revert loop); the vendor ledger is loaded once and the nested rollback receives it; the manifest removal is computed once (no third read, dead post-write not_found arm gone); the two hosted-unwind copies share unwind_hosted; the "(not installed)" line prints only when something was not installed. A zero-patch manifest is never deleted (D4). Shared helpers (rollback.rs): revert_vendor_entry + VendorRevertStep (the silent classifier both commands map to their own vocabulary), sweep_unused_artifacts (the blobs/diffs/packages GC pass), and vendored_purl_keys_of; remove.rs: vendor_entry_matches / vendor_entry_covers_purl (the ledger matching triple). Tests: apply.lock gone after every command, no .socket/ residue after a full hosted/vendored reversal, blobs/ removed when emptied, repair's lock warning retired, repair --dry-run also removes a leftover lock; new ledger-only remove tests for --preserve-state, drift-keeps and entries without the detached flag; the detached-drift silent test now pins exit 1. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ne, one socket-dir derivation apply: build the API client only after the no-manifest and --check exits (and before the lock), thread that one client into staging and the mismatch blob top-up; parse the manifest once (PnP gate + apply loop) and narrow it in place instead of cloning; decide the empty-scope no-op before the ledger read and crawl; mismatch_blob_gaps hashes only files whose afterHash blob is actually missing; the all-unmatched warning prints under --silent (it flips the exit code); "No patch manifest found; nothing to apply." replaces the false "No .socket folder found"; surface core's ownership-not-restored advisory as an ownership_not_restored run warning; release the lock before output/telemetry; merge go/npm local-scope predicates (dead `go` alias gone). list: fold the vendor ledger's detached records in as a third provenance (`vendored`, .socket/vendor/state.json) so vendored-only projects list and exit 0; ledger root = the new GlobalArgs::project_root(). setup: --exclude persistence moves behind the mutation gate (after discovery and confirm; also on the already-configured path for an explicit flag), takes apply.lock for its read-modify-write and reports write/lock failures instead of swallowing them; the manifest is read once per run; --check folds detached ledger records into the property-4 consistency pass; gem/composer discovered once and bundler probed once per run (add_plugin_directive_with); patch_setup telemetry fires only for a successful non-dry run; dead --ecosystems alias tables removed. vex: one vendor-ledger read per run (fold + no-verify + VendorContext via vendor_context_from); json_requires_output usage error no longer fires telemetry. fetch_stage: stage_patch_sources takes the caller's client; offline remedy per stager (vendored: re-run online, never `repair`); is_valid_blob_hash from core. bun_preflight: one preflight core behind the three entry points. path_scope: bind(cwd) absolutizes once. ecosystem_dispatch: "Using <X> at:" banner goes to stderr. update: dead dry_run assignment. update_notifier: core env_truthy. args: project_root()/socket_dir()/socket_dir_of(), --lock-timeout doc, VITEST scrub gone. Tests updated/added for every changed behavior (cli_apply_silent, cli_parse_list vendored twins, covgap apply/setup, setup_contract_gaps detached property-4). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`scan --vendor` no longer writes `.socket/manifest.json`; every ledger entry is detached with its record embedded. Eleven repair tests still read/edited that manifest or expected repair to recover records from it after the ledger was deleted. covgap_commands_repair_vendor.rs - new `to_legacy_manifest_mode` hand-migrates the fixture to the legacy manifest-mode shape (embedded records move into the manifest, entries lose `detached`/`record`) so the manifest-backed arms stay covered: dropped / moved-on manifest record, `(None, None)` uuid recovery (x2), and the three offline soft-restore legs (manifest = only offline record source once the ledger is gone). - no-ledger/offline synthetic-purl test now asserts no manifest exists. - jsr no-backend test stamps the record on the ledger entry instead of a manifest record. repair_vendor_e2e.rs - test 5 renamed vendor_rerun_is_a_noop_and_repair_recovers_registry_ resolution_from_ledger: standalone `vendor` is a `noManifest` no-op (D2 retired the ledger re-vendor path); `repair` rebuilds from the ledger's wiring original. - test 7: a manifest-less reconstruction embeds the API-recovered record (`detached: true`, `record.uuid`); inert blob mount / --download-mode dropped; test 6 cosmetic `--detached` wording. No source changes: every failure pinned pre-D2 behavior. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…lope; re-pin D2 vendored tests
Reconcile the cross-group round-2 breakage between get.rs and
scan/vendor_flow.rs. cli-scan-vendor-gc reshaped run_scan_vendor_step to a
2-tuple error (dropping the carried Envelope) while the contract
(CLI_CONTRACT.md `get --mode vendored`: "a vendor-step error folds the
partial envelope + {status:error, error} in"), get's covgap tests and
scan_vendor_step_error_e2e.rs pin that a step which ran hands its envelope
to the JSON fold demoted to partialFailure.
vendor_flow.rs
- run_scan_vendor_step returns VendorStepResult = Result<(bool, Envelope),
(code, message, Option<Box<Envelope>>)>: lock failures (before the step)
carry None; a staging failure (no_local_source, after the lock) carries
the step's envelope demoted to partialFailure (events-less: nothing
mutates before staging). Scan's JSON fold emits it as `vendor`, exactly
like get's fold already did.
- The pre-D2 legacy_manifest_vendor_step shim (reconcile_dropped /
invalid_manifest / detached=false) is deleted: get passes its records on
both paths, so its None arm had no caller. boxed_scan_vendor_step is now
the one boxed constructor (records by value, no clone) used by scan's two
arms and by get; stage_and_vendor / boxed_vendor_records lose their
always-true `detached` parameter.
get.rs
- run_get_vendored moves `records` into boxed_scan_vendor_step; its fold
was already shaped for the 3-tuple.
Tests
- covgap_commands_scan_vendor_flow.rs: the shared error helper no longer
asserts "no vendor key"; lock_held / lock_io / .socket-file tests assert
it explicitly (pre-lock, no envelope), the staging test asserts the
demoted, events-less envelope (post-lock).
- scan_vendor_step_error_e2e.rs: rewritten for D2. The old trigger (an
unstageable committed MANIFEST + a ledger entry the manifest reconcile
reverted) no longer exists: the step is manifest-free and reconciles
nothing. New trigger: discovery selects a patch whose view carries
hashes but no blobContent, so the detached download succeeds and staging
fails no_local_source. Pins exit 1, error.code, download.downloaded == 1
/ detached, vendor.status == partialFailure, an empty events[] (an
unselected legacy ledger entry is never reconciled; its bytes are
identical), no manifest written, no apply.lock left. Renamed
scan_vendor_staging_error_still_reports_the_reconcile ->
scan_vendor_staging_error_still_carries_the_demoted_vendor_envelope.
- cli_scan_silent.rs scan_vendor_silent_gc_prints_nothing: under D2 the
vendored uuid lives in .socket/vendor/state.json (detached: true, record
embedded), not in the manifest; the seeded agent manifest still loses
its uninstalled entry to the GC and survives as {"patches":{}}.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… line apply's no-manifest no-op now prints "No patch manifest found; nothing to apply." (decisions.md: name the manifest, not the folder — hosted and vendored projects have a `.socket/` with nothing for `apply` to do; CLI_CONTRACT.md's apply row already documents the v5.0 wording). The three integration tests that pinned the retired "No .socket folder found, skipping patch application." text are re-pinned to the new line; exit 0 and the --silent / --json gating they check are unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…; typed vendor-GC outcomes
S1-vendored-pipeline (integration worklist T-A1..A5, T-B1..B3, T-C1..C4, T-G3, T-G9, W-1, W-3, W-4, W-14).
Bug fix (T-A1): `scan --prune --json` labelled every non-purl vendor-GC marker
`gc.skipped.code == "lock_held"` — a `lock_io` fault or a failed ledger/manifest
rewrite read as benign contention. `VendorGcSummary` now carries typed
`skipped: Option<(code, message)>` and `write_failures: Vec<(code, detail)>`;
`failed` holds only purls. gc.rs absorbs them as `gc.skipped` (own reason wins)
and the additive `gc.warnings: [{code, detail}]` (+ human `GC: <detail>.`).
One lock window for `scan --prune` (T-A2): `run_vendor_gc` is split into the
self-locking wrapper and `run_vendor_gc_locked`; `run_apply_gc` gates on
`manifest exists || ledger non-empty` before ONE acquire and runs both halves
under it (a nested acquire reads as Held — flock is per open description — and
silently skipped every revert). The swallowed manifest write now records
`manifest_write_failed`.
One API client per vendored run (T-A3): scan/get thread the run-level client +
`use_public_proxy` through `download_patch_records_with` and into the vendor
step's service config; `download_patch_records` (2-arg, own client) and
vendor_flow's second `get_api_client_with_overrides` are gone (W-3).
`GlobalArgs::vendor_service_config` is the one assembler (T-C3/W-14).
No view fetched twice (T-A4/T-A5): `preverify_vendor_baselines` returns the
views it fetched and runs after the human dry-run return (a preview fetches no
views); the download phase serves records from them and returns a blob seed
(decoded `blobContent` by after-hash) that pre-populates
`stage_vendor_sources_in_memory`, which also takes the caller's ledger load
(`harvest_artifact_blobs_from`, T-G3) instead of re-reading state.json.
Also: repair passes its reference scan + ledger load into
`repair_vendored_artifacts_with_references` (T-C2; the re-scanning wrapper is
deleted, W-4); `vendor --revert` uses the shared `rollback::revert_vendor_entry`
(T-C1); the dead agent-engine Bun preflight gate is deleted (T-B1);
`is_valid_blob_hash` re-export → import (T-B2); `canonical_purl` adopted at the
five hand-rolled sites (T-B3); npm_flavor doc no longer claims detached entries
are lockfile-invisible (T-C4/T-G9).
Tests: gc.rs/vendor.rs GC unit tests re-pinned to the typed fields (+ locked-body
test); fetch_stage seed/ledger tests; get.rs blob-seed pin + cfg(test) 2-arg
helper; discovery preverify view-cache tests; service-config tests moved to
args.rs; covgap_commands_get staging-error tests re-fixtured on a contentless
view (they relied on the second fetch this removes); scan_vendor_e2e pins one
view fetch per patch for both arms.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s map, one JSON printer Hosted engine (T-A7, T-A8): the vendored ledger is loaded once per run under the apply lock (hoisted out of the takeover block) and the post-write overlap classification runs over the in-memory ledgers (`classify_overlap_takeover_with` over the merged redirect ledger and the post-takeover vendor state) instead of re-reading both files; the stale install probes (gem + python) take the single merged records map, so the last `records.clone()` is gone. `overlapping_ledger_purls` becomes a test-only load-then-derive wrapper over `overlap_from_states`. T-A6 adapted: the engine keeps loading the redirect ledger itself, under the lock (a pre-loaded copy handed in by scan would be read before the lock and could merge over a concurrent writer's edits, breaking the D1 lock-before-load invariant). The duplicate corrupt-ledger report is fixed at its source instead: a hosted scan mutes the lenient `updates[]` consult's warning, so the corruption prints exactly once as the engine's hard error (pinned in in_process_redirect). scan/mod.rs: the agent arms drive `download_and_apply_patches_with` with the run's client and `--lock-timeout` (T-A11); the 2-arg `download_and_apply_patches` wrapper and `api_client_for` (now test-only) leave production, with the 15 integration-test call sites ported through a local helper (W-2). The gem bundle-store re-probe is gone: the crawl hands back its `skipped_config_path` (`RubyCrawler::crawl_all_with_discovery`, `crawl_all_ecosystems` 3-tuple — T-A10/T-G7). The zero-discovery hosted JSON arm uses hosted.rs' shared builders (T-A13); `PathScope::bind` once outside the scan and rollback hot loops (T-A14); the vendor-GC comment corrected and every `--json` envelope goes through one `output::print_json` (T-A15, W-15). core (T-A16/T-G1, W-10): `VendorState::purl_keys` is the one vendor-ledger key derivation; `vendored_purl_keys(root)` delegates and the CLI copies in scan/discovery.rs and rollback.rs (`vendored_purl_keys_of`) are deleted. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… → nested apply; small consolidations T-A12 — `apply::run` is split at its acquire: `run` keeps the noManifest / PnP / `--check` gates, the client build and `acquire_or_emit`; `pub(crate) run_locked(args, manifest_path, &client, LockGuard)` is everything from the manifest read on (layout gate, apply loop, embedded VEX, output, telemetry), releasing the guard it was handed once the last mutation is done, exactly where apply's own release sat. Agent-mode `get` and `scan --apply/--sync` now keep their manifest-write guard alive and hand it — with the run's ONE client — to the nested apply through `run_nested_apply`, so download → manifest write → apply is a single lock window and the nested apply never re-acquires or builds a second client. `save_patch_record`'s acquire is hoisted into `save_and_apply_patch`; the uuid path's nested apply runs on the (possibly proxy-fallback) client the fetch used. `nested_apply_args_from_params` no longer re-threads the API flags (inert now); `resolved_api_overrides` folds into the test-only `api_client_for` and its two unit tests go with it (their subject — the nested apply's own org resolution — no longer exists). T-E1 — the yarn-PnP human-mode negative pin names the v5.0 no-manifest line. T-E2 — `output::read_yes_no` is the one stdin yes/no reader; `confirm` maps an empty answer to its default, setup's `confirm_proceed` proceeds only on an explicit yes; every pinned prompt string is kept. T-E3/T-G4/W-12 — `api::client::resolve_ambient_credentials` holds the credential chain (flag → SOCKET_NO_API_TOKEN veto → env → socket-cli config, with the debug echoes); `get_api_client_with_overrides` calls it, and the CLI's local commands (`list`, `setup`, `vex`) resolve telemetry attribution through `GlobalArgs::telemetry_credentials` instead of list.rs's hand-rolled mirror (which also served setup and vex). T-E4/W-6 — the dead `find_packages_for_purls` leaves production; the in-file tests keep a test-side helper to pin the base-keyed contrast, and every comment naming it is repointed. T-E5 — the `VITEST` telemetry kill-switch scrubs return to args.rs's telemetry harness and the repair telemetry lifecycle test (the switch itself was restored in f09f643). T-E6 — `Commands::Repair` doc drops the lock-reset claim. T-E7 — the gem setup branch takes a `GemEdit` (`Add(&BundlerProbe)` / `Remove`) from a `(project, probe)` pair discovered once, so the unreachable probing `add_plugin_directive` arm is gone. T-E8 — the machine-probe tests assert exactly one `bundle --version` spawn per run; the host gem roundtrip pins that `setup --remove` leaves no `.socket/`. Verified: cargo check --workspace --all-targets; clippy --workspace --all-features -D warnings; core api::client + setup::gem unit tests; the whole CLI lib unit suite (583); get_nested_apply_api_flags_e2e, covgap_commands_get, e2e_safety_lock, apply_invariants, output_modes_e2e, interactive_prompts_e2e, e2e_safety_yarn_pnp, covgap_commands_setup, covgap_setup_gem_version, in_process_remove_repair_lifecycle, telemetry_e2e, covgap_commands_list, ecosystem_dispatch_e2e, in_process_rollback_all_ecosystems, cli_config_fallback, covgap_commands_vex, and setup_matrix_gem under --features setup-e2e on the host (19/19). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… sweep failures, ledger helpers in core Integration step S4 of the cleanup (worklist T-D1, T-D2/T-G5, T-D4/T-G2, T-D5, T-D6, W-5, T-F7): - scan/gc.rs reuses rollback's sweep_unused_artifacts instead of a fourth inline GC-sweep copy; every sweep arm (repair, rollback, remove, gc) now reports per-file unlink failures through CleanupResult.failed instead of losing the partial counts behind the first error (core manifest/cleanup_blobs.rs records them after the pass). - The ledger-matching helpers (patch_matches, VendorEntry::matches_identifier / covers_purl) move into core next to lookup_entry; remove.rs and rollback.rs call them. - One .socket derivation via GlobalArgs in remove/repair/vendor/rollback (drops the wrong "." fallback under a non-default --cwd); the #[cfg(test)] path-taking rollback_patches wrapper is gone (tests use the inner engine). - New covgap_commands_get test: `get <uuid> --mode hosted` under a held lock reports top-level errorCode lock_held, exit 1, and --dry-run still exits 0. - Fixed the moved patch_matches unit test: a PURL identifier is compared against the purl field only, so the negative case needs a non-matching purl. Verified: cargo check --workspace --all-targets; clippy (CI invocation) clean; core purl/state/cleanup_blobs units; CLI lib rollback/remove/repair/gc/vendor; repair_invariants, covgap_commands_repair, in_process_remove_repair_lifecycle, covgap_commands_rollback/remove, rollback(_duality)_invariants, cli_remove_silent, covgap_commands_get, scan_vendor_e2e all green. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… five patches routes
`fetch_patch`, `search_patches_by_{cve,ghsa,package}` and
`search_patches_batch` took an `org_slug: Option<&str>` per-call override
that every caller passed as `None` or as the client's own `--org` value.
The client's configured slug (flag → SOCKET_ORG_SLUG → socket-cli config →
/v0/organizations auto-resolve) is what these routes need, so the
parameter is gone and `patches_path` falls through to it. Only
`fetch_registry_references_for_org` keeps a per-call override (unchanged).
CLI: the `effective_org_slug: Option<&str> = None` constant scan/mod.rs
threaded through discover_selected / fetch_patch_details /
run_redirect(_selected) / boxed_run_redirect_selected /
boxed_vendor_json_path / preverify_vendor_baselines and get's
run_get_hosted is deleted along with those parameters. fetch_stage.rs and
repair_vendor.rs passed `common.org.as_deref()` — the same value the
client was built from — so they simply drop it; as a side effect
`--org ""` no longer builds a `/v0/orgs//patches/view/…` URL there. Two
`#[allow(clippy::too_many_arguments)]` that fell to 6 params are removed,
as are the "org slug is already stored in the client" comments that
explained the now-absent argument.
Tests follow the new arity: core proxy_batch_e2e + client.rs in-file
tests, discovery.rs preverify units (positional `None` gone). No JSON
shape, exit code or human string changes.
Worklist: T-A9 / T-G8.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d code; re-pin the gated Bun tests; manifest-free backtest reads Integration step S6 (worklist T-H1 D-1…D-18, T-H3, T-H2 script reads, T-F5, T-F6, CHANGELOG [Unreleased] audit). CLI_CONTRACT.md — every drift verified against the current code first: hosted confirm literal follows the code (`Redirect N package(s) to the hosted patch server?`), plus the non-TTY default line and the empty hosted discovery exit; `socket_dir_unwritable` retired from the lock paragraph; rollback prompt clause is `delete M vendored artifact(s) and their ledger records`; vendored `oldUuid` + `[fetch] … (replacing …)` and get's `[dry-run] Would download and vendor N patch(es).`; the `gc` sub-object's `failedVendoredEntries` / `skipped` / `warnings` keys, the `GC:` human lines, the one-acquire gate (manifest OR ledger) and the GC-after-vendor-step order; the D2 migration warnings; hosted `lock_io` + envelope shape + takeover symlink pre-check; agent `get`'s lock window, lazy `.socket/` and same-uuid no-rewrite; `ownership_not_restored` (apply + rollback rows); apply/vendor no-manifest literals; `remove`'s manifest-less path, `not_found` vs `manifest_not_found`, `vendor_state_retained` on a ledger-only leftover, sorted removal; the rollback announce-line rule; setup property 5 (`--exclude` after confirm, under the lock, failures reported; `--check` reads the ledger without a manifest); `list`'s lenient-ledger warning; `--lock-timeout` rows name `setup` and every writing `scan`/`get` arm. D-18 (hosted.rs run_redirect doc) had already landed in S3 — no change. README.md — the 'Undo things' rows and the `rollback` section describe the v5 full-state rollback (`targets`, `--preserve-state` row, JSON vendored arrays); `--lock-timeout` row parity. CHANGELOG.md — prompt literal fixed; VITEST removed from the Retired bullet (the kill-switch is restored and load-bearing); the lock bullet covers lock_io / one-window agent get / setup / one-acquire GC; new Fixed bullets for every user-visible change in 278a387..HEAD that had no line: gc skipped/warnings, vendored oldUuid, agent get residue, blob fetcher envelope, ownership_not_restored, remove vocabulary, setup --exclude / --check / telemetry, list warning, repair qualified keys + pre-rebuild + "No blobs to clean up.", sweep-continues-on-failure, vendor_marker_write_failed + pypi marker-only warning, npm-family drift-keep parity, hatch replay credit, corrupt-ledger-once, manifest input validation + pnpm rollback fan-out, `Using <X> at:` → stderr, token advisories, telemetry connect timeout, --update lock errors. Gated tests (compile-verified): mode_migration_bun.rs snapshot() no longer skips apply.lock (every run removes it), assert_pristine_unwound always asserts the pruned `.socket/vendor/` (rollback prunes it too, so the flag was constant), and the takeover asserts the vendor ledger entry (+ detached/record and no manifest for `scan --mode vendored`; the manifest-fed `vendor` driver keeps its manifest record). e2e_bun_lockb.rs:782 verified: that manifest read seeds a legacy manifest-mode fixture for standalone `vendor --offline`; the takeover roundtrip already asserts no manifest. Comment sweeps: cow.rs / break_hardlink_if_needed references → the atomic_write_bytes rename-over guarantee (docker_e2e_pypi/npm, e2e_safety_pnpm/cow); `(requirements::rewrite)`; `scan --vendor --detached` framing → ledger-only vendored mode (e2e_vex_vendor, interactive_prompts_e2e, in_process_rollback_vendored). scripts/backtest-{pdm,pipenv,poetry}.py: vendored record reads come from `.socket/vendor/state.json` `entries[*].record`; pdm's `manifestLeftover` observation becomes the `noManifestWritten` check (rollback-after-refusal probe dropped with it); pipenv/poetry hosted+vendored rollback legs assert no manifest was ever written. No results regenerated (docs/testing pdm interim note stays). Verified: cargo check --workspace --all-targets; clippy (CI invocation) clean; py_compile ×3; `cargo test --no-run --test mode_migration_bun --test e2e_bun_lockb`; e2e_safety_cow, e2e_vex_vendor, in_process_rollback_vendored, interactive_prompts_e2e, in_process_get_hosted_ecosystems green. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… Python bytecode 17086b6 accidentally committed scripts/__pycache__/backtest-{pdm,pipenv,poetry}.cpython-314.pyc alongside the backtest-script edits. Drop them from the index (the files stay on disk) and ignore `__pycache__/` + `*.pyc` so a local run of the backtest harness cannot re-add them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…st stale `--detached` framing `rustfmt --check` over `git diff --name-only 278a387..HEAD -- '*.rs'` (162 files) reported drift in 88 of them: 40 introduced by this branch's refactors and 48 that were already off-format at the baseline and got worse. Format exactly that file set (`rustfmt --edition 2021 <files>`, never `cargo fmt --all`); the 11 untouched modules rustfmt reached through `mod` recursion (redirect/{golang_local,pdm,pipenv,poetry,requirements}.rs, utils/{pdm_lock,poetry_lock, python_lock,python_script}.rs, vendor/{registry_fetch,toml_surgery}.rs) were restored so the change stays inside the branch's footprint. Semantics unchanged; `cargo check --workspace --all-targets` and `cargo clippy --workspace --all-features -- -D warnings` are clean. Comment-only: bun_preflight.rs still described `scan --mode vendored` as having "the manifest-tracked AND the `--detached` download phases", and three in_process_vendor.rs doc/inline comments named `scan --vendor --detached` as the way a detached ledger entry comes to exist. Under D2 every vendored run is manifest-free and writes the detached shape, so the comments now say `scan --mode vendored` and call `--detached` the hidden no-op it is. Tests that literally pass `--detached` to pin the no-op keep their wording. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ke the vanished/delete-pending bounds per streak `LockGuard::drop` unlinked whatever the path named, not the inode it held. After a non-cooperating `rm .socket/apply.lock && touch .socket/apply.lock` a fresh acquirer legitimately locks the replacement, and the orphan's later drop then removed that live holder's file — the next acquirer created a third file and co-held alongside it, and so on: one user-caused race cascaded into a chain of double-holds. The unlink now runs only when `Handle::from_path` still equals the held handle (the exact probe `attempt()` already makes); `NotFound` skips it, and any other probe failure falls back to the unconditional unlink so a cooperating run never leaves a stale lock behind. Pinned by `orphan_drop_leaves_the_live_holders_replacement_file_alone`. The `vanished` / `delete_pending` counters were cumulative over the whole `acquire` call although their docs (and lock-spec.md) say "consecutive": a long `--lock-timeout` wait behind a hot loop of short commands could accumulate its way into `lock_io`, the classification reserved for genuine faults. A `Contended` outcome (a live holder was observed) now resets both streaks; termination is unchanged because every `Contended` pass still checks the deadline. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t behind Three reversal paths missed the D4 shared-helper sweep: - Go agent-mode reversal (`remove_go_redirect`, `reconcile_go_redirects` leg (b), `teardown_failed_redirect`) removed only the module's own copy dir. Go module paths carry slashes, so `.socket/go-patches/github.com/foo/` and `.socket/go-patches/` survived as empty directories after the last module was rolled back — and kept `.socket/` alive for the lock guard's non-recursive prune. All three now use `remove_tree_and_prune` fenced at `.socket/`; a sibling module under the same host/org stops the climb (pinned). - `repair`'s post-verify `remove_vendor_dir` left an empty `.socket/vendor/<eco>/` (and, on the lockfile-only reconstruction path, `vendor/` too) after removing an unverifiable last unit. - `write_json_ledger` created `.socket/vendor/` and then, when the write failed (ENOSPC, a squatter), left it as a husk from a run whose only write failed. It now prunes ONLY the parent it created, fenced at the nearest `.socket/` ancestor, and propagates the original error unchanged; a pre-existing parent is never removed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ollow the list label and v5 rollback `stage_vendor_sources_in_memory` built a second API client per run (a third on `repair`, whose uuid lookups built their own) although every caller already held one — repeating the token advisory and, in `vendor`'s case, the org-slug round-trip UNDER the apply lock. The stager now takes `Option<&ApiClient>`: scan's vendor step passes its client before moving it into the service config, `vendor` passes `service.client`, and repair's vendored phase takes the run's client, seeds its lazy uuid-lookup cache from it and hands it on; `None` (the unit tests' offline arms) keeps the on-demand build. Docs: the contract, README and CHANGELOG described a human `(vendored)` marker on `list` rows that no code prints — the label is `Mode: vendored (recorded in .socket/vendor/state.json)`, the pinned twin of the hosted line (round-3b D-1: docs follow the code; the `(vendored)` token stays the VEX impact-statement suffix). README's command table still said rollback "keeps the manifest"; it now matches the v5 rollback the same document describes twice. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…eep the artifact while a live wiring file still names it The contract promises the drift-kept artifact and ledger entry stay "every backend alike", but `revert_composer_opts`, `revert_maven_opts` and `revert_nuget_opts` still deleted the uuid dir after a `vendor_lock_entry_drifted` skip. For maven and nuget that stranded a live reference: an unrecognized or truncated record over a still-wired pom.xml left `<repository>` pointing at a removed dir, and a `nuget.config` re-serialized by tooling (neither authored element verbatim, same routing) lost the only source its exclusive `packageSourceMapping` allows — and the CLI then dropped the ledger entry, so `repair` had nothing to rebuild from. These three backends classify drift from whole-file fragments, so a converged file (our block already gone, a regenerated pom) reads as drift too; gating the keep on the warning alone — the lock backends' shape — would keep such artifacts forever, against the LIVENESS CONTRACT. The gate is therefore a live reference: the shared `any_live_file_references` probe keeps the artifact exactly while the live `pom.xml` / `nuget.config` / `composer.lock` still names `.socket/vendor/<eco>/<uuid>` (a converged file never does), and `kept_artifact` makes the caller keep the ledger entry and report `vendor_revert_kept`. Composer's stranded refusal stays in front of the loop; the keep is reachable there under `--preserve-state` or through a reference the structural scan does not see. Nuget probes only the root-level basenames its records name (a tampered `../` path is never read). Tests: the maven unrecognized-kind / key-less / original-less tests flip to "kept, dir exists, `vendor_artifact_kept` surfaced" (they pinned the stranding); the converged cases (regenerated pom / config, registry-rewired or emptied composer.lock) additionally pin `!kept_artifact` + dir removed; a new nuget test drives the re-serialized-config keep; the composer keep_artifact tests pin the drift-keep signal. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… engine One `scan --mode vendored` step read `.socket/vendor/state.json` under its single lock window in `stage_and_vendor` (harvest) and again inside `vendor_records` (whose "loaded ONCE for the whole run" was only true per function); the standalone `vendor` command read it in `reconcile_dropped` and again in the engine. `vendor_records` now takes the caller's `io::Result<VendorState>` — the same load that fed the staging harvest — and reports an unreadable one as `vendor_state_unreadable` at the same point as before (after the no-vendorable-purls early return), so the pinned once-only corrupt-ledger error is unchanged. Both callers hold the apply lock across harvest and engine, so the handed-over copy is the authoritative one. The legacy-manifest migration and the vendor-supersedes-redirect classify still reload (residuals): feeding them the engine's in-memory state would need a guard for the entry `persist_vendor_entry` keeps in memory after a failed save, and the redirect ledger the engine mutates per takeover. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…patches_with Comment-only leftover from the final review pass (the two-arg engine was deleted in the integration round). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Wenxin Jiang (Wenxin-Jiang)
left a comment
There was a problem hiding this comment.
maybe worth double checking the CodeQL findings
|
Claude (@claude) review |
|
BugBot review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit e1c8cb6. Configure here.
…timeout `test (windows-latest)` failed the acquire/release hammer test with "failed to open lock file ...: Access is denied. (os error 5)". Windows keeps an unlinked file's NAME unusable until its last handle closes, and every open in that window fails with ERROR_ACCESS_DENIED / ERROR_SHARING_VIOLATION / ERROR_DELETE_PENDING. The lock itself is FREE there — only the name is momentarily unusable — so the condition is transient, exactly like contention. The retry budget, however, was a fixed 40 x 5 ms per streak that ignored the caller's timeout entirely, so a `--lock-timeout 30` run could be failed out with a hard I/O error after 200 ms over something that resolves on its own, and two processes cycling the lock back-to-back on a loaded runner exceeded it outright. Give up only once BOTH a fixed grace floor (2s, generous because the window is bounded by another process's handle lifetime, not by our syscalls) and the caller's own timeout are spent: a zero-timeout try-once still rides out the window on the floor alone, a --lock-timeout run waits its whole budget, and an overflowing timeout waits indefinitely — the same shape contention already had. A live holder seen in between still resets the streak. Verified: apply_lock 21/21; the hammer test 10/10 in a loop; clippy. Windows CI is the real proving ground for the delete-pending path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rewrites `parse_with_uuid_fallback` performs TWO convenience rewrites — `--update [...]` to the hidden `self-update` subcommand, and a bare `<UUID>` to `get <UUID>` — so the name described half the function. It also made every CLI print of any user-supplied argument look like a credential leak: CodeQL's `rust/cleartext-logging` heuristic treats a value whose name matches /uuid/ as sensitive, so tainting the return of the ARGV PARSER marked the entire parsed command line sensitive. That is the source of 7 of the 11 alerts on this PR and a large share of the 45 already open on main — every `println!` of `args.identifier` was flagged. No credential is involved: the values are the user's own arguments and public patch identifiers. Rename to `parse_argv_with_shortcuts` (both rewrites, no false taint). Mechanical: 41 call sites across lib.rs, main.rs, three CLI parse test files and one CLI_CONTRACT.md reference. Verified: cli_parse_main / cli_parse_vendor / cli_argv_non_utf8 green, CLI lib 587/587, clippy and rustfmt clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`test (ubuntu-latest)` failed in `make_served_binary_output_execs` with "Text file busy" (os error 26). Linux refuses to exec a file while any process still holds it open for writing: `fs::write` had closed our handle, but the suite runs tests in parallel threads, so a sibling that forks between our open and its exec inherits the write fd and our exec lands in that window. The race is in the harness, not in the binary under test, so retry the spawn for up to 10s instead of failing the run. Same flake class as PR #139's `coverage` job, which was resolved by re-running. This PR does not otherwise touch update_fixture.rs. Verified: update_notifier_e2e 37/37; rustfmt and clippy clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI feedback addressed
|
* feat(cli): consolidated terminal UI and CLI-wide output polish Scan's progress lines garbled on a real terminal, e.g. "Found 7 patches for 1 packagesatch 7/7)". Hand-rolled `\r` rewrites never cleared to end of line, and warnings printed mid-line. An end-to-end audit of every command (243 verified findings) found the same class of glitch across the CLI. This commit fixes them and moves all terminal-UI logic into one small, unit-tested module. New `crates/socket-patch-cli/src/ui/` (replaces `output.rs`; drops the unused `indicatif`): - `StatusLine<W: Write>`: deterministic status line. Every update writes `\r\x1b[2K` and a width-truncated message. `println` puts a line above an active status. The line clears on finish and on Drop. It is never live off a TTY, under TERM=dumb, in debug mode, or under --json/--silent. - Prompts: `confirm` / `confirm_or_proceed` / `select_one`, built on the testable `confirm_with` core. EOF (Ctrl-D) declines instead of accepting. Typeahead is flushed before prompting. The non-TTY defaults are kept, but their note respects --silent. The dialoguer cursor is restored on Ctrl-C. - Color: one `color_enabled` policy honoring NO_COLOR / CLICOLOR / CLICOLOR_FORCE / TERM=dumb, shared with dialoguer. Table cells are padded before painting, so colored rows align. - Text: `plural`, and a char-safe word-boundary `truncate`. - Core: `utils::notice` quiet switch. Info advisories are muted under --silent/--json and print once per process. Warnings are muted only under --silent. Per-command polish (scan agent/hosted/vendored, get, apply, rollback, remove, vendor, repair, list, lock waits, --update, notifier, vex, setup): - counted nouns instead of "(s)"; - consistent "Error:"/"Warning:" prefixes, with progress and warnings on stderr; - --silent is errors-only, but a failing run still says why; - deterministic ordering; - no developer notes leaking into --help; - dry-run fixes for `get` and `vex`, plus `vex -O -`. Rebased onto #247; its behavior and contract are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * chore: clear Rust 1.93 clippy lints in untouched files Mechanical fixes so clippy --all-targets -D warnings passes on both crates: assert! for a literal-bool assert_eq!, slice::from_ref over a one-element clone, a Tamper alias for a complex test type, non-test items moved above bun_binary's test modules, an eta-reduced closure, an elided lifetime, and an initialized-in-place let. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs(cli): one-line vendor/repair summaries, every help page leak-checked The vendor summary lost its closing period to clap's first-paragraph trim; it is now a one-line summary with the rest in the long help. The repair about now names what it restores (blobs, diff/package archives, vendored artifacts). help_text_hygiene scans every subcommand's page, self-update included, instead of three owned pages. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(core/api): show the API's error message, not the raw JSON body A non-2xx response now reports the body's error.message / message / error string when it is JSON, the trimmed text otherwise, and no dangling colon when the body is empty. A failed binary body read keeps its cause chain like the send error does. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli/scan): hosted --json keeps VEX advisories in vex.warnings The hosted arm dropped the VEX summary's warnings, and under --json they are muted on stderr, so e.g. product_not_iri or an unreadable vendor ledger reached no channel. They now ride vex.warnings (skip-if-empty), as in the agent arm. apply --vex and scan --vex print their summary via the shared vex::format_vex_written. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli/repair): a failed vendored rebuild no longer ends on "Repair complete." repair exits 1 when a vendored artifact cannot be rebuilt, but the human run still closed with "Repair complete." on stdout. It now closes with "Repair finished with errors." on stderr, like a failed download. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli/get): vendored-mode step errors use the shared vendor-step line get --mode vendored printed a raw "Error (<code>): <message>" for a failed vendor step or bun refusal, without the capital, the period or the --lock-timeout hint that scan's vendored arm prints. Both now go through scan::vendor_flow::format_vendor_step_error. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor(cli/get): take the empty-crawl hint from scan's renderer get kept its own copy of "No packages found. Run your package manager's install first."; it now calls scan::render's no_packages_message, so the two cannot drift. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli/scan): vendored --prune prints the same GC line as agent mode The vendored arm hand-rolled "GC: pruned N manifest entries.", which dropped the swept orphan files and said nothing when only files were removed. It now prints gc::format_gc_line, like the agent arm. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli/scan): --json never stops at the interactive patch menu scan hands select_patches its args with json off (scan has no selection_required path), which also meant a free user's scan --json on a TTY with several patches per package got the interactive menu. --json now also counts as --yes there, taking the menu default (the top-ranked patch), as a non-TTY run already did. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli/get): decode the purl in the "already vendored" skip line The detached-ledger reuse printed the raw purl (pkg:npm/%40scope/...), unlike the "already in manifest" line beside it. Both now share format_record_skip, which displays the decoded purl. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor(cli): one lock_held retry hint for the vendor step and hosted scan vendor_flow and hosted each spelled out the same --lock-timeout hint; both now use lock_cli::HELD_RETRY_HINT. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(core/rollback): "roll back" is the verb in rollback errors "Cannot rollback: <file> - <why>" and "Cannot safely rollback." now read "Cannot roll back" / "Cannot safely roll back.", matching the CLI's own wording. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli/setup): the setup preview shows the dependencies hook it writes setup writes both scripts.postinstall and scripts.dependencies, but the preview listed only the postinstall line (the remove preview already showed both). Core UpdateResult now carries the old/new dependencies script and the preview prints "-> dependencies" when it changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor(cli): one "Skipping VEX generation" line for every --dry-run --vex scan's agent arm said "[dry-run] VEX generation skipped. No attestation written." while apply, vendor and hosted scan said "Skipping VEX generation (--dry-run: nothing was <done>).". All four now print vex::format_vex_dry_run_skip. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli): terminal-UX polish from the fresh-eyes e2e review Output fixes found by driving the binary (pty + pipes): - get/scan --json: the nested apply no longer prints per-package "Error: Failed to patch" lines on stderr (regression from eb131ca); ApplyArgs gains a non-CLI `nested` marker carrying the caller's JSON-ness. The closing "Some patches could not be applied" line drops its stale "re-run without --silent" hint (the per-package lines print under --silent now). - Deterministic order: apply walks packages in PURL order; core apply and rollback walk a patch's files in name order, so the file named in "Cannot apply patch: <file>" is stable run to run. - The nested apply's "Patched packages:" separator goes to stderr: piped stdout of `get`/`scan --yes` no longer carries a double blank line. - select_one fits the prompt and options to the terminal width, so dialoguer's line-count erase leaves no menu rows behind at 40-80 cols. - scan: one blank line opens the post-Summary skip/recorded block and the select menu; a report-only piped scan picks without the non-interactive note; the zero-patch result is said once (stdout); a failed API batch is a "Warning: API batch N of M failed" (none for a one-batch scan) and the fatal line reads "Error: The API query failed: ...". - Progress through StatusLine only: fetch_stage's artifact download and in-memory content fetch (no "(mode: diff)" tag), a per-package vendor progress line, the `--update` version check, setup's search/configure/ apply/remove phases, and the contended-lock wait for every direct apply-lock site (new lock_cli::acquire_with_status). get and hosted scan print the shared "Error: Another socket-patch process ..." line plus the lock-file hint (lock_cli::format_lock_error). - Wording: npm drift warning names the live resolution instead of a Debug `Some("...")`; missing-lockfile refusal lists lockfiles once and says "in the project root" for `.`; "All N packages are already vendored; nothing to do."; repair says "No manifest; no patch artifacts to download." on vendored-only projects, "Would rebuild/download ...:" instead of "Dry run - would ...", and ends the hosted-only sentence with a period; vendor warns when the ledger is unreadable instead of "No manifest found, nothing to vendor."; hosted "No patches could be redirected:" items drop their redundant lead; remove's dry-run footer ("Dry run: no changes made.") closes the whole preview; multi-advisory summaries read "(highest: HIGH)", colored; one non-interactive note wording; "Setup cancelled." / "Hook removal cancelled.". - Errors reach stderr: rollback and setup human runs that exit 1 end with a closing Error: line (the per-item detail stays in the stdout report). - setup --remove preview lists only the scripts the file has, values aligned; no "Proposed changes:" header over an empty preview. - Help: no protocol notes in --vendor-url/--patch-server-url/--vendor-source, --lock-timeout names get/scan, setup --remove names Composer, `--update --version` prints "socket-patch <ver>". Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs(changelog): terminal UI polish entry Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(e2e/nuget): accept the singular crawl summary "Found 1 package" scan now prints counted nouns ("Found 1 package (1 nuget)"), so the legacy-layout leg's "packages" substring match found no summary line. Match either number and build the breakdown needle with the right noun. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
First of seven commits that land the manifest-less VEX branch (feat/vex-lockfile-inventory) on top of #247. The branch's ~165-commit history, built on the pre-#247 main 9489b18 with per-PM merge commits, was squashed onto 09956d9 and re-split by concern; the full pre-rebase history is preserved on branch backup/vex-lockfile-pre-rebase. These are the product bugs the per-PM real-toolchain matrices turned up while the manifest-less VEX suites were written. Each is independent of VEX and pinned by a core regression test: - composer (hosted): the rewriter drops the entry's `source` block when it immediately precedes `dist` (one `redirect_composer_dist` fragment edit spanning both, reverted byte-for-byte). Composer 1 and 2.2 LTS silently installed the pristine upstream commit from git whenever the hosted download failed; a hand-ordered source that cannot be dropped warns `redirect_composer_source_kept`. Golden fixture: `source-and-dist`. - gem (hosted): the patch-registry `GEM` section is inserted in bundler's source order (sorted by remote), so a frozen `bundle install` on bundler >= 4.0.19 accepts the converged lock. The `basic` golden and the exact lock expectations move the Socket section first. - cargo (hosted + vendored): a v1 `Cargo.lock` (checksums in `[metadata]`, dependents naming the crate by its full package id) is redirected and vendored correctly: the `[metadata]` line and every dependent's full-id reference follow the source, each fragment its own ledger edit, and `cargo --locked` accepts the result; the vendored detach/restore of a v1 entry is byte-identical. `plan_cargo_lock` keeps #247's multi-source twin disambiguation (`Ambiguous`) and hoisted regexes; the block end now also stops at a trailing `[metadata]` / `[[patch.unused]]`, and the checksum is inserted after a block-final `source` line too. - yarn berry (hosted + vendored): written checksums follow the lock's own spelling — yarn 4.0.0–4.0.2 spell `10c0` checksums as bare hex, so the prefixed form failed `yarn install --immutable` with YN0028. - npm (vendored): npm 12 reifies from the `package-lock.json` it creates beside a committed shrinkwrap, so `vendor` now rewires every present npm lock (siblings first, primary last; an unrewirable sibling warns `vendor_npm_sibling_lock_unwired`), and the in-use/revert probes read every npm lock before deleting an artifact. `select_lockfile` reads through #247's guarded `read_regular_to_bytes`. - npm (hosted): a lockfileVersion 1 redirect warns `redirect_npm_legacy_client` — npm 6 ignores a v1 lock's `resolved` and fails EINTEGRITY against the patched pin. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
npm >= 12 defaults to `allow-remote=none` and refuses (EALLOWREMOTE) every
lockfile entry whose tarball is not served by the configured registry —
exactly what a hosted redirect writes. When `scan --mode hosted` / `get
--mode hosted` leaves a root package-lock.json / npm-shrinkwrap.json
carrying a granted hosted artifact URL, the run now ensures
`allow-remote=all` in the project `.npmrc` (creating the file, or
appending one line with the BOM, CRLF and every other byte preserved) and
records it in the redirect ledger as `redirect_npmrc_allow_remote`
(`created` / `added`).
- core `patch::redirect::npmrc`: npm's `.npmrc` grammar as measured
against npm 12.1.0 (exact `allow-remote` key, last top-level assignment
wins, `[section]` bodies are not top-level, bare-CR line splits, case-
sensitive value), the plan (create / append / already-all / respected
user / env / outer-layer value / unsupported), and the unwinds.
- Every reversal removes exactly what was added once no package-lock
entry needs it: the whole-ledger replay (a new `NpmrcAllowRemote`
inverse, grouped with the npm lock kinds), the per-purl npm revert
behind scoped rollback / remove / the vendored takeover ("last one
out", same transaction, flushed after the lock through #247's shared
`staged::flush_staged`), and the vendored-supersedes-hosted reconcile.
A modified created file keeps the user's lines
(`redirect_npmrc_allow_remote_modified`, surfaced by rollback, remove,
vendor and the reconcile). A symlinked `.npmrc` refuses an unwind at
plan time, before anything is written.
- Hosted run: the `.npmrc` edit rides `rewrite.files` / `rewrite.edits`,
so it is written under #247's apply-lock window, after its whole-run
SYMLINK GUARD, and only after the ledger persisted — never on
`--dry-run` (which previews the write, also for a vendored → hosted
takeover; the root locks are now read for such a preview so the pnpm
`trustLockfile` preview sees the lock the wet run splices). An explicit
user value (project `.npmrc`, user/global/builtin config, or an
`npm_config_allow_remote` env var) is respected and named; a symlinked,
unreadable or bare-CR `.npmrc` is left alone; `--no-npm-allow-remote-
config` / `SOCKET_NO_NPM_ALLOW_REMOTE_CONFIG` opts out. Every variant
warns `redirect_npm_allow_remote` with the whole-tree tradeoff.
- `atomic_write_bytes_preserving_mode` creates its stage with the
destination's permission bits (kept inside #247's `commit_stage`
structure), so a 0600 token-bearing `.npmrc` is never staged world-
readable.
- remove: the hosted leg's advisories are printed inside #247's
`unwind_hosted` (so a run that then fails still reports them) and carried
into the success envelope's `warnings[]`.
Tests: npmrc/replay/takeover/scan unit tests, the flag's parse coverage,
`redirect_npm_allow_remote` (plus #247's invariants: the lock never
outlives the run, dry runs create no `.socket/`, a full rollback leaves no
`.socket/` and never the user's `.npmrc`), and the dry-run takeover
previews in `coverage_fix_scan_hosted_dryrun_vendored`.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…m lockfiles `socket-patch vex` and the embedded `apply` / `scan` / `vendor --vex` now attest hosted and vendored patches with no `.socket/manifest.json` — and with no `.socket/vendor` ledgers either — by reading the wiring out of the project's lockfiles (`vex::discover`). This covers a depscan-opened PR, a clone that never committed its ledgers, and a lock-only CI checkout. Record view (`commands::vex_sources::plan`): four sources — the manifest, the redirect ledger's records, the vendor ledger's embedded records and lockfile discovery — merge into one view. A candidate's record must carry the uuid the lockfile actually WIRES; it comes from the first source that has one, else (online only) from the patch API by uuid, 10 fetches at a time with `get`'s one-shot 401/403 → public-proxy fallback (#247's uuid-only `ApiClient::fetch_patch`). Nothing is written: vex never takes the apply lock, never creates `.socket/`, never writes the manifest, and the `socket-patch.vendor.json` marker is never a record. Gates, applied before hashing and kept under `--no-verify` (which now skips only the hashing): - a vendor-ledger entry attests only while a lockfile still wires its artifact (`vendor_unwired`); a redirect-ledger record only while a lockfile wires its hosted patch (`redirect_unwired`); discovery is authoritative for every uuid a read file mentions, so a rejected mention keeps nothing alive; - `record_unavailable` (offline, 404, refused, transport error — the run continues), `record_mismatch`, `wiring_conflict` (lockfiles wire one package to several patches); - a malformed / unreadable `.socket/vendor/state.json` is the hard error `vendor_ledger_corrupt` (exit 2), mirroring `redirect_ledger_corrupt` — this supersedes #247's degrade-and-disclose posture for `vex` only; `setup --check` keeps #247's `vendor_context_from` / `warn_unreadable_vendor_state` path. Evidence: vendored refs hash the committed artifact (the ledger entry when it names the wired artifact, else one synthesized from the ref); hosted refs hash the installed copies the build CONSUMES (`vex_consumed::hosted_consumed_copies` → core `VendorContext::hosted` / `HostedCopies`: the Go replacement module, the Socket-registry cargo src dir, maven's suffixed version — never a pristine sibling), and with nothing installed a discovered pinned reference attests from its lockfile pin, like in-run `scan --mode hosted --vex`. Discovered refs bypass the Property 7 ecosystem filter. Commands: - `apply --vex` / `vendor --vex` with no manifest attest what the lockfiles and ledgers wire (nothing anywhere keeps the calm exit 0 and removes a stale document; `apply --check` and `--dry-run` never generate); #247's no-manifest lines ("No patch manifest found; nothing to apply.", "No manifest found, nothing to vendor.") are kept; - failed VEX runs carry the discovery diagnostics into `warnings[]` (standalone envelope, embedded envelopes, scan JSON — hosted included); - `manifest_not_found` now means no manifest AND nothing wired; - human output: `Note:` lines for superseded records / fetch failures, phrased omission reasons. Writer hardening: the manifest-driven standalone `vendor` now embeds the patch `record` in its ledger entries too (vendored mode already does, as `detached` entries). `detached` stays the "no manifest owner" flag, so the manifest reconcile, legacy-manifest migration and get/scan idempotency from #247 are unaffected. Every reader of embedded records shares one ownership rule (`commands::vendor_record_is_unowned`): a detached entry's record always stands alone, a standalone `vendor` entry's fallback copy only when no manifest entry covers it (by ledger key or base purl). `vex`, `list` and `setup --check` (`fold_vendor_records`, formerly `fold_detached_records`) all apply it, so one tree never lists "no patches" while its VEX document attests one. `repair` stays narrower: it keeps preferring a manifest that moved on to a newer uuid and falls back to the embedded copy only with no manifest at all. One liveness rule for vex and scan: `scan`'s cross-mode takeover classification (`classify_overlap_takeover`), its `hosted_wiring_retained` warning and `redirectState.wiringLive` ask the same core discovery (`commands::discover_wiring`) and liveness rule (`Discovery::redirect_record_live` / `vendor_entry_live` / `wires_package`, through one `LedgerLiveness` holder per call site) that gate attestation, replacing scan's private cargo / hosted / vendored checks and its looser text scan. The CLI also shares one purl splitter (`utils::purl::purl_parts`), one vendor-ledger lookup (`vendor::state::lookup_entry_kv`) and one npm alias-aware identity crawl (`ecosystem_dispatch::npm_paths_by_identity`) across vex, scan and vendor, and pairs PEP 723 script locks through `utils::python_lock::script_of_lock` like the rewriters do. Output follows the conventions from #248: `ui::plural` counts, a `ui::StatusLine` progress line for record fetches, and the shared `format_vex_written` / `format_vex_dry_run_skip` lines on the manifest-less `apply --vex` / `vendor --vex` paths. Manifest-less `vex` honours `--dry-run` and `-O -` like the manifest path. Product auto-detection adds go.mod, composer.json, pom.xml, a single `*.csproj` and a single `*.gemspec`, after the existing probes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…pers
Hermetic suites (wiremock patch API, run in the default `test` job on
every OS): one `e2e_vex_lockfile_<pm>` per package manager — npm, pnpm,
yarn (classic + berry), bun, cargo, golang, uv, poetry, pdm, hatch,
pipenv, pip, gem, composer, maven, nuget, deno (negative) — covering
spoofed hosts, record mismatch, pristine / tampered installs, contested
and orphaned wiring, `--offline`, reverted locks (also `--no-verify`) and
the embedded commands; plus the cross-cutting `e2e_vex_manifestless_
embedded`, `e2e_vex_redirect`, `e2e_vex_vendor` and the
`vex_e2e_common_selftest` harness checks.
Real-PM capstones (gated like the existing suites, `_REQUIRED` env
pattern): every hosted / vendored build suite now ends in the manifest-
less VEX matrix over a fresh checkout — manifest absent (ledgers kept),
ledgers deleted (lockfile + API), offline (`record_unavailable`, zero
requests), reverted lock (never attested, verify or not) — through shared
helpers: `vex_e2e_common` (+ bun / uv matrices), `npm_e2e_common`
(+ `manifestless`), `yarn_berry_common`, `common/yarn_classic_vex`,
`common/bundler_e2e`, `cargo_e2e_matrix`, `golang_e2e_matrix`,
`maven_build_common`, `composer_e2e_common`, `vex_pdm_hatch_common`,
`vex_pipenv_pip_*`, `vex_pypi_real_common`. New real-toolchain suites:
`e2e_redirect_{composer,maven,uv}_build`, `e2e_vendor_maven_build`,
`e2e_golang_workspace_build`, `e2e_nuget_dotnet_build`,
`e2e_poetry_vex_build`, `e2e_vex_{pdm,pip,pipenv,hatch}_build`,
`e2e_deno_vex_build`; the docker / setup-matrix / production legs gain
their manifest-less VEX steps.
Re-pinned to #247 while rebasing:
- vendored mode writes no manifest: the "manifest deleted" steps are
naturally manifest-free, so the npm manifest-less matrix and the uv /
poetry vendored suites add a `legacy-manifest` cell (the record a
pre-5.0 vendored run left beside its ledger, via
`vex_e2e_common::seed_legacy_manifest`, must attest the same way), the
yarn 2/3 refusal suite seeds that legacy record explicitly (and still
requires `not_applied`, never an attestation), and `--detached` twins
assert no manifest in either spelling;
- a fully reverted project keeps no `.socket/`: suites that plant a stale
ledger back after a rollback recreate the directory first, and the
hosted rollback suite checks that `vex` does not recreate it;
- the no-manifest human lines follow #247's wording;
- a corrupt vendor ledger on a manifest-free project is now
`vendor_ledger_corrupt` (exit 2, the ledger named, never rewritten),
replacing #247's disclose-then-`manifest_not_found` expectation;
- repair: two tests pin the standalone-`vendor` embedded record (a
moved-on manifest still wins; with no manifest the embedded copy stands
in offline), replacing the pre-rebase test that assumed a non-detached
`scan --vendor` entry.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ci.yml:
- `test` exports `SOCKET_PATCH_GO_E2E_REQUIRED` / `_VERSION` so the
real-go hosted/vendored suites fail instead of skipping on the Go that
is already installed for vexctl;
- `e2e` gains pinned-toolchain legs, each ending in the manifest-less VEX
matrix: composer 1 / 2.2 / 2 (hosted + vendored), bundler 1.17.3 →
4.0.21 (hosted + vendored + setup_matrix_gem), bun text/binary-lock
eras, the named corepack pnpm legs (3 OS), uv 0.1.45 → 0.12.x hosted +
vendored, poetry / pdm / hatch / pipenv / pip, maven 3.6.3 → 4.0.0-rc-6,
dotnet 6 → 10, deno; npm legs hard-require npm where
`Command::new("npm")` can resolve it;
- new jobs: `yarn-classic-matrix` (1.0.2 → 1.22.22), `yarn-berry-e2e`
(4.0.2 → 4.18.0 + macOS/Windows), `cargo-vex-matrix` (toolchains ×
Cargo.lock v1–v4); `e2e-docker` also runs the pypi vendored-PM suite.
Compatibility workflows: npm (npm 6–12 capstones, new), go (1.18.10 →
1.26.3, new), poetry (new), and the pnpm / pdm / pipenv / bun ones run
the manifest-less VEX steps and trigger on the vex sources. The bun
workflow keeps #247's two modes (hosted, vendored — the vendored-
detached leg collapsed into vendored).
Scripts: the bun / pdm / pipenv / poetry / uv backtests gain the
manifest-less VEX checks (a fresh checkout of the committed state must be
attested, also with the ledgers deleted; `--offline` with no ledger omits
`record_unavailable`; a reverted lock never attests, also under
`--no-verify`; a refused lock format attests nothing — the pdm refusal
check follows #247's manifest-free footprint), plus the
`{uv,yarn-berry,yarn-classic}-vex-matrix.sh` drivers and harness unit
tests.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- CHANGELOG [Unreleased] (both entries are v5 MAJOR): the semver note now lists #247's changes and `vex` refusing stale ledger records and corrupt vendor ledgers; Changed (BREAKING) gains the ledger-liveness / `vendor_ledger_corrupt` entry (and #247's Fixed bullet that had `vex` disclose an unreadable ledger now points at it); Added gains the npm 12 `allow-remote` auto-config, manifest-less VEX, standalone `vendor` record embedding and the new product probes; Fixed gains the composer / gem / cargo v1 / yarn 4.0 / npm dual-lock rewriter fixes after #247's own entries. - CLI_CONTRACT.md: new "Manifest-less VEX (lockfile discovery)" section (inputs, per-ecosystem recognition table, record resolution, verification basis, liveness gates, run warnings), "Patch hosts", the embedded-VEX no-manifest rules, `manifest_not_found` for `vex`, the vendor ledger's `record` semantics (vendored-mode `detached` entries plus the standalone `vendor` fallback copy; the reconcile exemption keys on `detached`), and the new rollback warning code. - README: "No manifest needed for hosted and vendored patches", the rewritten `vex` how-it-works steps and product probes, "npm compatibility (hosted mode and npm 12)", the hosted `.npmrc` commit hint, and the ledger-liveness note under "Undo things". - docs/testing: npm-compatibility.md (npm 6–12, new), uv and bun tables, ecosystems.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…allow-remote auto-config (v5) (#251) * fix(core): hosted/vendored rewriter fixes found by the real-PM matrices First of seven commits that land the manifest-less VEX branch (feat/vex-lockfile-inventory) on top of #247. The branch's ~165-commit history, built on the pre-#247 main 9489b18 with per-PM merge commits, was squashed onto 09956d9 and re-split by concern; the full pre-rebase history is preserved on branch backup/vex-lockfile-pre-rebase. These are the product bugs the per-PM real-toolchain matrices turned up while the manifest-less VEX suites were written. Each is independent of VEX and pinned by a core regression test: - composer (hosted): the rewriter drops the entry's `source` block when it immediately precedes `dist` (one `redirect_composer_dist` fragment edit spanning both, reverted byte-for-byte). Composer 1 and 2.2 LTS silently installed the pristine upstream commit from git whenever the hosted download failed; a hand-ordered source that cannot be dropped warns `redirect_composer_source_kept`. Golden fixture: `source-and-dist`. - gem (hosted): the patch-registry `GEM` section is inserted in bundler's source order (sorted by remote), so a frozen `bundle install` on bundler >= 4.0.19 accepts the converged lock. The `basic` golden and the exact lock expectations move the Socket section first. - cargo (hosted + vendored): a v1 `Cargo.lock` (checksums in `[metadata]`, dependents naming the crate by its full package id) is redirected and vendored correctly: the `[metadata]` line and every dependent's full-id reference follow the source, each fragment its own ledger edit, and `cargo --locked` accepts the result; the vendored detach/restore of a v1 entry is byte-identical. `plan_cargo_lock` keeps #247's multi-source twin disambiguation (`Ambiguous`) and hoisted regexes; the block end now also stops at a trailing `[metadata]` / `[[patch.unused]]`, and the checksum is inserted after a block-final `source` line too. - yarn berry (hosted + vendored): written checksums follow the lock's own spelling — yarn 4.0.0–4.0.2 spell `10c0` checksums as bare hex, so the prefixed form failed `yarn install --immutable` with YN0028. - npm (vendored): npm 12 reifies from the `package-lock.json` it creates beside a committed shrinkwrap, so `vendor` now rewires every present npm lock (siblings first, primary last; an unrewirable sibling warns `vendor_npm_sibling_lock_unwired`), and the in-use/revert probes read every npm lock before deleting an artifact. `select_lockfile` reads through #247's guarded `read_regular_to_bytes`. - npm (hosted): a lockfileVersion 1 redirect warns `redirect_npm_legacy_client` — npm 6 ignores a v1 lock's `resolved` and fails EINTEGRITY against the patched pin. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(hosted): auto-configure npm 12 allow-remote in the project .npmrc npm >= 12 defaults to `allow-remote=none` and refuses (EALLOWREMOTE) every lockfile entry whose tarball is not served by the configured registry — exactly what a hosted redirect writes. When `scan --mode hosted` / `get --mode hosted` leaves a root package-lock.json / npm-shrinkwrap.json carrying a granted hosted artifact URL, the run now ensures `allow-remote=all` in the project `.npmrc` (creating the file, or appending one line with the BOM, CRLF and every other byte preserved) and records it in the redirect ledger as `redirect_npmrc_allow_remote` (`created` / `added`). - core `patch::redirect::npmrc`: npm's `.npmrc` grammar as measured against npm 12.1.0 (exact `allow-remote` key, last top-level assignment wins, `[section]` bodies are not top-level, bare-CR line splits, case- sensitive value), the plan (create / append / already-all / respected user / env / outer-layer value / unsupported), and the unwinds. - Every reversal removes exactly what was added once no package-lock entry needs it: the whole-ledger replay (a new `NpmrcAllowRemote` inverse, grouped with the npm lock kinds), the per-purl npm revert behind scoped rollback / remove / the vendored takeover ("last one out", same transaction, flushed after the lock through #247's shared `staged::flush_staged`), and the vendored-supersedes-hosted reconcile. A modified created file keeps the user's lines (`redirect_npmrc_allow_remote_modified`, surfaced by rollback, remove, vendor and the reconcile). A symlinked `.npmrc` refuses an unwind at plan time, before anything is written. - Hosted run: the `.npmrc` edit rides `rewrite.files` / `rewrite.edits`, so it is written under #247's apply-lock window, after its whole-run SYMLINK GUARD, and only after the ledger persisted — never on `--dry-run` (which previews the write, also for a vendored → hosted takeover; the root locks are now read for such a preview so the pnpm `trustLockfile` preview sees the lock the wet run splices). An explicit user value (project `.npmrc`, user/global/builtin config, or an `npm_config_allow_remote` env var) is respected and named; a symlinked, unreadable or bare-CR `.npmrc` is left alone; `--no-npm-allow-remote- config` / `SOCKET_NO_NPM_ALLOW_REMOTE_CONFIG` opts out. Every variant warns `redirect_npm_allow_remote` with the whole-tree tradeoff. - `atomic_write_bytes_preserving_mode` creates its stage with the destination's permission bits (kept inside #247's `commit_stage` structure), so a 0600 token-bearing `.npmrc` is never staged world- readable. - remove: the hosted leg's advisories are printed inside #247's `unwind_hosted` (so a run that then fails still reports them) and carried into the success envelope's `warnings[]`. Tests: npmrc/replay/takeover/scan unit tests, the flag's parse coverage, `redirect_npm_allow_remote` (plus #247's invariants: the lock never outlives the run, dry runs create no `.socket/`, a full rollback leaves no `.socket/` and never the user's `.npmrc`), and the dry-run takeover previews in `coverage_fix_scan_hosted_dryrun_vendored`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor(core/vendor): split lock_inventory into per-format submodules A verbatim move with no behavior change, so the next commit can give each lock format its shared entry model beside its registry view: `vendor/lock_inventory.rs` becomes a directory module with one file per format (`npm`, `npm_family`, `pnpm`, `yarn`, `bun`, `cargo`, `golang`, `composer`, `gem`, `pypi`), ledger recovery (`recover`) and the rewired-lock trust anchor (`wired`). `mod.rs` keeps the public API (`LockIntegrity`, `LockfileEntry`, `UnsupportedNpmLayout`, `lookup`, `inventory_project(_diagnosed)`, `recover_lock_entry`, `wired_vendor_integrity`); the three test modules move to `tests.rs`, `recover_tests.rs` and `python_lock_union_tests.rs`. Only imports, visibility (helpers another file calls become `pub(super)`), sibling-module paths (`super::state` -> `crate::vendor::state`), module docs and the single-file section banners changed; the test modules lost one indentation level and were re-wrapped by rustfmt. `git show --color-moved` shows everything else as moved. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(core/vex): discover hosted and vendored patch references from lockfiles New read-only module `vex::discover`: `discover_patched_refs(_with)` reads every supported ROOT lockfile / package-manager config and returns the Socket patch wiring it finds — `Discovery { refs, diagnostics, recognized, unlocked_pins, elsewhere }`, one `PatchedRef { purl, uuid, mode (Hosted | Vendored), source_file, artifact_rel, locked_integrity, integrity_required, url }` per live reference. It never touches the network, never writes, and never fails the run: a malformed file is a diagnostic (`lockfile_unreadable`, `lockfile_unparseable`, `patched_ref_invalid`, `patched_ref_unattributable`). One extractor per package-manager family: npm (package-lock / shrinkwrap, pnpm every lock generation + Rush locks), yarn classic and berry, bun (`bun.lock` / `bun.lockb`), cargo, golang (go.mod / go.work + sums), pypi locks (uv, PEP 723 script locks, pylock, poetry, pdm), pypi other (Pipfile.lock, requirements + `-r`, Hatch / PEP 621 direct refs), gem, composer, maven, nuget; deno is explicitly empty. Every file present is read — no precedence chain — because the hosted rewriter edits every candidate it finds. Every value is committed, tamperable data and is validated fail-closed: - hosted: `hosted_patch_uuid` accepts only `https://patch.socket.dev` or a configured `--patch-server-url` origin, no userinfo, and takes the LAST canonical-uuid path segment (grant tokens may be uuid-shaped); percent-encoding, `\/` escapes and fragments are handled; - vendored: root-anchored `.socket/vendor/<eco>/<uuid>/…` paths whose leaf names the entry's own artifact (`vendor_ref` takes the path literally; only yarn / URL-form pip strip their `#…` / `::…` decorations); - pins, not definitions: a registry / index / source definition alone (cargo `[registries]`, nuget `<add>`, pom `<repository>`, uv index tables, `.npmrc`) never makes a reference; - contested locks: a lock that resolves the same name@version from a non-Socket source drops the ref (`patched_ref_unattributable`); - lockless cargo pins / exclusive nuget mappings are recorded as `UnlockedPin`s that can only keep a ledger record live, never create a ref; `recognized` lists every uuid a read file mentions, so a rejected mention keeps nothing alive downstream. Supporting core changes: `patch::redirect::{SOCKET_PATCH_SERVER_HOST, hosted_patch_uuid, hosted_patch_url_uuids}` (the pipenv owner check uses the shared host constant); the pnpm lock grammar and `hosted_url_version` exported crate-wide so readers parse exactly what the writers write; `utils::digest` (the SRI pin rule and the hex digest shapes — one copy for the inventory, discovery, ledger recovery and the rewriters, each call site keeping its case policy); and `utils::purl`'s validating purl builders, which discovery and the inventory's registry views share. A few writer helpers become `pub(crate)` so the extractors' tests derive their fixtures from the writers themselves. One lockfile traversal layer: discovery and `vendor::lock_inventory` (the scan / get / vendor / repair inventory) read each format through ONE reader that yields every entry, Socket-owned ones included — the inventory's registry views drop those, the extractors classify them: - `lock_inventory` becomes a directory module, one file per format, each laid out as a pure entry model, a stat-only file-selection section and the registry view (an architecture test enforces the layering). Entry models: `npm_lock_nodes`; `pnpm_packages` over the hosted rewriter's pnpm grammar (every key generation, CRLF included); the yarn `classic_entries` / `berry_entries` models with one berry locator, cache-key and checksum rule; `BunLockb::parse_packages`; and `composer_lock_packages`, whose array index the composer writer's lock walks use too. - Every other format reads through the reader its writer owns: `cargo_lock::locked_packages` and `cargo_config`'s `[patch]` / `[registries]` walks; `go_mod_edit` / `go_sum_edit` read helpers; a new `vendor::gemfile_lock` model; the `utils::python_lock` / `poetry_lock` / `hatch` readers (uv source fields, script-lock pairing, the pyproject / Hatch declaration walk); a new `utils::requirements` lexer lifted out of the vendored requirements planner; and two new XML readers, `vendor::maven_pom` and `vendor::nuget_config`, with `nuget_feed::nuget_lock_entries` shared by discovery and the feed writer. - `DiscoverCtx::locate` classifies a lock location once (vendored path, hosted uuid, decorated leaf) for every extractor. The npm-family extractors iterate the entry models only, never the grammar primitives, and every extractor reads content only through the recognizing ctx (rule 11) — both enforced by architecture tests. Ledger liveness is one rule too: `Discovery::{wires_package, vendor_entry_live, redirect_record_live}`, held per call site by `LedgerLiveness` (the sorted redirect-ledger files, the lock inventory loaded lazily at most once), which the CLI's vex and scan both use. Cargo crates.io provenance is explicit (`LockfileEntry::source_kind`), not inferred from the checksum variant. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(vex): manifest-less VEX — attest hosted and vendored patches from lockfiles `socket-patch vex` and the embedded `apply` / `scan` / `vendor --vex` now attest hosted and vendored patches with no `.socket/manifest.json` — and with no `.socket/vendor` ledgers either — by reading the wiring out of the project's lockfiles (`vex::discover`). This covers a depscan-opened PR, a clone that never committed its ledgers, and a lock-only CI checkout. Record view (`commands::vex_sources::plan`): four sources — the manifest, the redirect ledger's records, the vendor ledger's embedded records and lockfile discovery — merge into one view. A candidate's record must carry the uuid the lockfile actually WIRES; it comes from the first source that has one, else (online only) from the patch API by uuid, 10 fetches at a time with `get`'s one-shot 401/403 → public-proxy fallback (#247's uuid-only `ApiClient::fetch_patch`). Nothing is written: vex never takes the apply lock, never creates `.socket/`, never writes the manifest, and the `socket-patch.vendor.json` marker is never a record. Gates, applied before hashing and kept under `--no-verify` (which now skips only the hashing): - a vendor-ledger entry attests only while a lockfile still wires its artifact (`vendor_unwired`); a redirect-ledger record only while a lockfile wires its hosted patch (`redirect_unwired`); discovery is authoritative for every uuid a read file mentions, so a rejected mention keeps nothing alive; - `record_unavailable` (offline, 404, refused, transport error — the run continues), `record_mismatch`, `wiring_conflict` (lockfiles wire one package to several patches); - a malformed / unreadable `.socket/vendor/state.json` is the hard error `vendor_ledger_corrupt` (exit 2), mirroring `redirect_ledger_corrupt` — this supersedes #247's degrade-and-disclose posture for `vex` only; `setup --check` keeps #247's `vendor_context_from` / `warn_unreadable_vendor_state` path. Evidence: vendored refs hash the committed artifact (the ledger entry when it names the wired artifact, else one synthesized from the ref); hosted refs hash the installed copies the build CONSUMES (`vex_consumed::hosted_consumed_copies` → core `VendorContext::hosted` / `HostedCopies`: the Go replacement module, the Socket-registry cargo src dir, maven's suffixed version — never a pristine sibling), and with nothing installed a discovered pinned reference attests from its lockfile pin, like in-run `scan --mode hosted --vex`. Discovered refs bypass the Property 7 ecosystem filter. Commands: - `apply --vex` / `vendor --vex` with no manifest attest what the lockfiles and ledgers wire (nothing anywhere keeps the calm exit 0 and removes a stale document; `apply --check` and `--dry-run` never generate); #247's no-manifest lines ("No patch manifest found; nothing to apply.", "No manifest found, nothing to vendor.") are kept; - failed VEX runs carry the discovery diagnostics into `warnings[]` (standalone envelope, embedded envelopes, scan JSON — hosted included); - `manifest_not_found` now means no manifest AND nothing wired; - human output: `Note:` lines for superseded records / fetch failures, phrased omission reasons. Writer hardening: the manifest-driven standalone `vendor` now embeds the patch `record` in its ledger entries too (vendored mode already does, as `detached` entries). `detached` stays the "no manifest owner" flag, so the manifest reconcile, legacy-manifest migration and get/scan idempotency from #247 are unaffected. Every reader of embedded records shares one ownership rule (`commands::vendor_record_is_unowned`): a detached entry's record always stands alone, a standalone `vendor` entry's fallback copy only when no manifest entry covers it (by ledger key or base purl). `vex`, `list` and `setup --check` (`fold_vendor_records`, formerly `fold_detached_records`) all apply it, so one tree never lists "no patches" while its VEX document attests one. `repair` stays narrower: it keeps preferring a manifest that moved on to a newer uuid and falls back to the embedded copy only with no manifest at all. One liveness rule for vex and scan: `scan`'s cross-mode takeover classification (`classify_overlap_takeover`), its `hosted_wiring_retained` warning and `redirectState.wiringLive` ask the same core discovery (`commands::discover_wiring`) and liveness rule (`Discovery::redirect_record_live` / `vendor_entry_live` / `wires_package`, through one `LedgerLiveness` holder per call site) that gate attestation, replacing scan's private cargo / hosted / vendored checks and its looser text scan. The CLI also shares one purl splitter (`utils::purl::purl_parts`), one vendor-ledger lookup (`vendor::state::lookup_entry_kv`) and one npm alias-aware identity crawl (`ecosystem_dispatch::npm_paths_by_identity`) across vex, scan and vendor, and pairs PEP 723 script locks through `utils::python_lock::script_of_lock` like the rewriters do. Output follows the conventions from #248: `ui::plural` counts, a `ui::StatusLine` progress line for record fetches, and the shared `format_vex_written` / `format_vex_dry_run_skip` lines on the manifest-less `apply --vex` / `vendor --vex` paths. Manifest-less `vex` honours `--dry-run` and `-O -` like the manifest path. Product auto-detection adds go.mod, composer.json, pom.xml, a single `*.csproj` and a single `*.gemspec`, after the existing probes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(cli): manifest-less VEX suites, real-PM capstones and shared helpers Hermetic suites (wiremock patch API, run in the default `test` job on every OS): one `e2e_vex_lockfile_<pm>` per package manager — npm, pnpm, yarn (classic + berry), bun, cargo, golang, uv, poetry, pdm, hatch, pipenv, pip, gem, composer, maven, nuget, deno (negative) — covering spoofed hosts, record mismatch, pristine / tampered installs, contested and orphaned wiring, `--offline`, reverted locks (also `--no-verify`) and the embedded commands; plus the cross-cutting `e2e_vex_manifestless_ embedded`, `e2e_vex_redirect`, `e2e_vex_vendor` and the `vex_e2e_common_selftest` harness checks. Real-PM capstones (gated like the existing suites, `_REQUIRED` env pattern): every hosted / vendored build suite now ends in the manifest- less VEX matrix over a fresh checkout — manifest absent (ledgers kept), ledgers deleted (lockfile + API), offline (`record_unavailable`, zero requests), reverted lock (never attested, verify or not) — through shared helpers: `vex_e2e_common` (+ bun / uv matrices), `npm_e2e_common` (+ `manifestless`), `yarn_berry_common`, `common/yarn_classic_vex`, `common/bundler_e2e`, `cargo_e2e_matrix`, `golang_e2e_matrix`, `maven_build_common`, `composer_e2e_common`, `vex_pdm_hatch_common`, `vex_pipenv_pip_*`, `vex_pypi_real_common`. New real-toolchain suites: `e2e_redirect_{composer,maven,uv}_build`, `e2e_vendor_maven_build`, `e2e_golang_workspace_build`, `e2e_nuget_dotnet_build`, `e2e_poetry_vex_build`, `e2e_vex_{pdm,pip,pipenv,hatch}_build`, `e2e_deno_vex_build`; the docker / setup-matrix / production legs gain their manifest-less VEX steps. Re-pinned to #247 while rebasing: - vendored mode writes no manifest: the "manifest deleted" steps are naturally manifest-free, so the npm manifest-less matrix and the uv / poetry vendored suites add a `legacy-manifest` cell (the record a pre-5.0 vendored run left beside its ledger, via `vex_e2e_common::seed_legacy_manifest`, must attest the same way), the yarn 2/3 refusal suite seeds that legacy record explicitly (and still requires `not_applied`, never an attestation), and `--detached` twins assert no manifest in either spelling; - a fully reverted project keeps no `.socket/`: suites that plant a stale ledger back after a rollback recreate the directory first, and the hosted rollback suite checks that `vex` does not recreate it; - the no-manifest human lines follow #247's wording; - a corrupt vendor ledger on a manifest-free project is now `vendor_ledger_corrupt` (exit 2, the ledger named, never rewritten), replacing #247's disclose-then-`manifest_not_found` expectation; - repair: two tests pin the standalone-`vendor` embedded record (a moved-on manifest still wins; with no manifest the embedded copy stands in offline), replacing the pre-rebase test that assumed a non-detached `scan --vendor` entry. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * ci: run the manifest-less VEX real-PM matrices and backtest steps ci.yml: - `test` exports `SOCKET_PATCH_GO_E2E_REQUIRED` / `_VERSION` so the real-go hosted/vendored suites fail instead of skipping on the Go that is already installed for vexctl; - `e2e` gains pinned-toolchain legs, each ending in the manifest-less VEX matrix: composer 1 / 2.2 / 2 (hosted + vendored), bundler 1.17.3 → 4.0.21 (hosted + vendored + setup_matrix_gem), bun text/binary-lock eras, the named corepack pnpm legs (3 OS), uv 0.1.45 → 0.12.x hosted + vendored, poetry / pdm / hatch / pipenv / pip, maven 3.6.3 → 4.0.0-rc-6, dotnet 6 → 10, deno; npm legs hard-require npm where `Command::new("npm")` can resolve it; - new jobs: `yarn-classic-matrix` (1.0.2 → 1.22.22), `yarn-berry-e2e` (4.0.2 → 4.18.0 + macOS/Windows), `cargo-vex-matrix` (toolchains × Cargo.lock v1–v4); `e2e-docker` also runs the pypi vendored-PM suite. Compatibility workflows: npm (npm 6–12 capstones, new), go (1.18.10 → 1.26.3, new), poetry (new), and the pnpm / pdm / pipenv / bun ones run the manifest-less VEX steps and trigger on the vex sources. The bun workflow keeps #247's two modes (hosted, vendored — the vendored- detached leg collapsed into vendored). Scripts: the bun / pdm / pipenv / poetry / uv backtests gain the manifest-less VEX checks (a fresh checkout of the committed state must be attested, also with the ledgers deleted; `--offline` with no ledger omits `record_unavailable`; a reverted lock never attests, also under `--no-verify`; a refused lock format attests nothing — the pdm refusal check follows #247's manifest-free footprint), plus the `{uv,yarn-berry,yarn-classic}-vex-matrix.sh` drivers and harness unit tests. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs: manifest-less VEX, npm 12 allow-remote and the v5 changelog - CHANGELOG [Unreleased] (both entries are v5 MAJOR): the semver note now lists #247's changes and `vex` refusing stale ledger records and corrupt vendor ledgers; Changed (BREAKING) gains the ledger-liveness / `vendor_ledger_corrupt` entry (and #247's Fixed bullet that had `vex` disclose an unreadable ledger now points at it); Added gains the npm 12 `allow-remote` auto-config, manifest-less VEX, standalone `vendor` record embedding and the new product probes; Fixed gains the composer / gem / cargo v1 / yarn 4.0 / npm dual-lock rewriter fixes after #247's own entries. - CLI_CONTRACT.md: new "Manifest-less VEX (lockfile discovery)" section (inputs, per-ecosystem recognition table, record resolution, verification basis, liveness gates, run warnings), "Patch hosts", the embedded-VEX no-manifest rules, `manifest_not_found` for `vex`, the vendor ledger's `record` semantics (vendored-mode `detached` entries plus the standalone `vendor` fallback copy; the reconcile exemption keys on `detached`), and the new rollback warning code. - README: "No manifest needed for hosted and vendored patches", the rewritten `vex` how-it-works steps and product probes, "npm compatibility (hosted mode and npm 12)", the hosted `.npmrc` commit hint, and the ledger-liveness note under "Undo things". - docs/testing: npm-compatibility.md (npm 6–12, new), uv and bun tables, ecosystems.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(vex): golden snapshot of discovery over committed fixtures Pin the exact output of lockfile discovery over every committed fixture project, so a behavior change to discovery (or to the lock readers it shares with the rewriters and vendor::lock_inventory) shows up as a golden diff to review. - Corpus: 182 projects under crates/socket-patch-core/tests/fixtures/ — every redirect case's input/ and expected/ tree, pnpm-hosted, poetry, pipenv, bun-lockb captures, and each pdm-native lock staged as pdm.lock — run through the full orchestrator and checked against rule 11's recognition-covers-refs invariant. - Rendering (src/vex/discover/testing/golden.rs): every PatchedRef field (plus lockfile_basis_ok), diagnostics (tempdir root and OS error numbers normalized), recognized identities, unlocked pins, resolved-elsewhere entries, and the live hosted / vendored ledger claims. The destructuring is exhaustive, so a new field fails to compile until the golden covers it. - Goldens: one JSON per fixture family (redirect-<eco>, bun-lockb, pdm-native, pipenv, pnpm-hosted, poetry) under tests/fixtures/vex-discover-golden/, mapping fixture path to output. Missing and orphaned entries fail. Regenerate after an intended change with SOCKET_PATCH_UPDATE_GOLDEN=1 cargo test -p socket-patch-core --lib vex::discover::testing::golden. Unix only: Windows checkouts convert some fixtures' line endings. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ee ledger The poetry vendored capstone (added in #251, written before its rebase onto #247) still asserted `scan --vendor` writes `.socket/manifest.json`. Since #247 vendored mode is manifest-free (CLI_CONTRACT `scan --vendor`): the run writes only `.socket/vendor/**`, and each ledger entry is `detached: true` with the patch record embedded. The assertion therefore failed on every Poetry release in the CI matrix the first time the leg actually ran (#253). Assert the contract instead: no manifest is written, and the ledger entry for the vendored uuid is detached and embeds its record. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ee ledger The poetry vendored capstone (added in #251, written before its rebase onto #247) still asserted `scan --vendor` writes `.socket/manifest.json`. Since #247 vendored mode is manifest-free (CLI_CONTRACT `scan --vendor`): the run writes only `.socket/vendor/**`, and each ledger entry is `detached: true` with the patch record embedded. The assertion therefore failed on every Poetry release in the CI matrix the first time the leg actually ran (#253). Assert the contract instead: no manifest is written, and the ledger entry for the vendored uuid is detached and embeds its record. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…es after #251 (#253) * test(hosted): match the allow-remote env var name case-insensitively On Windows env var names are case-insensitive: run_isolated blanks NPM_CONFIG_ALLOW_REMOTE before the test sets npm_config_allow_remote=none, so the child sees a single variable under the first spelling and the warning (which names the variable as the OS reports it) says NPM_CONFIG_ALLOW_REMOTE=none. The product is right; the assertion was POSIX-only. Fixes outer_npm_config_layers_are_respected on test (windows-latest) after #251. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * ci: run the cargo matrix toolchain install under bash on every OS The cargo-vex-matrix windows-latest leg used the default pwsh shell, where "$CARGO_TEST_TOOLCHAIN" is an unset PowerShell variable, so it ran `rustup toolchain install ""` and failed. The leg was skipped on #251's own CI, so this is its first real run. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(e2e_vex_build): poetry vendored capstone asserts the manifest-free ledger The poetry vendored capstone (added in #251, written before its rebase onto #247) still asserted `scan --vendor` writes `.socket/manifest.json`. Since #247 vendored mode is manifest-free (CLI_CONTRACT `scan --vendor`): the run writes only `.socket/vendor/**`, and each ledger entry is `detached: true` with the patch record embedded. The assertion therefore failed on every Poetry release in the CI matrix the first time the leg actually ran (#253). Assert the contract instead: no manifest is written, and the ledger entry for the vendored uuid is detached and embeds its record. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(yarn-berry): pin enableImmutableInstalls off so fixture installs work under CI yarn 3 and later turn enableImmutableInstalls on by default when ci-info detects CI (CI / GITHUB_ACTIONS). With that default, the plain `yarn install` that writes each fixture's first yarn.lock fails with YN0028 ("The lockfile would have been created by this install, which is explicitly forbidden"): exit 1, nothing on stderr. Every hosted, vendored, pnpm-linker, workspaces and yarn-3 refusal fixture on the new yarn-berry legs (4.0.2, 4.1.0, 4.6.0, 4.12.0 ubuntu+macOS, 4.18.0) failed that way. The yarn 2 refusal legs passed because yarn 2 keeps the default off. The suites were never run under CI before #251 added the legs; the main test job soft-skips them. yarn_berry_common::pin_berry_ci_defaults now sets CI=true (local runs get the runner's defaults) and YARN_ENABLE_IMMUTABLE_INSTALLS=false. It is applied in every berry suite's corepack helper, after the YARN_* scrub and cache_env::isolate. The fresh-checkout installs still pass --immutable explicitly, and yarn's flag outranks the setting, so lock enforcement is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(yarn-berry): spawn corepack.cmd on Windows Node ships corepack on Windows as the corepack.cmd batch shim, and Command::new("corepack") only resolves corepack.exe. On the windows-latest yarn-berry leg every suite's availability probe therefore reported "`corepack yarn@4.12.0` unavailable" (yarn@2.4.3 / 3.8.7 in the refusal suite), and SOCKET_PATCH_YARN_E2E_REQUIRED=1 turned each of those into a failure. yarn_berry_common::corepack_command() picks the spawnable name. Every berry suite's has_corepack_pm probe and corepack helper uses it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(yarn-berry): show yarn's stdout when a fixture install fails yarn berry prints its errors (YN0028, YN0018, …) on stdout and leaves stderr empty. The fixture and bootstrap skip messages printed only stderr, so all 11 failures on each yarn-berry CI leg read "fixture `yarn install` failed (registry unreachable?):" followed by nothing. The real cause was YN0028 under CI's implicit immutable default. yarn_berry_common::yarn_output formats both streams, and every berry fixture/bootstrap skip now uses it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * ci: select the pinned bundler with BUNDLER_VERSION on every bundler leg The Ruby 3.1 legs pinned to bundler 2.1.4 and 2.2.33 never ran those bundlers. setup-ruby's `bundler:` input only `gem install`s the release; with no lockfile to read, RubyGems' `bundle` binstub then activates the HIGHEST installed bundler, which on Ruby 3.1.7 is its default gem 2.3.27. tests/common/bundler_e2e.rs correctly panicked on every test ("bundle on PATH is 2.3.27"). The 1.17.3 legs passed only because the Bundler 1.x step already exported BUNDLER_VERSION. Export BUNDLER_VERSION for every pinned-bundler leg (a new step after the 1.x install), which makes the binstub pick exactly the pinned release in every process and also turns off bundler >= 2.3's lockfile-driven self-switch. Verified locally in Docker (Ruby 3.1.7 / 3.3.10 / 3.4.9, setup-ruby layout): all 16 e2e_{redirect,vendor}_gem_build legs and the setup_matrix_gem leg green with the export; the 2.1.4/2.2.33 legs fail exactly as in CI without it. The harness panic now names the fix. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(vex/deno): stop expecting a manifest from a refused vendored scan The real-deno negative capstone asserted that `scan --mode vendored` "left a manifest" for the patch it could not wire, then checked that the unapplied manifest attests nothing. Vendored mode is manifest-free: its download phase is detached (download_patch_records_with writes nothing; the vendor ledger alone carries records), and a vendor step refused with vendor_lockfile_missing records nothing. So the assertion failed on the first CI run of both deno legs (1.46.3, 2.9.7) at deno.rs:454. Assert the real contract instead: the download is detached, no ledger entry names the package, and there is nothing to attest (no manifest, exit 2 manifest_not_found, zero patch-API requests). The "unapplied manifest patch attests nothing" check moves to step 4, where the test stages the manifest itself, before `apply --vex` runs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(e2e_vex_build): compare hatch env paths canonically on macOS The macOS hatch 1.18.1 leg failed all four cases with `".../private/var/.../six.py" outside "/var/.../app"`. The hatch bootstrap venv is built on the runner's actions/setup-python CPython, a macOS framework build, and a framework interpreter realpaths `sys.prefix` — so `six.__file__` names `/private/var/folders/...` while `hatch env find` echoes the `/var/folders/...` spelling of the same temp dir. Linux legs (and a uv-managed standalone Python locally) keep one spelling, which is why only the macOS leg tripped. Canonicalize both sides for the containment check only; the env dir handed on as VIRTUAL_ENV is unchanged. Reproduced locally by bootstrapping hatch 1.18.1 on Homebrew's framework CPython (4/4 fail before, 4/4 pass after). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(maven e2e): scrub Maven 4's CI markers so a runner logs like a laptop The ubuntu Maven 4.0.0-rc-6 leg of e2e_vendor_maven_build failed at the TAMPER probe: Maven rejected the tampered file:// jar and built Central's pristine one (the load-bearing assertion held), but the output carried no "checksum" line, so "the file:// copy was rejected on its checksum" tripped. The CI dump shows no transfer lines at all, not even the Central download that must have happened after the purge. Root cause: Maven 4's CIDetectors (generic CI, GITHUB_ACTIONS, CIRCLECI, Jenkins WORKSPACE, TEAMCITY_VERSION, TRAVIS) make MavenInvoker pick the QuietMavenTransferListener whenever a CI is detected and --force-interactive is absent, even under -B. That listener drops both "Downloading from ..." and the "Checksum validation failed" warning. Maven 3 has no such detection, so only the 4.x legs log differently on a GitHub runner (and only this probe greps a warning a successful build prints; the redirect suite's checksum greps are on failed builds, whose exception text survives the quiet listener). Scrub those markers (plus the Maven config vars run() already dropped) in one mvn_command() used by both detect() and run(). --force-interactive was rejected: it flips the run interactive (progress-bar listener) and Maven 3 refuses the flag. The checksum assertion is unchanged. Repro: CI=true GITHUB_ACTIONS=true on the unfixed tree reproduces the CI panic at e2e_vendor_maven_build.rs:353 locally; with the fix both e2e_vendor_maven_build and e2e_redirect_maven_build pass under the same env on Maven 4.0.0-rc-6 and 3.9.16. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(e2e_nuget_dotnet_build): serialize dotnet spawns around a .NET 9 PAL race The ubuntu SDK 9 leg failed nuget_vendored_dotnet_restore_then_manifestless_vex at its first fixture restore (the hosted test passed): System.IO.IOException: The system cannot open the device or file specified. : 'NuGet-Migrations'. One or more system calls failed: mkdir("/tmp/.dotnet/shm/session2027", AllUsers_ReadWriteExecute) == -1; errno == EEXIST; at System.Threading.Mutex..ctor ... at NuGet.Common.Migrations.MigrationRunner.Run ... at Microsoft.DotNet.Configurer.DotnetFirstTimeUseConfigurer.Configure() Both tests run in parallel, each with a fresh HOME, so each first `dotnet restore` runs the first-use NuGet migrations under the named mutex `NuGet-Migrations`. On a fresh runner `/tmp/.dotnet` does not exist yet, and the .NET 9 runtime's named-mutex setup races when two processes create the shared-memory tree at once: the loser's session directory mkdir fails with EEXIST. Environment/tool race (SDK 9 PAL), exposed by the harness running two SDK processes concurrently. Reproduced in mcr.microsoft.com/dotnet/sdk:9.0 (9.0.318, the leg's SDK) with two concurrent first-run CLI commands per round, fresh HOMEs, `/tmp/.dotnet` wiped before each of 60 rounds: 9, 0, 2 and 7 of 120 processes died with the exact CI message across four batches; 0/120 with the root pre-created and 0/40 rounds run one at a time. Hold one binary-wide lock around every `dotnet` spawn (the --version probe and every restore). Only the SDK phases serialize; the socket-patch runs between them stay parallel. No assertion changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(yarn-berry): pin hardened mode off so PR runs install from the lock yarn 4 enables hardened mode when it detects a GitHub Actions run for a public pull request and then re-resolves every lock entry against the registry. The fresh-checkout installs point the registry at an unreachable address on purpose, so the hosted berry suites failed with ECONNREFUSED 127.0.0.1:1 on PR runs only (seen in the coverage job once the fixture installs stopped failing). Reproduced locally with a simulated public-PR event: 3 failures without the pin, 11/11 with it. --immutable --check-cache still verifies every checksum. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(yarn-berry): pin hardened mode off only where yarn has the setting The previous commit pinned YARN_ENABLE_HARDENED_MODE=false for every berry suite, including e2e_yarn_legacy_cachekey_refusal_build, which drives yarn 2.4.3 and 3.8.7. Those releases predate hardened mode and refuse every command while the variable is set: Usage Error: Unrecognized or legacy configuration settings found: enableHardenedMode so all four refusal cells failed at their fixture install under SOCKET_PATCH_YARN_E2E_REQUIRED=1 — how the yarn-berry-e2e job runs the suite — and soft-skipped everywhere else. pin_berry_ci_defaults now takes the yarn spec and pins hardened mode off for yarn 4+ only, removing the variable for yarn 2/3. Reproduced locally: 4/4 cells fail with the Usage Error before, 4/4 pass after. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(yarn-berry): redirect and vendor CRLF (Windows) berry files byte-exactly yarn berry writes a file it creates with the OS line ending and keeps an existing file's majority ending on every later write (normalizeLineEndings in yarnpkg-fslib FakeFS.ts, called by Project.persistLockfile and, through changeFilePromise's automaticNewlines, Workspace.persistManifest; the same rule from 2.4.3 through 4.18.0). On Windows a fresh yarn.lock is therefore CRLF, and so is the package.json yarn first pretty-prints; a core.autocrlf checkout produces the same on any OS. Once the Windows yarn-berry suites ran (corepack.cmd), both modes failed on those files: - hosted: rewrite_yarn_berry refused every CRLF lock (redirect_yarn_berry_crlf_unsupported, redirected 0); - vendored: package.json was re-serialized LF on both the wiring and the revert, so `vendor --revert` never restored the CRLF manifest byte-for-byte. Hosted: the rewriter works on the LF-normalized lock and re-expands its output (utils::line_endings), keeps a leading BOM, and records the lock's on-disk CRLF fragments in the ledger, so the per-purl takeover and the whole-ledger replay restore them byte-exactly. Both replays now also match yarn blocks respelled in the live lock's ending when a checkout flipped its uniform ending since the redirect (the committed ledger keeps its fragments verbatim). Vendored: package.json is re-serialized in its own layout (vendor::common::JsonLayout: BOM, indent, line ending, trailing-newline shape) and parsed past a BOM, and lock entries are spliced in the terminator of the block they replace (yarn_classic_lock::block_eol, also used by the shared revert, so a lock whose endings were mixed after vendoring keeps every other line as it was). A yarn.lock or package.json that mixes CRLF and LF, or holds a bare CR, has no single ending to keep and fails yarn's own `--immutable` check (YN0028): both modes refuse it before any write (redirect_yarn_berry_mixed_line_endings / vendor_yarn_berry_mixed_line_endings) with `yarn install` as the remedy. Reverts never refuse on line endings. Readers: is_berry_lock, the vendor flavor sniff, repair's sniff, scan_blocks and the .yarnrc.yml compressionLevel read skip a leading BOM (a header-less `__metadata:` lock is berry; a BOM'd yarnrc's first-line knob is no longer read as unset). The lock inventory and manifest-less VEX already split CRLF lines; unit tests pin both. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(yarn-berry): run every berry shape on CRLF, BOM and mixed files Hermetic CLI coverage of the Windows file shapes: `scan --mode hosted` over CRLF and BOM + CRLF locks (every line kept CRLF, the ledger's fragments the on-disk CRLF bytes, re-run a no-op, `rollback` restoring the pristine lock byte-for-byte) and its mixed-ending refusal; `vendor` + `vendor --revert` over CRLF (+ BOM) package.json / yarn.lock pairs (byte-exact round trip) and the mixed-ending failed event; both mode takeovers on CRLF files (hosted -> vendored -> revert, vendored -> hosted -> rollback, BOM kept); and a manifest-less VEX cell over a CRLF + BOM vendored lock and manifest. Real yarn: SOCKET_PATCH_YARN_BERRY_EOL=crlf respells the files each fixture's first `yarn install` wrote CRLF — what yarn itself writes on Windows (a new lockfile and a freshly pretty-printed manifest get os.EOL) — and yarn keeps them CRLF on every later write, so the hosted, vendored, pnpm-linker, workspaces, legacy-refusal and mode-migration suites run on CRLF files on macOS / Linux as they do on windows-latest. Every fixture prints `BERRY-EOL|<yarn>|<flow>|<file>|yarn=…|flow=…`, the ending yarn wrote and the one the flow ran on. Against the pre-fix code this mode reproduces both Windows CI failures (redirect_yarn_berry_crlf_unsupported on the hosted suites; "revert must restore package.json byte-identical" on the vendored ones); with the fix, yarn 4.12.0 passes all five suites in both modes (103 VEX-MATRIX cells each) and both mode takeovers. Also drops a doubled doc-comment line in yarn_berry_common. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs: yarn berry line endings in the contract, changelog and a compatibility page CLI_CONTRACT: the hosted redirect keeps a CRLF lock's own line ending and BOM and records on-disk fragments; a mixed-ending lock is refused with redirect_yarn_berry_mixed_line_endings, which replaces v4's redirect_yarn_berry_crlf_unsupported (no longer emitted); the vendored yarn berry row keeps both files' layout, with the new vendor_yarn_berry_mixed_line_endings refusal in the code table; the per-purl revert and whole-ledger replay respell yarn blocks across a uniform LF <-> CRLF checkout flip. CHANGELOG [Unreleased]: the fix (hosted + vendored CRLF support, BOM tolerance) under Fixed, the two refusal codes and the CRLF test mode under Added. docs/testing/yarn-berry-compatibility.md (new): supported releases, the CI matrix, how yarn berry chooses line endings — cited to FakeFS.ts, Project.ts, Workspace.ts, Manifest.ts and syml.ts at @yarnpkg/cli/4.12.0 — the git autocrlf paths to CRLF, socket-patch's contract per mode, and how to run the suites locally in CRLF mode. docs/ecosystems.md links it from the yarn berry notes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(setup): keep a CRLF / BOM package.json's layout through setup and --remove `setup` re-serialized package.json through `serialize_json`, which always emits bare LF and never writes a BOM back. On a Windows yarn berry project (persistManifest pretty-prints the manifest with os.EOL) a two-key script edit became a whole-file CRLF -> LF diff that yarn then keeps (it follows the majority ending), and `setup --remove` could never land byte-identical on the pre-setup file. Render through the vendored backends' `JsonLayout` instead (BOM, indent, line ending, trailing-newline shape). The two BOM tests now assert the BOM survives; a new round-trip test covers LF, CRLF, BOM+CRLF, BOM+LF, no final newline and two final newlines, asserting setup keeps each shape and setup --remove restores the original bytes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(yarn-berry): run the new mode's berry gates before a takeover reverts the old one The reverts keep line endings as they are (a vendored or hosted revert never refuses on them, so a lock mixed after wiring stays mixed), while the forward hosted rewriter and vendored backend refuse a mixed file. Neither takeover checked first: - `scan`/`get --mode hosted` over a vendored berry purl reverted its wiring, ledger entry and artifact (`redirect_takeover_reverted_vendored`: "now fully hosted"), then the rewriter refused the mixed lock - `redirected: 0`, and the next `yarn install` pulled the unpatched registry package. - `vendor` / `scan --mode vendored` over a hosted berry purl reverted the hosted edits and dropped the redirect-ledger record (`vendor_takeover_reverted_redirect`), then failed `vendor_yarn_berry_mixed_line_endings`. Extract the rewriter's project gates into `redirect::preflight_yarn_berry_hosted` (mixed endings, cacheKey, `.yarnrc.yml` compressionLevel) and the backend's into `vendor::yarn_berry_vendor_preflight` (both files' endings, cacheKey, compressionLevel; berry flavor only), and run each before the matching takeover revert, mirroring the bun preflights - wet and --dry-run alike. A refused purl keeps the old mode's wiring byte-identical and is skipped / failed with the new mode's code. Tests: a hermetic in_process_vendor test drives both directions (mixed lock, mixed package.json, compressionLevel 9; wet and dry-run) and asserts the wiring snapshot is unchanged and no takeover is announced (fails on the pre-fix code in both directions); core unit tests pin that each preflight matches its forward gate's code and detail. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(yarn-berry): serialize yarn spawns on Windows around a shared-cache rename race The berry suites' parallel tests share one yarn cache folder. Two yarn processes fetching the same package both rename a .tmp over the cache zip, and on Windows the loser fails with EPERM while the winner holds the file (windows-latest yarn-berry 4.12.0: e2e_yarn4_workspaces_build hosted test, EPERM rename left-pad-npm-1.3.0-....zip-....tmp). A static lock in yarn_berry_common serializes yarn processes on Windows only (Unix rename-over is atomic), mirroring the DOTNET_SPAWN fix. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test: strip in-process env toggles unconditionally before spawning the CLI Test binaries that mix in-process command runs with spawned CLI runs raced: the in-process runs call apply_env_toggles, which std::env::set_var's SOCKET_OFFLINE / SOCKET_DEBUG / SOCKET_API_URL / SOCKET_PROXY_URL on the shared test process, and the spawn helpers only removed SOCKET_* vars that existed when they scanned the environment. A toggle set by a parallel test between that scan and the spawn was inherited. On test-release this made in_process_vendor's berry takeover test run its hosted scan offline ("cannot run with --offline/SOCKET_OFFLINE"). The helpers in all eight such binaries now remove those keys unconditionally; Command applies the removals to the environment captured at spawn time. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Branch
cleanup/socket-dir-hygiene(41 commits over main9489b18; 181 files, +20.7k/−11.1k — roughly a third is tests re-pinned to the new behavior, a third is deleted duplication).Why
A file-by-file audit (19 read-only reviewers, every finding checked by two adversarial verifiers: 339 confirmed / 14 refuted, 90 files, no P0) traced every stray file under
.socket/to three design choices rather than bugs: the advisory lock file was retained between runs by design, vendored mode wrote an agent-mode manifest as an intermediate, and a non-TTYscanauto-accepted its own prompt. This PR changes those three, removes the residue they left behind, and lands the bug fixes and consolidations the audit surfaced along the way.Measured against production (
minimist@1.2.2, free tier) before this PR: a plain humanscanwith stdin not a TTY downloaded, wrote.socket/manifest.json+ blobs, patchednode_modulesand left.socket/apply.lock;scan --mode vendoredwrote.socket/manifest.jsonand left the lock; hosted mode took no lock at all;rollbackandvendor --revertleft the lock and empty directories. After: 15 of 15 footprint checks pass (script in the scratchpad; the assertions are the table below).Behavior changes (MAJOR — see CHANGELOG
[Unreleased])apply.lockrepairdeleted itacquirecreates.socket/itself, verifies handle identity (same-file) after locking, unlinks while held on drop, then removes an empty.socket/; hosted mode now takes the lock around its first wet write; dry-runs never lock.socket/manifest.json;--detachedwas opt-in.socket/vendor/**(artifact,socket-patch.vendor.jsonmarker,state.jsonledger with embedded records);--detachedhidden no-op; legacy manifest records migrate into the ledger;list/rollback/remove/repair/vex/setup --checkread the ledgerscan--yesand no intent flag (--mode/--apply/--sync/--vendor/--redirect/--prune) is report-only, exit 0; human hosted scan gained the confirm prompt and the results table.socket/vendor/,blobs/,.socket/left behind.socket/; the zero-patch agent manifest is deliberately kept (pinned exit codes)Bug fixes riding along (each pinned by a test):
gcJSON mislabelled every non-purl vendor-GC marker aslock_held;removedropped a detached ledger entry on drift-keep and ignored--preserve-state;repairresolved qualified ledger keys with the base-purl resolver; the hosted takeover ran its symlink guard after the first wet write;replayflushed lockfiles with a truncating write;takeoverread project files without the FIFO/symlink guards;applyread blob/archive paths from unvalidated manifest strings and reported a patched file as failed on a post-rename chown error; gem revert deleted the copy dir without the drift-keep gate; the pypi flavors clobbered a lock changed during the wheel build;apply --silentexited 1 with no output on an all-unmatched manifest; the hammer test exposed a macOSEINVALrace in the new lock acquire (fixed and pinned).Consolidations: one fetch loop for the two download engines, one API client and one lock window per vendored run, one lock window across download → manifest → nested apply (
apply::run_locked), one vendored-revert loop, one artifact sweep, one.socketderivation, one ledger load per hosted run, shared atomic staging for redirect replay/takeover, the ~25 private copies of the guarded file readers replaced by theutils::fshelpers,cow.rsdeleted (rename-over is the documented defense), constantorg_slugparameters dropped from five API routes.Verification
cargo test --workspace --no-fail-fastat the final HEAD: 239 suites / 7,442 passed / 0 failed / 99 ignored (baseline on main: 7,317 passed; +125 tests, no new ignores).cargo clippy --workspace --all-features -- -D warnings(CI invocation): clean.vendor/**only; every reversal leaves nothing and restorespackage-lock.jsonbyte-identical; agent mode keeps manifest+blobs with no lock).16ee176..5f26b47) — including the drop-time unlink now gated on inode identity,.socket/go-patcheshusks pruned on Go reversal, orphan blobs unwound on error, the Bun preflight refusal ordered before the same-uuid ledger skip, the legacy-manifest migration scoped to the purls the run vendored,setup --checkandvexsurfacing an unreadable vendor ledger, hosted zero-grant runs never quarantining lock-free, and drift-keep parity extended to composer/maven/nuget.Residuals (deliberately not in this PR)
--manifest-pathwith a custom parent:acquirecreates the parent (the guard prunes only a dir literally named.socket); vendor/redirect ledgers still resolve against--cwdwhilelistuses the manifest's project root..socket-stage-*/.socket-dl-*leftovers inrepair.applymixed-vs-all-miss exit asymmetry stays as pinned.docs/testing/pdm-compatibility.md).e2e_vendor_*_build,mode_migration_*, nativee2e_bun_lockb, gem capstones) carry re-pinned D2 assertions that only CI can run.apply_with_unreadable_socket_dir_fails_closed.ApiClient::fetch_patch/search_patches_by_*/search_patches_batchlost their constant first parameter;patch::cowremoved.🤖 Generated with Claude Code
Note
High Risk
Breaking CLI contract across rollback, vendored footprint, scan non-TTY behavior, and lock/hosted write ordering touches security-sensitive patch application, manifest/ledger I/O, and reversal paths used in CI and production workflows.
Overview
This is a major (v5.0) hygiene and semantics pass:
.socket/should not linger after report-only or fully reverted runs, and several commands that used to mutate implicitly in CI now stay read-only unless you opt in..socket/footprint and locks.apply.lockis held only for the duration of mutating work (including hosted’s first wet write), then unlinked; empty.socket/dirs are removed on drop, so docs no longer tell users to gitignore the lock. Report-only / dry-run / zero-discoveryscanpaths avoid creating.socket/at all. Reversal commands (rollback,remove, hosted unwind, GC) prune emptied ledgers, artifact dirs, and the root when nothing remains (setup-owned files and a zero-patch agent manifest are kept on purpose).Vendored mode is manifest-free.
scan/get --mode vendoredstore patch records only in the vendor ledger under.socket/vendor/**;--detachedis a hidden no-op.list,rollback,remove,repair, andvexwere updated to read ledger-only state. CI’s Bun native matrix drops the separate vendored-detached leg because it matches vendored now.Command behavior (breaking).
rollbackfully unpatches across agent, vendored, and hosted, drops rolled-back manifest entries, and GCs blobs (unless--preserve-state). A plain non-TTYscanwithout--yesor mode/intent flags is report-only instead of auto-applying. README/CHANGELOG spell out JSON envelope changes (e.g. narrowedrollbackvendored: []).Supporting repo changes. Workspace adds
same-file(used for lock handle identity after acquire)..gitignorepicks up Python bytecode andsimplify-output/.Reviewed by Cursor Bugbot for commit e1c8cb6. Configure here.