Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -800,6 +800,19 @@ jobs:
- name: Run npm dispatch tests
run: node --test npm/socket-patch/bin/socket-patch.test.mjs

- name: Run npm schema tests
# The `./schema` export's zod tests. `npm install --no-save`, not
# `npm ci`, for the reason publish-npm.yml gives (a version-synced
# lock can name platform packages not yet on the registry); the
# platform binaries are optional and not needed here. The pack
# test runs again once `npm test` has built dist/, so it also
# checks that the compiled schema ships.
working-directory: npm/socket-patch
run: |
npm install --no-save --ignore-scripts --no-audit --no-fund --omit=optional
npm test
node --test --test-name-pattern="npm package contents" bin/socket-patch.test.mjs

# Compiles the CLI and every CLI test target once per OS (--all-features,
# so this is also the feature-gated suites' compile-rot check) and uploads
# the binaries the e2e, e2e-full and cargo-vex legs run. The legs run the
Expand Down
4 changes: 2 additions & 2 deletions crates/socket-patch-cli/CLI_CONTRACT.md
Original file line number Diff line number Diff line change
Expand Up @@ -904,7 +904,7 @@ A bare `rollback` (or a scoped one, for its scope) restores the SYSTEM to unpatc
5. **Manifest cleanup** — entries are removed ONLY for in-scope purls whose legs fully succeeded, were not-installed, or were release-variant siblings narrowed away by an attempted variant that succeeded (half a variant group never lingers — `remove` parity); drift-kept and failed purls keep their records, and a failed variant holds its whole group. No-op removals never rewrite the file. A failed write surfaces as `manifest_write_failed` (warning + `partial_failure` exit 1; GC still runs against the unchanged manifest).
6. **GC** — blob, diff and legacy package-archive sweeps against the post-removal manifest, using the same artifact-reference policy as `remove`, retaining beforeHash blobs for (a) removed-but-not-installed entries (a crawler miss must not destroy the only local revert data — `remove` parity) and (b) EVERY entry remaining in the post-removal manifest — still-active patches (failed, drift-kept, eco-/path-excluded) keep their revert data, so a scoped or failed run never destroys the blobs a later rollback needs; only blobs referenced solely by genuinely-removed entries are swept. GC errors warn (`cleanup_failed`) and continue — they never affect the exit (repair's posture).

**Confirmation prompt.** A wet, non-preserve run with work prompts once, remove-style, composing only the clauses that apply into one English list (`a and b`, `a, b, and c`) with counted nouns: `Roll back N patches`, `remove them from the local manifest`, `delete M vendored artifacts and their ledger records`, `restore H hosted packages to the upstream registry` (e.g. `Roll back 1 patch, remove it from the local manifest, and restore 1 hosted package to the upstream registry?`) — default yes, auto-accepted under `--yes`/`--json`/non-TTY (the shared `confirm` semantics; CI unaffected). Decline prints `Rollback cancelled.` and exits 0. `--dry-run` and `--preserve-state` runs are prompt-free (they delete no local state).
**Confirmation prompt.** A wet, non-preserve run with work prompts once, remove-style, composing only the clauses that apply into one English list (`a and b`, `a, b, and c`) with counted nouns: `Roll back N patches`, `remove them from the local manifest`, `delete M vendored artifacts and their ledger records`, `restore H hosted packages to the upstream registry` (e.g. `Roll back 1 patch, remove it from the local manifest, and restore 1 hosted package to the upstream registry?`) — default yes, auto-accepted under `--yes`/`--json`/non-TTY (the shared `confirm` semantics; CI unaffected). Decline prints `Cancelled; no changes made.` (stdout) and exits 0. `--dry-run` and `--preserve-state` runs are prompt-free (they delete no local state).

### `--preserve-state` (opt-out, both `rollback` and `remove`)

Expand Down Expand Up @@ -1133,7 +1133,7 @@ The v3.0 legacy names `SOCKET_PATCH_PROXY_URL`, `SOCKET_PATCH_DEBUG` and `SOCKET

## JSON output shapes

Every `--json` invocation emits a single JSON object that follows the **unified envelope** below. The envelope was introduced in v3.0; older per-command shapes are deprecated. See `src/json_envelope.rs` for the source of truth and `tests/cli_parse_*.rs` for snapshot tests that lock the shape.
Every `--json` invocation emits a single JSON object that follows the **unified envelope** below. The envelope was introduced in v3.0; older per-command shapes are deprecated. See `src/json_envelope.rs` for the source of truth; its unit tests pin the serialized names, and each command's e2e tests assert the envelope it emits. The `tests/cli_parse_*.rs` files pin the parsed clap arguments, not this shape (a few, such as `cli_parse_list.rs`, also spot-check `list`'s envelope).

### Envelope shape

Expand Down
74 changes: 35 additions & 39 deletions crates/socket-patch-cli/src/commands/scan/hosted.rs
Original file line number Diff line number Diff line change
Expand Up @@ -137,8 +137,9 @@ fn refuse(
}

/// The apply lock for a WET hosted run: the same `<manifest dir>/apply.lock`
/// `apply`/`rollback`/`remove`/`vendor` hold, so the takeover pre-reverts,
/// the ledger merge and the lockfile writes never race them. `acquire`
/// `apply`/`rollback`/`remove`/`vendor` hold, so the takeover pre-reverts
/// (lockfiles + the vendored ledger) and the lockfile writes never race
/// them. `acquire`
/// creates a missing `.socket/` and the guard's drop unlinks the lock file
/// and prunes an otherwise-empty `.socket/`, so a run that ends up writing
/// nothing leaves no residue. Contention / IO failures render through the
Expand Down Expand Up @@ -299,11 +300,10 @@ async fn installed_stale_positive_evidence(
/// refuses as a write root is still READ here
/// (`verification_only_gem_paths`): bundler installs into it.
/// * Records are found BY UUID (the fetch key, stable across purl
/// spellings): this run's fetched records first, then the redirect
/// ledger's persisted ones — a re-scan whose `/patches/view` fetch failed
/// transiently still re-fires from the ledger instead of silently
/// dropping the warning (`record_fetch_failed` covers the fetch failure
/// itself). Record availability is part of the candidate filter, and the
/// spellings) among this run's fetched records; v5 keeps no hosted
/// ledger to fall back on, so a uuid whose `/patches/view` fetch failed
/// is not judged (`record_fetch_failed` surfaces that failure). Record
/// availability is part of the candidate filter, and the
/// probe returns before any crawler work (or `gem env` subprocess spawn)
/// when no judgment is possible.
/// * PATCHED means [`verify_patch_record`] `Ok` — the one shared oracle
Expand All @@ -326,8 +326,7 @@ async fn gem_stale_install_warnings(
global: bool,
global_prefix: Option<std::path::PathBuf>,
confirmed: &[(String, String)],
// This run's fetched records MERGED with the ledger's persisted ones
// (the caller hands the post-merge ledger map).
// This run's fetched records, by uuid.
records: &std::collections::BTreeMap<String, socket_patch_core::manifest::schema::PatchRecord>,
gem_artifact_shas: &std::collections::BTreeMap<(String, String), String>,
) -> StaleInstallOutcome {
Expand Down Expand Up @@ -629,20 +628,20 @@ pub(super) async fn run_redirect(
/// ([`socket_patch_core::hosted::engine`], over a
/// [`ProjectView::Disk`](socket_patch_core::vendor::lock_inventory::ProjectView));
/// what stays here is what needs the host: reference grants and the other
/// network fetches, the apply lock (wet runs with a grant), the redirect
/// ledger load, the vendored→hosted takeover pre-revert (symlink-checked
/// first), the `pipenv --version` probe, the symlink guard, the ledger
/// merge-then-persist and the file writes, the gem / Python / vlt
/// network fetches, the apply lock (wet runs with a grant), the
/// vendored→hosted takeover pre-revert (symlink-checked first), the
/// `pipenv --version` probe, the symlink guard, the file writes, the gem /
/// Python / vlt
/// stale-install probes, and the optional VEX. Shared VERBATIM by `scan
/// --mode hosted` (its `--json` arm through the `run_redirect` wrapper, its
/// human arm through [`boxed_run_redirect_selected`] in `scan/mod.rs`; both
/// select via `discover_selected`, with no prompt) and by `get --mode
/// hosted` (which pins the advisory-resolved uuid), so all produce
/// identical on-disk results for the same selection. The redirect ledger
/// is loaded HERE, under the apply lock whenever this run holds one (never
/// handed in pre-loaded: a copy read before the lock could merge over a
/// concurrent writer's edits); a dry run or a zero-grant run reads it
/// strictly but writes nothing, quarantine included.
/// identical on-disk results for the same selection. v5 hosted mode keeps
/// no ledger (the lockfiles are the only record); the VENDORED ledger the
/// takeover needs is loaded HERE, under the apply lock whenever this run
/// holds one (never handed in pre-loaded: a copy read before the lock could
/// be saved over a concurrent writer's edits).
///
/// `scan_result` must be `Some` exactly when `common.json` is set (the
/// human/JSON split keys on `common.json`; a `--json` caller passing `None`
Expand Down Expand Up @@ -1091,10 +1090,10 @@ pub(crate) async fn run_redirect_selected(
std::collections::BTreeMap::new();
let mut record_warnings: Vec<socket_patch_core::patch::redirect::RewriteWarning> = Vec::new();

// SYMLINK GUARD (see `engine::guard`) — before the ledger and before any
// write, dry runs included, so a dry run predicts the refusal. The
// revert side (replay.rs) already refuses linked files, so the write
// side must too.
// SYMLINK GUARD (see `engine::guard`) — before any write, dry runs
// included, so a dry run predicts the refusal. The revert side (the
// hosted → upstream restore's staged flush) already refuses linked
// files, so the write side must too.
if let Some(refusal) = engine::guard(&view, &done, &candidates) {
return refuse(common, scan_result.take(), &refusal);
}
Expand Down Expand Up @@ -1176,8 +1175,8 @@ pub(crate) async fn run_redirect_selected(
// the writes so the warning describes the project as this run leaves it.
// Idempotent re-scans re-confirm and re-probe, so the warning keeps
// firing until the stale materialization is actually gone. Skipped
// EXPLICITLY on --dry-run: the probe's ledger-record fallback would
// otherwise judge state the run did not (re)create.
// EXPLICITLY on --dry-run: nothing was written, so the probe would
// judge state the run did not (re)create.
let gem_stale: StaleInstallOutcome = if common.dry_run {
StaleInstallOutcome::default()
} else {
Expand Down Expand Up @@ -1330,7 +1329,7 @@ pub(crate) async fn run_redirect_selected(
// Stale-flagged purls are EXCLUDED from assume_applied: the same-run
// envelope carries a redirect_gem_stale_install warning proving the
// installed materialization unpatched, so attesting that purl from
// the ledger would contradict the run's own warning. Excluded purls
// this run's records would contradict the run's own warning. Excluded purls
// fall back to `vex`'s normal installed-tree verification.
// A confirmed uuid whose bundled instance the rewriter had to skip
// (#469) leaves that copy unpatched, so it too is verified, never
Expand Down Expand Up @@ -3382,8 +3381,8 @@ mod tests {

/// Probe invocation with the default surface (project-local discovery,
/// no artifact shas) — tests override the knobs they exercise.
/// `records` is the merged map production hands over (this run's
/// fetched records plus the ledger's persisted ones).
/// `records` is the map production hands over: this run's fetched
/// records, by uuid.
async fn probe(
cwd: &std::path::Path,
confirmed: &[(String, String)],
Expand Down Expand Up @@ -3712,25 +3711,22 @@ mod tests {
);
}

/// RE-FIRE guarantee: when this run's record fetch failed (no fresh
/// records), the merged map the caller hands over still carries the
/// redirect ledger's PERSISTED record under whatever purl key the
/// ledger used — and the probe's uuid lookup judges from it, so a
/// transient /patches/view failure cannot silently retire the warning
/// while the stale materialization is still there.
/// The probe links a record to a confirmed purl by uuid alone: a
/// record keyed under the API's qualified purl spelling (not the
/// confirmed purl) must still judge the stale materialization.
#[tokio::test]
async fn gem_stale_probe_judges_from_persisted_ledger_records() {
async fn gem_stale_probe_matches_records_by_uuid_not_purl_key() {
let stale = tempfile::tempdir().unwrap();
materialize_gem(stale.path(), GEM_UPSTREAM);
// Persisted under the API's qualified spelling, not the confirmed
// Keyed under the API's qualified spelling, not the confirmed
// purl: only the uuid links them.
let mut ledger_only = std::collections::BTreeMap::new();
ledger_only.insert(format!("{GEM_PURL}?platform=ruby"), gem_record());
let out = probe(stale.path(), &one_confirmed(), &ledger_only).await;
let mut qualified = std::collections::BTreeMap::new();
qualified.insert(format!("{GEM_PURL}?platform=ruby"), gem_record());
let out = probe(stale.path(), &one_confirmed(), &qualified).await;
assert_eq!(
out.warnings.len(),
1,
"the ledger records must keep the warning firing across flaky fetches"
"a record keyed under another purl spelling must still match by uuid"
);
}

Expand Down
5 changes: 4 additions & 1 deletion crates/socket-patch-cli/src/commands/scan/policy.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,10 @@ pub(crate) fn load_invocation_policy(args: &ScanArgs) -> Result<InvocationPolicy

/// The marker files of a disk project root: the same lock markers the
/// in-memory engine detects roots by, plus the maven/nuget markers disk
/// scans support. Manifests are not markers (so both engines agree).
/// scans support. When a directory has none of those, its manifests
/// ([`MANIFEST_MARKERS`] and the JVM build files) are its markers instead;
/// the in-memory engine has no such fallback (manifests alone never make
/// a root there), so for a lockless directory the two engines disagree.
pub(crate) fn dir_markers(dir: &Path) -> Vec<String> {
let mut markers: Vec<String> = std::fs::read_dir(dir)
.map(|entries| {
Expand Down
16 changes: 7 additions & 9 deletions crates/socket-patch-core/src/hosted/engine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
//! 1. [`build_candidates`] — reference grants → rewriter overrides.
//! 2. [`bun_lockb_symlinked`] — the binary-lock symlink refusal.
//! 3. vlt artifact preflight ([`super::vlt`]) + [`withhold_everywhere`].
//! 4. (caller) the apply lock, the ledger, the vendored→hosted takeover.
//! 4. (caller) the apply lock and the vendored→hosted takeover.
//! 5. [`read_candidate_files`] → [`wheel_targets`] → (caller) wheel metadata,
//! and [`yarn_berry_manifest_targets`] → (caller) served npm manifests.
//! 6. [`rewrite`] — the rewriters, the pnpm `trustLockfile` and npm
Expand All @@ -17,8 +17,7 @@
//!
//! Nothing here writes, spawns, reads the environment or touches the
//! network: every host effect (locking, probes, record fetches, the commit
//! of the rewritten files, the redirect ledger in [`super::ledger`]) stays
//! with the caller.
//! of the rewritten files) stays with the caller.

use std::collections::{BTreeMap, BTreeSet, HashMap};

Expand Down Expand Up @@ -1233,14 +1232,13 @@ pub async fn rewrite(
);
if let Some((text, edit)) = trust_config_write {
rewrite.files.insert(PNPM_WORKSPACE_REL.to_string(), text);
// Appended last: `--revert` walks edits in reverse, so the trust key
// is unwound before the lock originals are restored.
// Appended last, after the lock edits it serves. v5 keeps no hosted
// ledger, so nothing replays these edits; the order is write order.
rewrite.edits.push(edit);
}
if let Some((text, edit)) = npmrc_config_write {
rewrite.files.insert(NPMRC_REL.to_string(), text);
// Appended after the lock edits for the same reason: a whole-ledger
// replay unwinds the setting before the lock originals it served.
// Appended after the lock edits, like the pnpm trust key above.
rewrite.edits.push(edit);
}
let rewritten: Vec<String> = rewrite
Expand Down Expand Up @@ -1903,8 +1901,8 @@ fn file_ecosystem(rel: &str) -> Option<&'static str> {
.then_some("pypi")
}

/// SYMLINK GUARD — fail-closed, whole rewrite, before the ledger and before
/// any write (hosted rewrites are transactional). The writer stages next to
/// SYMLINK GUARD — fail-closed, whole rewrite, before any write (hosted
/// rewrites are transactional). The writer stages next to
/// the path and renames over it, which REPLACES a symbolic link with a
/// detached regular copy: the link target goes stale and a revert restores
/// bytes but never the link. Applies to every ecosystem's files and to dry
Expand Down
17 changes: 8 additions & 9 deletions crates/socket-patch-core/src/patch/redirect/staged.rs
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
//! Staged, fail-closed file I/O shared by the hosted-redirect reverts — the
//! per-purl takeover ([`super::takeover`]) and the whole-ledger replay
//! ([`super::replay`]).
//! Staged, fail-closed file I/O for the hosted → upstream restore
//! ([`super::upstream`]).
//!
//! Both reverts resolve every inverse against a STAGED view of the project
//! and let nothing reach disk until all of them have resolved, so a drift
//! refusal leaves the project byte-identical. This module is that staging
//! layer: FIFO-safe reads of untrusted project files, the staged view, and
//! one flush with the same guards on both sides (a symlink or FIFO squatting
//! a path refuses; every write is atomic and keeps the file's mode).
//! The restore resolves every pin against a STAGED view of the project and
//! lets nothing reach disk until all of them have resolved, so a refusal
//! leaves the project byte-identical. This module is that staging layer:
//! FIFO-safe reads of untrusted project files, the staged view, and one
//! flush with the same guards on both sides (a symlink or FIFO squatting a
//! path refuses; every write is atomic and keeps the file's mode).

use std::collections::BTreeMap;
use std::path::Path;
Expand Down
5 changes: 3 additions & 2 deletions crates/socket-patch-core/src/update/release.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@
//! for a fallback that only fires when the redirect shape drifts.
//!
//! All fetch sizes are capped and every request carries an explicit
//! timeout: a hung self-update is strictly worse than a hung scan, so this
//! module does not inherit the API client's no-timeout posture.
//! whole-request deadline ([`UpdateTimeouts`]), unlike the API client's
//! connect + per-read bounds (`api::retry::ApiTimeouts`): a hung
//! self-update is strictly worse than a hung scan.

use std::time::Duration;

Expand Down
9 changes: 5 additions & 4 deletions crates/socket-patch-core/src/vex/discover/pypi_other.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,11 @@
//! a url naming another package, is diagnosed, never trusted. When the
//! url does not carry the `patch/pypi/…` levels (a self-hosted
//! `--patch-server-url` layout), the artifact filename alone supplies
//! the version. This is the host-allowlisted twin of
//! `vendor::pypi_pipenv::is_socket_hosted_reference`, which accepts the
//! same path shape on ANY https host because it only decides ownership
//! of a lock entry, not attestation;
//! the version. The path is parsed by the same
//! `vendor::lock_inventory::pypi::hosted_artifact_url` that
//! `hosted_pypi_reference` (hosted Pipenv rotation and the vendored
//! Pipenv guard's "is this lock entry ours" check) uses, and both apply
//! the same patch-server origin allowlist;
//! * a root-anchored `.socket/vendor/pypi/<uuid>/<wheel>` path
//! ([`vendor_ref`]) → [`WiringMode::Vendored`]. The wheel must be a single
//! PEP 427 filename (or a server sdist's `dist-version.tar.gz`) naming the
Expand Down
Loading
Loading