v5 WS4/WS6: one hosted engine for disk + memory; unified Ledgers view

CHANGELOG.md

@@ -1817,6 +1817,19 @@ into the new version's section — see docs/releasing.md.
 
 ### Changed
 
+- **One owner rule for the patch stores.** `list`, `vex`, `scan`'s
+  `updates[]`, `rollback` and `remove` now read the
+  manifest and the vendor ledger (plus, in `vex`, the hosted records)
+  through one view (`socket_patch_core::ledgers`) with one precedence:
+  manifest, then vendor ledger, then hosted records, by ledger key; a
+  manifest key claims every vendor entry filed under it or naming it as
+  base purl. Visible differences: `scan`'s `updates[]` no longer folds a
+  vendor entry the manifest claims by base purl; `vex` treats every vendor
+  entry a manifest key claims as a fallback copy of that key's record (a
+  second variant of the same base purl used to become its own candidate).
+  `get`'s installed-version narrowing now uses
+  `scan`'s lockfile and vendored-ledger discovery, so a corrupt vendor
+  ledger falls back to the committed artifacts there too.
 - **The npm crawl skips tagged cache directories.** The walk that finds
   workspace `node_modules` trees no longer descends into a directory that
   carries a [Cache Directory Tagging](https://bford.info/cachedir/)

crates/socket-patch-cli/CLI_CONTRACT.md

@@ -30,7 +30,7 @@ Rows are in `--help` order (v5.0): the hosted/vendored workflow (`scan` → `vex
 
 **Root `--update` flag.** `socket-patch --update [VERSION]` updates the binary itself from GitHub Releases. It is a root flag, not a subcommand: argv is rewritten (the same mechanism as the bare-UUID fallback) onto an internal hidden subcommand whose name carries no stability guarantee — script the flag, never the internal name. Combining the flag with a subcommand (`socket-patch --update scan`) is a usage error (exit 2). Full contract: [Self-update contract](#self-update-contract-socket-patch---update).
 
-**Internal `hosted-bundle` subcommand.** `socket-patch hosted-bundle` is a hidden, INTERNAL parity/debug harness for the in-memory hosted engine (`src/hosted_memory/`, the engine the Node addon embeds): it reads a JSON bundle `{"files": {path: text}, "binaryFiles"?: {path: base64}, "presentOnly"?: [path], "symlinks"?: [path], "projectRoots"?: [dir], "pipenvMajor"?: n, "batchSize"?: n}` on stdin, queries the authenticated org API built from `--api-url` / `--api-token` / `--org` only (both of the latter are required; no public-proxy fallback), and prints the engine result — or `{"status":"error","error":{"code","message"}}` with exit 1 (exit 2 for unusable input or missing credentials). It never touches the filesystem. Its name, input and output carry NO stability guarantee; do not script it.
+**Internal `hosted-bundle` subcommand.** `socket-patch hosted-bundle` is a hidden, INTERNAL parity/debug harness for the in-memory hosted engine (`socket-patch-core` `src/hosted/memory/`, the engine the Node addon embeds): it reads a JSON bundle `{"files": {path: text}, "binaryFiles"?: {path: base64}, "presentOnly"?: [path], "symlinks"?: [path], "projectRoots"?: [dir], "pipenvMajor"?: n, "batchSize"?: n}` on stdin, queries the authenticated org API built from `--api-url` / `--api-token` / `--org` only (both of the latter are required; no public-proxy fallback), and prints the engine result — or `{"status":"error","error":{"code","message"}}` with exit 1 (exit 2 for unusable input or missing credentials). It never touches the filesystem. Its name, input and output carry NO stability guarantee; do not script it.
 
 ## Global arguments
 

crates/socket-patch-cli/src/commands/context.rs

@@ -0,0 +1,100 @@
+//! The project a command reads, loaded lazily and at most once per run:
+//! the patch stores ([`LoadedLedgers`]: manifest + both ledgers), the lock
+//! set (the lockfile inventory and its refused npm layouts) and the
+//! lockfile wiring discovery. `scan`, `vendor`, `vex`, `list` and `get`
+//! read these through one [`ProjectContext`] instead of each re-loading
+//! and re-merging them its own way.
+//!
+//! The lock set and the discovery read `--cwd` through one
+//! [`DiskSnapshot`], so each lock and config file is read once and both see
+//! the same bytes.
+//!
+//! Everything here is a read-only snapshot. A command that writes a store
+//! under the apply lock (the hosted engine, rollback, remove) re-loads it
+//! under that lock instead of trusting a pre-lock snapshot, and an embedded
+//! `--vex` after the writes loads its own inputs.
+
+use std::path::PathBuf;
+
+use socket_patch_core::ledgers::{Ledgers, LoadedLedgers};
+use socket_patch_core::vendor::lock_inventory::{
+    DiskSnapshot, LockfileEntry, ProjectView, UnsupportedNpmLayout,
+};
+use socket_patch_core::vex::discover::Discovery;
+use tokio::sync::OnceCell;
+
+use crate::args::GlobalArgs;
+
+/// The project's lockfile inventory and the npm layouts it refused.
+pub(crate) struct LockSet {
+    pub(crate) entries: Vec<LockfileEntry>,
+    pub(crate) unsupported: Vec<UnsupportedNpmLayout>,
+}
+
+pub(crate) struct ProjectContext<'a> {
+    pub(crate) common: &'a GlobalArgs,
+    /// Where the ledgers live: the manifest's project (see
+    /// [`GlobalArgs::project_root`]).
+    pub(crate) root: PathBuf,
+    snapshot: DiskSnapshot<'a>,
+    ledgers: OnceCell<LoadedLedgers>,
+    locks: OnceCell<LockSet>,
+    discovery: OnceCell<Discovery>,
+}
+
+impl<'a> ProjectContext<'a> {
+    pub(crate) fn new(common: &'a GlobalArgs) -> Self {
+        Self::rooted(common, common.project_root())
+    }
+
+    /// A context whose ledgers load from `root` (commands that read the
+    /// ledgers of `--cwd` rather than of the manifest's project).
+    pub(crate) fn rooted(common: &'a GlobalArgs, root: PathBuf) -> Self {
+        Self {
+            common,
+            root,
+            snapshot: DiskSnapshot::new(&common.cwd),
+            ledgers: OnceCell::new(),
+            locks: OnceCell::new(),
+            discovery: OnceCell::new(),
+        }
+    }
+
+    /// The three stores, each with its own load outcome.
+    pub(crate) async fn loaded(&self) -> &LoadedLedgers {
+        self.ledgers
+            .get_or_init(|| async {
+                LoadedLedgers::load(&self.root, &self.common.resolved_manifest_path()).await
+            })
+            .await
+    }
+
+    /// The readable stores as one view (a failed store reads as absent).
+    pub(crate) async fn ledgers(&self) -> Ledgers<'_> {
+        self.loaded().await.view()
+    }
+
+    /// The lockfile inventory of `--cwd`.
+    pub(crate) async fn locks(&self) -> &LockSet {
+        self.locks
+            .get_or_init(|| async {
+                let (entries, unsupported) =
+                    socket_patch_core::vendor::lock_inventory::inventory_project_diagnosed_in(
+                        &ProjectView::Snapshot(&self.snapshot),
+                    )
+                    .await;
+                LockSet {
+                    entries,
+                    unsupported,
+                }
+            })
+            .await
+    }
+
+    /// The lockfile wiring discovery of `--cwd` ([`super::discover_wiring`]).
+    pub(crate) async fn discovery(&self) -> &Discovery {
+        self.discovery
+            .get_or_init(|| super::discover_wiring_in(self.common, &self.snapshot))
+            .await
+    }
+}

crates/socket-patch-cli/src/commands/get.rs

@@ -13,8 +13,10 @@ use socket_patch_core::crawlers::fuzzy_match::fuzzy_match_packages;
 use socket_patch_core::crawlers::{CrawlerOptions, Ecosystem};
 use socket_patch_core::formats::pnpm::PnpmLock;
 use socket_patch_core::manifest::operations::{read_manifest, write_manifest};
+pub(crate) use socket_patch_core::manifest::records::record_from_patch_response;
+use socket_patch_core::manifest::records::{build_patch_record, files_for_manifest};
 use socket_patch_core::manifest::schema::{
-    PatchFileInfo, PatchManifest, PatchRecord, VulnerabilityInfo,
+    PatchFileInfo, PatchManifest, PatchRecord,
 };
 use socket_patch_core::patch::apply::{is_valid_blob_hash, select_installed_variants};
 use socket_patch_core::patch::apply_lock::{LockError, LockGuard};
@@ -366,43 +368,6 @@ async fn unwind_new_blobs(blobs_dir: &Path, hashes: &[String]) {
     }
 }
 
-/// Convert the API-shaped vulnerability map on `PatchResponse` into the
-/// serialization-shaped map stored in the manifest.
-fn vulnerabilities_for_manifest(
-    vulns: &HashMap<String, VulnerabilityResponse>,
-) -> HashMap<String, VulnerabilityInfo> {
-    vulns
-        .iter()
-        .map(|(id, v)| {
-            (
-                id.clone(),
-                VulnerabilityInfo {
-                    cves: v.cves.clone(),
-                    summary: v.summary.clone(),
-                    severity: v.severity.clone(),
-                    description: v.description.clone(),
-                },
-            )
-        })
-        .collect()
-}
-
-/// Build the `PatchRecord` that will be inserted into the manifest for
-/// `patch`. `files` is the (purl-keyed) before/after-hash map the
-/// caller built — semantics for what counts as a "patchable file" differ
-/// between the get and download flows, so the caller owns that decision.
-fn build_patch_record(patch: &PatchResponse, files: HashMap<String, PatchFileInfo>) -> PatchRecord {
-    PatchRecord {
-        uuid: patch.uuid.clone(),
-        exported_at: patch.published_at.clone(),
-        files,
-        vulnerabilities: vulnerabilities_for_manifest(&patch.vulnerabilities),
-        description: patch.description.clone(),
-        license: patch.license.clone(),
-        tier: patch.tier.clone(),
-    }
-}
-
 /// Build a file map keyed by path, keeping only files that carry BOTH
 /// hashes — the rule used ONLY for installed-distribution matching in
 /// [`filter_to_installed_releases`]. New files (no `beforeHash`) can
@@ -426,44 +391,6 @@ fn files_with_both_hashes(patch: &PatchResponse) -> HashMap<String, PatchFileInf
     files
 }
 
-/// Build the manifest-shaped `files` map from a fetched patch view,
-/// keeping EVERY file the patch touches — including net-new files the
-/// patch ADDS, which carry an `afterHash` but no `beforeHash`. A new
-/// file is recorded with an empty-string `beforeHash` sentinel, the same
-/// convention `save_and_apply_patch`'s by-uuid path relies on: apply
-/// treats an empty `beforeHash` as "create this file" and
-/// [`select_installed_variants`] treats it as non-discriminating.
-///
-/// This is the shared record-building rule for the scan/download/vendor
-/// flows AND the single-uuid apply path, so `get <uuid>` and
-/// `scan`/`apply`/`vendor` all record and write the same set of files.
-/// A both-hashes rule here would drop every added file (e.g. a whole-crate
-/// cargo export where ALL files lack a `beforeHash`, recorded as `files:{}`
-/// while reporting `applied:1`).
-fn files_for_manifest(patch: &PatchResponse) -> HashMap<String, PatchFileInfo> {
-    let mut files = HashMap::new();
-    for (file_path, file_info) in &patch.files {
-        if let Some(after) = &file_info.after_hash {
-            files.insert(
-                file_path.clone(),
-                PatchFileInfo {
-                    before_hash: file_info.before_hash.clone().unwrap_or_default(),
-                    after_hash: after.clone(),
-                },
-            );
-        }
-    }
-    files
-}
-
-/// `(purl, manifest record)` from a fetched patch view — retains
-/// patch-added new files via [`files_for_manifest`].
-pub(crate) fn record_from_patch_response(patch: &PatchResponse) -> (String, PatchRecord) {
-    (
-        patch.purl.clone(),
-        build_patch_record(patch, files_for_manifest(patch)),
-    )
-}
 
 #[derive(Args)]
 pub struct GetArgs {
@@ -1523,9 +1450,10 @@ struct InstalledNarrowing {
 ///   installed copy (CI manifest-maintenance);
 /// * hosted/vendored modes only: resolved in the project lockfile(s)
 ///   (hosted rewrites the lock; vendored auto-fetches pristine) or claimed
-///   by the vendor ledger (fresh-clone re-vendor) — mirroring scan's
-///   lockfile/vendored-ledger discovery supplements, including their
-///   global-scan gate.
+///   by the vendor ledger (fresh-clone re-vendor) — scan's own
+///   lockfile/vendored-ledger discovery supplements (a corrupt vendor
+///   ledger falls back to the committed artifacts, as in scan), including
+///   their global-scan gate.
 ///
 /// PnP layouts are surfaced, never silently misreported: yarn PnP packages
 /// are structurally unpatchable in every mode (skip records carry
@@ -1564,24 +1492,27 @@ async fn filter_to_installed_purls(
     let found = find_packages_for_rollback(&partitioned, &common.crawler_options(), true).await;
     let mut present: HashSet<String> = found.keys().map(|k| canon(k)).collect();
 
+    let ctx = super::context::ProjectContext::rooted(common, common.cwd.clone());
     // Manifest membership counts as presence (read-only probe: a corrupt
     // manifest degrades to "no extension" here — the download path's
     // fail-closed read still guards every write).
-    if let Ok(Some(manifest)) = read_manifest(&common.resolved_manifest_path()).await {
+    if let Some(manifest) = ctx.ledgers().await.manifest {
         present.extend(manifest.patches.keys().map(|k| canon(k)));
     }
 
-    // Lockfile + vendor-ledger supplements (scan's discovery gate: never on
-    // global scans, which target the machine tree, not this project).
+    // scan's lockfile + vendored-ledger discovery supplements (and their
+    // gate: never on global scans, which target the machine tree, not this
+    // project).
     let mut pnp_diags: Vec<lock_inventory::UnsupportedNpmLayout> = Vec::new();
-    if !common.global && common.global_prefix.is_none() {
-        let (entries, unsupported) = lock_inventory::inventory_project_diagnosed(&common.cwd).await;
-        pnp_diags = unsupported;
+    if !common.is_global() {
+        let supplement = super::scan::project_lockfile_supplement(&ctx, &[], None).await;
+        pnp_diags = supplement.unsupported;
         if mode != super::scan::ScanMode::Agent {
-            present.extend(entries.iter().map(|e| canon(&e.purl)));
-            if let Ok(state) = socket_patch_core::vendor::load_state(&common.cwd).await {
-                present.extend(state.entries.values().map(|e| canon(&e.base_purl)));
-            }
+            present.extend(supplement.entries.iter().map(|e| canon(&e.purl)));
+            let vendored =
+                super::scan::project_vendored_supplement(common, &[], &ctx.loaded().await.vendor)
+                    .await;
+            present.extend(vendored.iter().map(|p| canon(&p.purl)));
         }
     }