Skip to content

Commit 4868030

Browse files
committed
feat(vex): manifest-less VEX — attest hosted and vendored patches from lockfiles
`socket-patch vex` and the embedded `apply` / `scan` / `vendor --vex` now attest hosted and vendored patches with no `.socket/manifest.json` — and with no `.socket/vendor` ledgers either — by reading the wiring out of the project's lockfiles (`vex::discover`). This covers a depscan-opened PR, a clone that never committed its ledgers, and a lock-only CI checkout. Record view (`commands::vex_sources::plan`): four sources — the manifest, the redirect ledger's records, the vendor ledger's embedded records and lockfile discovery — merge into one view. A candidate's record must carry the uuid the lockfile actually WIRES; it comes from the first source that has one, else (online only) from the patch API by uuid, 10 fetches at a time with `get`'s one-shot 401/403 → public-proxy fallback (#247's uuid-only `ApiClient::fetch_patch`). Nothing is written: vex never takes the apply lock, never creates `.socket/`, never writes the manifest, and the `socket-patch.vendor.json` marker is never a record. Gates, applied before hashing and kept under `--no-verify` (which now skips only the hashing): - a vendor-ledger entry attests only while a lockfile still wires its artifact (`vendor_unwired`); a redirect-ledger record only while a lockfile wires its hosted patch (`redirect_unwired`); discovery is authoritative for every uuid a read file mentions, so a rejected mention keeps nothing alive; - `record_unavailable` (offline, 404, refused, transport error — the run continues), `record_mismatch`, `wiring_conflict` (lockfiles wire one package to several patches); - a malformed / unreadable `.socket/vendor/state.json` is the hard error `vendor_ledger_corrupt` (exit 2), mirroring `redirect_ledger_corrupt` — this supersedes #247's degrade-and-disclose posture for `vex` only; `setup --check` keeps #247's `vendor_context_from` / `warn_unreadable_vendor_state` path. Evidence: vendored refs hash the committed artifact (the ledger entry when it names the wired artifact, else one synthesized from the ref); hosted refs hash the installed copies the build CONSUMES (`vex_consumed::hosted_consumed_copies` → core `VendorContext::hosted` / `HostedCopies`: the Go replacement module, the Socket-registry cargo src dir, maven's suffixed version — never a pristine sibling), and with nothing installed a discovered pinned reference attests from its lockfile pin, like in-run `scan --mode hosted --vex`. Discovered refs bypass the Property 7 ecosystem filter. Commands: - `apply --vex` / `vendor --vex` with no manifest attest what the lockfiles and ledgers wire (nothing anywhere keeps the calm exit 0 and removes a stale document; `apply --check` and `--dry-run` never generate); #247's no-manifest lines ("No patch manifest found; nothing to apply.", "No manifest found, nothing to vendor.") are kept; - failed VEX runs carry the discovery diagnostics into `warnings[]` (standalone envelope, embedded envelopes, scan JSON — hosted included); - `manifest_not_found` now means no manifest AND nothing wired; - human output: `Note:` lines for superseded records / fetch failures, phrased omission reasons. Writer hardening: the manifest-driven standalone `vendor` now embeds the patch `record` in its ledger entries too (vendored mode already does, as `detached` entries). `detached` stays the "no manifest owner" flag, so the manifest reconcile, legacy-manifest migration and get/scan idempotency from #247 are unaffected. Every reader of embedded records shares one ownership rule (`commands::vendor_record_is_unowned`): a detached entry's record always stands alone, a standalone `vendor` entry's fallback copy only when no manifest entry covers it (by ledger key or base purl). `vex`, `list` and `setup --check` (`fold_vendor_records`, formerly `fold_detached_records`) all apply it, so one tree never lists "no patches" while its VEX document attests one. `repair` stays narrower: it keeps preferring a manifest that moved on to a newer uuid and falls back to the embedded copy only with no manifest at all. One liveness rule for vex and scan: `scan`'s cross-mode takeover classification (`classify_overlap_takeover`), its `hosted_wiring_retained` warning and `redirectState.wiringLive` ask the same core discovery (`commands::discover_wiring`) and liveness rule (`Discovery::redirect_record_live` / `vendor_entry_live` / `wires_package`, through one `LedgerLiveness` holder per call site) that gate attestation, replacing scan's private cargo / hosted / vendored checks and its looser text scan. The CLI also shares one purl splitter (`utils::purl::purl_parts`), one vendor-ledger lookup (`vendor::state::lookup_entry_kv`) and one npm alias-aware identity crawl (`ecosystem_dispatch::npm_paths_by_identity`) across vex, scan and vendor, and pairs PEP 723 script locks through `utils::python_lock::script_of_lock` like the rewriters do. Output follows the conventions from #248: `ui::plural` counts, a `ui::StatusLine` progress line for record fetches, and the shared `format_vex_written` / `format_vex_dry_run_skip` lines on the manifest-less `apply --vex` / `vendor --vex` paths. Manifest-less `vex` honours `--dry-run` and `-O -` like the manifest path. Product auto-detection adds go.mod, composer.json, pom.xml, a single `*.csproj` and a single `*.gemspec`, after the existing probes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1 parent 52e71ee commit 4868030

20 files changed

Lines changed: 4089 additions & 693 deletions

File tree

‎crates/socket-patch-cli/src/args.rs‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -188,8 +188,11 @@ pub struct GlobalArgs {
188188
/// Override the host of the prebuilt-archive download URL the vendoring
189189
/// service returns. When set, the CLI rewrites the
190190
/// scheme + host (+ port) of the returned URL to this base, preserving the
191-
/// path. Mainly for local-dev / testing, where the host the server bakes
192-
/// into the URL is not the one to actually fetch from.
191+
/// path. `vex` and `scan` also accept lockfile URLs on this origin as
192+
/// hosted patch references (attested, and counted as live hosted wiring)
193+
/// next to patch.socket.dev. Mainly for local-dev / testing, where the
194+
/// host the server bakes into the URL is not the one to actually fetch
195+
/// from.
193196
// A dev/testing knob: listed in `--help`, left out of the `-h` summary.
194197
#[arg(
195198
help_heading = GLOBAL_OPTIONS,

‎crates/socket-patch-cli/src/commands/apply.rs‎

Lines changed: 83 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,9 @@ use std::time::Duration;
2424
use crate::args::{apply_env_toggles, GlobalArgs};
2525
use crate::commands::fetch_stage::{stage_patch_sources, StageOutcome, StagedSources};
2626
use crate::commands::lock_cli::acquire_or_emit;
27-
use crate::commands::vex::{generate_vex_from_manifest_path, VexEmbedArgs};
27+
use crate::commands::vex::{
28+
generate_vex_from_manifest_path, generate_vex_without_manifest, ManifestlessVex, VexEmbedArgs,
29+
};
2830
use crate::ecosystem_dispatch::{find_all_packages_for_purls, partition_purls};
2931
use crate::json_envelope::{
3032
AppliedVia, Command, Envelope, EnvelopeError, PatchAction, PatchEvent, PatchEventFile,
@@ -791,18 +793,90 @@ pub async fn run(args: ApplyArgs) -> i32 {
791793
{
792794
return refuse_yarn_pnp(&args);
793795
}
796+
// Nothing to apply — but `--vex` may still have something to
797+
// attest: hosted / vendored patches are wired by the lockfiles (and
798+
// the `.socket/vendor` ledgers), not the manifest, and a
799+
// `scan --mode hosted|vendored` or depscan checkout (both modes are
800+
// manifest-free) has none. Before, this branch returned before VEX
801+
// generation, so `apply --vex` exited 0 having written NO document
802+
// (and left a previous run's at the path). Nothing referenced
803+
// anywhere keeps the historical calm exit 0 (a stale document is
804+
// still removed); any other VEX failure flips the exit like the
805+
// with-manifest path. A dry run applies nothing, so it skips
806+
// generation (main-path parity), and so does `--check`: it is
807+
// read-only, lock-free and offline-safe — it never crawls, fetches
808+
// or writes — and the with-manifest path returns from it before any
809+
// VEX work (an ambient `SOCKET_VEX` must not turn an audit job's
810+
// `apply --check` into a VEX run).
811+
// The host line first: the VEX run below prints its own warnings to
812+
// stderr as it goes, and they read as part of the `--vex` side
813+
// effect only after the command has said what it did.
814+
if !args.common.json && !args.common.silent {
815+
// Names the manifest, not the folder: hosted- and vendored-mode
816+
// projects have a `.socket/` (their ledgers live under
817+
// `.socket/vendor/`) and still nothing for `apply` to do.
818+
println!("No patch manifest found; nothing to apply.");
819+
}
820+
let vex_result = if !args.common.dry_run && !args.check && args.vex.vex.is_some() {
821+
let params = args.vex.to_build_params();
822+
Some(generate_vex_without_manifest(&args.common, &params, &manifest_path).await)
823+
} else {
824+
None
825+
};
826+
let vex_path = || {
827+
args.vex
828+
.vex
829+
.as_ref()
830+
.expect("vex_result is Some only when --vex was given")
831+
};
794832
if args.common.json {
795833
let mut env = Envelope::new(Command::Apply);
796834
env.status = Status::NoManifest;
797835
env.dry_run = args.common.dry_run;
836+
match &vex_result {
837+
Some(ManifestlessVex::Written(summary)) => {
838+
env.vex = Some(VexSummary {
839+
path: vex_path().display().to_string(),
840+
statements: summary.statements,
841+
format: "openvex-0.2.0".to_string(),
842+
warnings: summary.warnings.clone(),
843+
});
844+
}
845+
Some(ManifestlessVex::Failed(e)) => {
846+
// The discovery diagnostics and the omitted patches are
847+
// the only explanation of the failure (why a lockfile
848+
// mention is not live wiring, which gate refused what):
849+
// same channel as the success path's advisories.
850+
env.warnings.extend(e.embedded_warnings());
851+
env.mark_error(EnvelopeError::new(e.code, e.message.clone()));
852+
}
853+
Some(ManifestlessVex::NothingToAttest(warnings)) => {
854+
env.warnings.extend(warnings.iter().cloned());
855+
}
856+
None => {}
857+
}
798858
println!("{}", env.to_pretty_json());
799-
} else if !args.common.silent {
800-
// Names the manifest, not the folder: hosted- and vendored-mode
801-
// projects have a `.socket/` (their ledgers live under
802-
// `.socket/vendor/`) and still nothing for `apply` to do.
803-
println!("No patch manifest found; nothing to apply.");
859+
} else {
860+
match &vex_result {
861+
Some(ManifestlessVex::Written(summary)) if !args.common.silent => println!(
862+
"{}",
863+
crate::commands::vex::format_vex_written(summary.statements, vex_path())
864+
),
865+
// Errors print even under --silent (same as the main path).
866+
Some(ManifestlessVex::Failed(e)) => e.print_embedded(&args.common),
867+
Some(ManifestlessVex::NothingToAttest(_)) if !args.common.silent => {
868+
println!("{}", crate::commands::vex::format_vex_nothing_to_attest())
869+
}
870+
None if !args.common.silent && args.common.dry_run && args.vex.vex.is_some() => {
871+
println!(
872+
"{}",
873+
crate::commands::vex::format_vex_dry_run_skip("applied")
874+
);
875+
}
876+
_ => {}
877+
}
804878
}
805-
return 0;
879+
return i32::from(matches!(vex_result, Some(ManifestlessVex::Failed(_))));
806880
}
807881

808882
// Read-only Go `replace`-redirect verification for CI / GitHub-App auditing.
@@ -1094,6 +1168,7 @@ pub(crate) async fn run_locked(
10941168
});
10951169
}
10961170
Some(Err(e)) => {
1171+
env.warnings.extend(e.embedded_warnings());
10971172
env.mark_error(EnvelopeError::new(e.code, e.message.clone()));
10981173
}
10991174
None => {}
@@ -1119,12 +1194,7 @@ pub(crate) async fn run_locked(
11191194
);
11201195
}
11211196
}
1122-
Some(Err(e)) => {
1123-
// Errors print even under --silent ("errors only",
1124-
// never "nothing"): exit 1 with no message would be
1125-
// undiagnosable.
1126-
eprintln!("Error: VEX generation failed: {}", e.message);
1127-
}
1197+
Some(Err(e)) => e.print_embedded(&args.common),
11281198
None => {
11291199
// Only a dry run that itself succeeded skips VEX
11301200
// *because* of --dry-run; a failed one would have

‎crates/socket-patch-cli/src/commands/list.rs‎

Lines changed: 97 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ enum Source {
3636
/// `scan`/`get --mode vendored` patch lives ONLY in
3737
/// `.socket/vendor/state.json`, as a `detached` entry's embedded record
3838
/// (the hosted rule again — a vendored-only project lists and exits 0).
39+
/// A standalone `vendor` entry's fallback copy lists here too once no
40+
/// manifest entry covers it — the checkout `vex` attests from it.
3941
Vendored,
4042
}
4143

@@ -62,10 +64,14 @@ struct ListEntry<'a> {
6264
/// PURL, then manifest < hosted < vendored when one purl appears in more
6365
/// than one. The record maps (`HashMap` manifest and vendor ledger /
6466
/// `BTreeMap` redirect ledger) never impose an order shared consumers could
65-
/// diff, so the sort here is the contract. Only vendor entries that carry
66-
/// an embedded record fold in — a legacy manifest-tracked entry has no
67-
/// record of its own (the manifest's IS the record) and would otherwise
68-
/// double-list its purl.
67+
/// diff, so the sort here is the contract. Only vendor entries whose
68+
/// embedded record stands on its own fold in
69+
/// ([`crate::commands::vendor_record_is_unowned`], the rule `vex` attests
70+
/// by): a `detached` entry always (coexisting with a manifest entry is real
71+
/// state, shown labeled apart), a standalone `vendor` entry's fallback copy
72+
/// only when the manifest does not cover it — while it does, the manifest's
73+
/// record IS that entry's record and listing the copy would double-list the
74+
/// purl. A legacy entry with no embedded record never folds in.
6975
fn combined_entries<'a>(
7076
manifest: Option<&'a PatchManifest>,
7177
redirect: Option<&'a RedirectState>,
@@ -88,7 +94,10 @@ fn combined_entries<'a>(
8894
}
8995
if let Some(vendor) = vendor {
9096
entries.extend(vendor.iter().filter_map(|(purl, entry)| {
91-
let record = entry.record.as_ref().filter(|_| entry.detached)?;
97+
let record = entry
98+
.record
99+
.as_ref()
100+
.filter(|_| crate::commands::vendor_record_is_unowned(purl, entry, manifest))?;
92101
Some(ListEntry {
93102
purl,
94103
record,
@@ -817,6 +826,89 @@ mod tests {
817826
assert_eq!(v["summary"]["discovered"], 2, "{v}");
818827
}
819828

829+
/// The ledger entry the manifest-driven standalone `vendor` writes: NOT
830+
/// detached, with the record embedded as a fallback copy.
831+
fn standalone_vendor_entry(base_purl: &str, record: PatchRecord) -> VendorEntry {
832+
let mut entry = vendor_entry(base_purl, Some(record));
833+
entry.detached = false;
834+
entry
835+
}
836+
837+
/// A standalone `vendor` entry's embedded fallback record lists exactly
838+
/// when `vex` would attest from it — no manifest entry covers the entry
839+
/// (by ledger key or base purl). With no manifest at all it lists
840+
/// labeled `vendored`, so a manifest-less checkout never reads "no
841+
/// patches" while its VEX document attests the patch; while the manifest
842+
/// covers it, the manifest's record IS its record (no double listing).
843+
#[test]
844+
fn standalone_vendor_fallback_record_lists_only_when_the_manifest_does_not_cover_it() {
845+
let manifest = sample_manifest();
846+
let record = manifest.patches["pkg:npm/minimist@1.2.2"].clone();
847+
let mut other = record.clone();
848+
other.uuid = "55555555-5555-4555-8555-555555555555".to_string();
849+
let mut vendor = HashMap::new();
850+
// Covered by the manifest's exact key.
851+
vendor.insert(
852+
"pkg:npm/minimist@1.2.2".to_string(),
853+
standalone_vendor_entry("pkg:npm/minimist@1.2.2", record.clone()),
854+
);
855+
// Covered through its base purl (a qualified ledger key).
856+
vendor.insert(
857+
"pkg:npm/minimist@1.2.2?variant=x".to_string(),
858+
standalone_vendor_entry("pkg:npm/minimist@1.2.2", record.clone()),
859+
);
860+
// Dropped from the manifest while the ledger still holds it.
861+
vendor.insert(
862+
"pkg:npm/left-pad@1.3.0".to_string(),
863+
standalone_vendor_entry("pkg:npm/left-pad@1.3.0", other),
864+
);
865+
866+
let listed = |entries: &[ListEntry<'_>]| -> Vec<(String, String, String)> {
867+
let env = build_list_envelope(entries);
868+
let v: serde_json::Value = serde_json::from_str(&env.to_pretty_json()).unwrap();
869+
v["events"]
870+
.as_array()
871+
.unwrap()
872+
.iter()
873+
.map(|e| {
874+
(
875+
e["purl"].as_str().unwrap().to_string(),
876+
e["details"]["mode"]
877+
.as_str()
878+
.unwrap_or("manifest")
879+
.to_string(),
880+
e["uuid"].as_str().unwrap().to_string(),
881+
)
882+
})
883+
.collect()
884+
};
885+
886+
assert_eq!(
887+
listed(&combined_entries(Some(&manifest), None, Some(&vendor))),
888+
vec![
889+
(
890+
"pkg:npm/left-pad@1.3.0".to_string(),
891+
"vendored".to_string(),
892+
"55555555-5555-4555-8555-555555555555".to_string(),
893+
),
894+
(
895+
"pkg:npm/minimist@1.2.2".to_string(),
896+
"manifest".to_string(),
897+
"11111111-1111-4111-8111-111111111111".to_string(),
898+
),
899+
],
900+
"covered fallback copies stay behind the manifest record; the uncovered one lists"
901+
);
902+
903+
// No manifest at all: every fallback copy stands on its own.
904+
let only = listed(&combined_entries(None, None, Some(&vendor)));
905+
assert_eq!(only.len(), 3, "{only:?}");
906+
assert!(
907+
only.iter().all(|(_, mode, _)| mode == "vendored"),
908+
"{only:?}"
909+
);
910+
}
911+
820912
#[test]
821913
fn ordering_is_deterministic_across_builds() {
822914
// Two independent builds of the same manifest must be byte-identical.

0 commit comments

Comments
 (0)