From 7b7e1199fabc88fac397bb62b9e503735d3c0c36 Mon Sep 17 00:00:00 2001 From: antony Date: Sat, 17 Oct 2026 08:46:50 +1000 Subject: [PATCH 1/3] fix(fix-target): never recommend a yanked or deprecated release (FX-H5) lodash 4.17.21 was told "update to >= 4.18.0"; npm marks 4.18.0 deprecated ("Bad release"), 4.18.1 is the release to take. A yanked crate target would make `cargo update --precise` fail outright. - fourda-depengine: new `registry::RegistryReleases {published, excluded}` input. The walk skips excluded releases exactly as it skips pre-releases, can step to any published release, and reports the withdrawn releases it passed over (`WalkOutcome.withheld`). A window whose every exit is withdrawn has no way out: no target, honestly named. `installable_fix` / `installable_line_fix` apply the same rule to line fixes and listed fixes. `match_inventory_with` takes a `RegistryIndex`; `MatchedDependency.withheld` carries the names. JSON API ops accept an optional `registry`. Crate stays pure (wasm: 0 imports). - app: `osv::release_facts` reads crates.io `yanked`, PyPI `yanked` (all files) and npm `deprecated` (abbreviated packument; not when `latest` is deprecated too) for packages with a confirmed advisory, after each OSV sync, cached in kv_store (24 h fresh, 30 d max). Go `retract` is not covered. Offline / never fetched: advisory-data-only, as before. - Upgrade Plan: when every fixing release is withdrawn the row reads "No installable fix" and names the withdrawn release, never "No fix published". The scorer's security verdict fix goes through the same rule. - Vectors: ft-007 now 4.18.1 (excluded 4.18.0); new ft-062 (h2 0.3.17 yanked -> 0.3.18), ft-063 (brace-expansion 3.0.3 deprecated -> 3.0.4), ft-064 (all clearing releases withdrawn -> no target, withheld named). Optional `published`/`excluded`/`withheld` fields documented in the vectors README. NETWORK.md lists the new registry reads. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7 --- NETWORK.md | 13 +- .../fourda-depengine/scripts/check-wasm.cjs | 13 +- crates/fourda-depengine/src/api.rs | 80 ++++- crates/fourda-depengine/src/fix_target.rs | 258 +++++++++++++-- .../fourda-depengine/src/fix_target_tests.rs | 128 ++++++++ crates/fourda-depengine/src/lib.rs | 5 +- crates/fourda-depengine/src/lines.rs | 1 + crates/fourda-depengine/src/matching.rs | 94 ++++-- crates/fourda-depengine/src/matching_tests.rs | 81 ++++- crates/fourda-depengine/src/registry.rs | 87 +++++ src-tauri/src/evidence/upgrade_plan.rs | 31 +- src-tauri/src/evidence/upgrade_plan_tests.rs | 62 ++++ src-tauri/src/osv/fix_target_tests.rs | 1 + src-tauri/src/osv/identity.rs | 1 + src-tauri/src/osv/matching.rs | 4 +- src-tauri/src/osv/mod.rs | 1 + src-tauri/src/osv/reachability_tests.rs | 1 + src-tauri/src/osv/release_facts.rs | 309 ++++++++++++++++++ src-tauri/src/osv/release_facts_tests.rs | 96 ++++++ src-tauri/src/osv/sync.rs | 5 + src-tauri/src/osv/vectors_tests.rs | 45 ++- src-tauri/src/preemption.rs | 4 + src-tauri/src/scoring/security_verdict.rs | 29 +- src-tauri/tests/conformance/vectors/README.md | 24 +- .../conformance/vectors/fix_target.jsonl | 5 +- 25 files changed, 1294 insertions(+), 84 deletions(-) create mode 100644 crates/fourda-depengine/src/fix_target_tests.rs create mode 100644 crates/fourda-depengine/src/registry.rs create mode 100644 src-tauri/src/osv/release_facts.rs create mode 100644 src-tauri/src/osv/release_facts_tests.rs diff --git a/NETWORK.md b/NETWORK.md index 21978c443..ad044ce9a 100644 --- a/NETWORK.md +++ b/NETWORK.md @@ -282,7 +282,17 @@ module proxy (§1b, §1e) receive them too; §4 has the complete list. |---|---|---| | `api.osv.dev` | `POST /v1/querybatch` | Your discovered package names + ecosystems, batched (≤1000/batch) | | `osv-vulnerabilities.storage.googleapis.com` | `GET /{ecosystem}/all.zip` | Plain `GET` — downloads the public advisory mirror (no user data) | - +| `registry.npmjs.org` | `GET /{package}` (abbreviated packument, `Accept: application/vnd.npm.install-v1+json`) | The package name only | +| `crates.io` | `GET /api/v1/crates/{crate}` | The crate name only. User-Agent `4DA-Developer-OS/1.0 (https://4da.ai)`, 1.1 s apart | +| `pypi.org` | `GET /pypi/{package}/json` | The package name only | + +- **Registry release check** (the last three rows, `src-tauri/src/osv/release_facts.rs`): after each + sync, for packages that have a **version-confirmed advisory** only, 4DA reads which releases exist + and which the registry withdrew (crates.io and PyPI *yanked*, npm *deprecated*), so a fix target + never names one. At most 60 packages per sync, each re-read at most once a day; the answer is + kept in the local database. **Your installed version is never sent.** Go modules are not + checked (their `retract` directives are not read). Offline, targets come from the advisory data + alone, as before. - **Trigger:** background, at startup, **only if dependencies exist** and the local mirror was last synced **> 6 hours ago** (`app_setup.rs`). - **Disable:** if 4DA has discovered no dependencies, nothing is sent. Removing/disabling ACE project @@ -525,6 +535,7 @@ so it is fetched **once, on demand** (`src-tauri/src/embeddings_providers/model_ | `crates.io` | your crate names | as above | | `static.crates.io` / `registry.npmjs.org` (§1e) | the crate / package name and the new public version of a graded release — never your installed version | the release card's "what changed" | | `api.github.com/repos/{owner}/{repo}/releases` (§2m) — **only with `github_release_notes = true`** (off by default) | the GitHub repository of a dependency whose package shipped no usable changelog for a graded release | the release card's "what changed" | + | `registry.npmjs.org` / `crates.io` / `pypi.org` registry release check (§2c) | the name of each package with a version-confirmed advisory — name only | no fix target names a yanked or deprecated release | | `registry.npmjs.org` full packument (MCP `dependency_check`, §2j) | the npm package name being checked — name only | release vetting | | `crates.io/api/v1/crates/{crate}/versions` (MCP `dependency_check`, §2j) | the crate name being checked — name only | release vetting | | `proxy.golang.org` | your **full Go module paths** | as above | diff --git a/crates/fourda-depengine/scripts/check-wasm.cjs b/crates/fourda-depengine/scripts/check-wasm.cjs index 9229b738e..1cdcd3d44 100644 --- a/crates/fourda-depengine/scripts/check-wasm.cjs +++ b/crates/fourda-depengine/scripts/check-wasm.cjs @@ -69,7 +69,18 @@ const checks = [ [{ op: 'affected', eco: 'npm', version: '1.2.3', ranges }, { affected: true, confirmed: true }], [ { op: 'fix_target', eco: 'npm', installed: '1.2.3', ranges: [ranges] }, - { line_fixes: ['1.2.4'], walk: { version: '1.2.4', clears_all: true } }, + { line_fixes: ['1.2.4'], walk: { version: '1.2.4', clears_all: true }, withheld: [] }, + ], + // The registry withdrew the fix (yanked / deprecated): the next release it offers. + [ + { + op: 'fix_target', + eco: 'npm', + installed: '1.2.3', + ranges: [ranges], + registry: { published: ['1.2.4', '1.2.5'], excluded: ['1.2.4'] }, + }, + { line_fixes: ['1.2.5'], walk: { version: '1.2.5', clears_all: true }, withheld: ['1.2.4'] }, ], ]; for (const [request, want] of checks) { diff --git a/crates/fourda-depengine/src/api.rs b/crates/fourda-depengine/src/api.rs index e2b7e2701..f89b80c8c 100644 --- a/crates/fourda-depengine/src/api.rs +++ b/crates/fourda-depengine/src/api.rs @@ -12,8 +12,10 @@ //! enumerated `versions` as an `ENUMERATED` range): //! - `{"op":"version_cmp","eco":"PyPI","a":"1.0.post1","b":"1.0"}` → `"<" | "=" | ">" | null` //! - `{"op":"affected","eco":"npm","version":"1.2.3","ranges":[...]}` → `{"affected":bool,"confirmed":bool}` -//! - `{"op":"fix_target","eco":"npm","installed":"1.2.3","ranges":[[...],[...]]}` -//! → `{"line_fixes":[..],"walk":{"version":"..","clears_all":bool}|null}` +//! - `{"op":"fix_target","eco":"npm","installed":"1.2.3","ranges":[[...],[...]], +//! "registry":{"published":[..],"excluded":[..]}}` (`registry` optional: the +//! package's releases and the ones its registry withdrew, `RegistryReleases`) +//! → `{"line_fixes":[..],"walk":{"version":"..","clears_all":bool}|null,"withheld":[..]}` //! - `{"op":"match_inventory","inventory":[Package..],"osv":[verbatim OSV records], //! "records":[AdvisoryRecord..]}` → `{"records":n,"findings":[{"advisory_id", //! "package_name","ecosystem","installed_version","fixed_version", @@ -25,17 +27,20 @@ //! (`ON CONFLICT DO UPDATE`), so there the LAST write of a row wins; give //! the current rows as `records` to reproduce a mirror state. //! Every `inventory` row must say `primary`; copies with no primary row -//! are an error, not an empty result. +//! are an error, not an empty result. Optional `registry`: +//! `[{"ecosystem","name","published":[..],"excluded":[..]}]`. use std::cmp::Ordering; use serde::Deserialize; use serde_json::{json, Value}; -use crate::fix_target::{fix_for_version, walk_target}; -use crate::matching::{inventory_problem, match_inventory, Package}; +use crate::ecosystem::normalize as normalize_ecosystem; +use crate::fix_target::{installable_line_fix, walk_target_in}; +use crate::matching::{inventory_problem, match_inventory_with, package_key, Package}; use crate::osv::{stored_records, AdvisoryRecord, OsvRecord}; use crate::range::check_version_affected; +use crate::registry::{RegistryIndex, RegistryReleases}; use crate::version_order::OrderedVersion; /// The schema every response carries. @@ -58,6 +63,8 @@ enum Request { eco: String, installed: String, ranges: Vec>, + #[serde(default)] + registry: RegistryReleases, }, MatchInventory { inventory: Vec, @@ -65,9 +72,32 @@ enum Request { records: Vec, #[serde(default)] osv: Vec, + #[serde(default)] + registry: Vec, }, } +/// One package's registry facts in a `match_inventory` request. +#[derive(Deserialize)] +struct PackageReleases { + ecosystem: String, + name: String, + #[serde(flatten)] + releases: RegistryReleases, +} + +fn registry_index(rows: Vec) -> RegistryIndex { + rows.into_iter() + .map(|r| { + let key = ( + package_key(&r.name, &r.ecosystem), + normalize_ecosystem(&r.ecosystem).to_string(), + ); + (key, r.releases) + }) + .collect() +} + /// The `match_inventory` op: stored rows plus verbatim records, matched. An /// inventory with copies but no primary rows is an error, never an empty /// all-clear ([`inventory_problem`]). @@ -75,6 +105,7 @@ fn match_op( inventory: &[Package], mut records: Vec, osv: &[OsvRecord], + registry: &RegistryIndex, ) -> Result { if let Some(problem) = inventory_problem(inventory) { return Err(problem); @@ -93,7 +124,7 @@ fn match_op( records.push(row); } } - let findings: Vec = match_inventory(inventory, &records) + let findings: Vec = match_inventory_with(inventory, &records, registry, |_, _, _| {}) .into_iter() .map(|f| { let r = &records[f.record]; @@ -143,22 +174,26 @@ fn answer(request: Request) -> Result { eco, installed, ranges, + registry, } => { let stored: Vec> = ranges.iter().map(|r| stored(r.as_ref())).collect(); let line_fixes: Vec = stored .iter() - .filter_map(|r| fix_for_version(&eco, &installed, r)) + .filter_map(|r| installable_line_fix(&eco, &installed, r, ®istry).fix) .collect(); let refs: Vec<&Option> = stored.iter().collect(); - let walk = walk_target(&eco, &installed, &refs) + let outcome = walk_target_in(&eco, &installed, &refs, ®istry); + let walk = outcome + .target .map(|w| json!({ "version": w.version, "clears_all": w.clears_all })); - json!({ "line_fixes": line_fixes, "walk": walk }) + json!({ "line_fixes": line_fixes, "walk": walk, "withheld": outcome.withheld }) } Request::MatchInventory { inventory, records, osv, - } => match_op(&inventory, records, &osv)?, + registry, + } => match_op(&inventory, records, &osv, ®istry_index(registry))?, }) } @@ -208,8 +243,31 @@ mod tests { )); assert_eq!( r, - json!({"line_fixes": ["1.2.4"], "walk": {"version": "1.2.4", "clears_all": true}}) + json!({"line_fixes": ["1.2.4"], "walk": {"version": "1.2.4", "clears_all": true}, "withheld": []}) + ); + // The registry withdrew 1.2.4: the next release it offers is the fix. + let r = result(&format!( + r#"{{"op":"fix_target","eco":"npm","installed":"1.2.3","ranges":[{ranges}], + "registry":{{"published":["1.2.3","1.2.4","1.2.5"],"excluded":["1.2.4"]}}}}"# + )); + assert_eq!( + r, + json!({"line_fixes": ["1.2.5"], "walk": {"version": "1.2.5", "clears_all": true}, "withheld": ["1.2.4"]}) + ); + } + + #[test] + fn match_inventory_reads_registry_facts_per_package() { + let r = result( + r#"{"op":"match_inventory", + "inventory":[{"ecosystem":"rust","project":"/p","name":"h2","version":"0.3.16","primary":true}], + "osv":[{"id":"RUSTSEC-2023-0034","affected":[{"package":{"name":"h2","ecosystem":"crates.io"}, + "ranges":[{"type":"SEMVER","events":[{"introduced":"0"},{"fixed":"0.3.17"}]}]}]}], + "registry":[{"ecosystem":"crates.io","name":"H2","published":["0.3.16","0.3.17","0.3.18"],"excluded":["0.3.17"]}]}"#, ); + let copy = &r["findings"][0]["dependency_instances"][0]; + assert_eq!(copy["target_version"], "0.3.18"); + assert_eq!(copy["withheld"], json!(["0.3.17"])); } #[test] diff --git a/crates/fourda-depengine/src/fix_target.rs b/crates/fourda-depengine/src/fix_target.rs index 02a866a37..6c94f8a9e 100644 --- a/crates/fourda-depengine/src/fix_target.rs +++ b/crates/fourda-depengine/src/fix_target.rs @@ -18,6 +18,7 @@ use std::cmp::Ordering; use crate::range::{ check_version_affected, is_unknown_bound, parse_version, Range, ENUMERATED_RANGE, }; +use crate::registry::{is_listed, RegistryReleases}; use crate::version_order::{in_window, Lower, OrderedVersion, Upper}; /// One affected window of a SEMVER/ECOSYSTEM range: from `lower` up to @@ -116,12 +117,109 @@ pub struct WalkTarget { pub clears_all: bool, } +/// Where [`walk_target_in`] lands, and what it had to step over. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct WalkOutcome { + /// As [`walk_target`]; never a release the registry withdrew. + pub target: Option, + /// Withdrawn releases (yanked, deprecated) a step would otherwise have + /// named. With no target, a non-empty list means a fix WAS published and + /// then withdrawn, so the honest answer is "no installable fix", not "no + /// fix". Empty when the walk could not evaluate an advisory. + pub withheld: Vec, +} + +/// A line fix as a release the registry still offers ([`installable_fix`]). +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct LineFix { + pub fix: Option, + /// The withdrawn releases passed over to reach `fix` (see + /// [`WalkOutcome::withheld`]). + pub withheld: Vec, +} + +/// The releases the walk can name, and the ones it must not. +/// +/// `releases`: every stable release the advisory data names (every bound and +/// every enumerated version) plus every release the registry lists, when the +/// host supplied that list. Without it these are the only versions above a +/// `last_affected` bound the walk can name; a release the data never mentions +/// can only make the target higher than the true minimum, never vulnerable. +/// +/// `excluded`: releases the registry withdrew. They stay in `releases` so a +/// step can tell it passed one, but no step lands on them, exactly as no step +/// lands on a pre-release. +struct Known { + releases: Vec, + excluded: Vec, +} + +impl Known { + fn new( + ecosystem: &str, + advisories: &[Option>], + registry: &RegistryReleases, + ) -> Self { + let mut releases = advisory_releases(ecosystem, advisories); + releases.extend(registry.published.iter().filter_map(|v| { + OrderedVersion::parse_in(v, ecosystem) + .filter(|p| !p.is_prerelease()) + .map(|p| (p, v.trim().to_string())) + })); + Known { + releases, + excluded: registry.excluded_in(ecosystem), + } + } + + fn allowed(&self, version: &OrderedVersion) -> bool { + !is_listed(&self.excluded, version) + } + + /// The lowest release above `bound` that `keep` accepts and the registry + /// has not withdrawn. Every withdrawn release `keep` accepts below it + /// (all of them, when there is none) goes into `withheld`. + fn next_above( + &self, + bound: &OrderedVersion, + keep: impl Fn(&OrderedVersion) -> bool, + withheld: &mut Vec, + ) -> Option { + let next = first_above(&self.releases, bound, |r| keep(r) && self.allowed(r)); + for (version, raw) in &self.releases { + let passed = version.compare(bound) == Some(Ordering::Greater) + && keep(version) + && !self.allowed(version) + && next + .as_ref() + .is_none_or(|(n, _)| version.compare(n) == Some(Ordering::Less)); + if passed { + withhold(withheld, raw); + } + } + next + } + + /// A `fixed` bound as a step: the bound itself, unless the registry + /// withdrew it; then the lowest allowed release above it. + fn fixed_step(&self, fix: Release, withheld: &mut Vec) -> Option { + if self.allowed(&fix.0) { + return Some(fix); + } + withhold(withheld, &fix.1); + self.next_above(&fix.0, |_| true, withheld) + } +} + +fn withhold(withheld: &mut Vec, raw: &str) { + if !withheld.iter().any(|w| w == raw) { + withheld.push(raw.to_string()); + } +} + /// The releases the advisory data names for this package (every bound and -/// every enumerated version, pre-releases left out): the only versions above -/// a `last_affected` bound the walk can name without a registry. Offline by -/// design; a release the data never mentions can only make the target higher -/// than the true minimum, never vulnerable. -fn known_releases(ecosystem: &str, advisories: &[Option>]) -> Vec { +/// every enumerated version, pre-releases left out). +fn advisory_releases(ecosystem: &str, advisories: &[Option>]) -> Vec { let mut out: Vec = Vec::new(); for range in advisories.iter().flatten().flatten() { for event in range.events.iter().flatten() { @@ -200,34 +298,57 @@ enum Exit { Stuck, } +/// The versions one enumerated `versions` list names. +fn enumerated(ecosystem: &str, range: &Range) -> Vec { + range + .events + .iter() + .flatten() + .filter_map(|v| v.as_str()) + .filter_map(|v| OrderedVersion::parse_in(v, ecosystem)) + .collect() +} + +/// Whether one advisory's `ranges` hold `version` (a readable window, or an +/// enumerated list naming it). +fn affects(ecosystem: &str, ranges: &[Range], version: &OrderedVersion) -> bool { + windows(ecosystem, ranges) + .iter() + .any(|w| w.contains(version) == Some(true)) + || ranges + .iter() + .filter(|r| r.range_type == ENUMERATED_RANGE) + .any(|r| is_listed(&enumerated(ecosystem, r), version)) +} + /// The exit from every window of one advisory that holds `here`: a `fixed` /// bound; above a `last_affected` bound, the first known release past it; for /// an enumerated `versions` list, the first known release past `here` the list -/// does not name. The highest exit wins, since every window must be left. -fn exit(ecosystem: &str, here: &OrderedVersion, ranges: &[Range], releases: &[Release]) -> Exit { +/// does not name. A release the registry withdrew is never an exit: the next +/// allowed one is, and the withdrawn one goes into `withheld`. The highest +/// exit wins, since every window must be left. +fn exit( + ecosystem: &str, + here: &OrderedVersion, + ranges: &[Range], + known: &Known, + withheld: &mut Vec, +) -> Exit { let mut exits: Vec> = Vec::new(); for window in windows(ecosystem, ranges) { if window.contains(here) != Some(true) { continue; } exits.push(match window.end { - Some((fix, raw, true)) => Some((fix, raw)), - Some((last, _, false)) => first_above(releases, &last, |_| true), + Some((fix, raw, true)) => known.fixed_step((fix, raw), withheld), + Some((last, _, false)) => known.next_above(&last, |_| true, withheld), None => None, }); } for range in ranges.iter().filter(|r| r.range_type == ENUMERATED_RANGE) { - let listed: Vec = range - .events - .iter() - .flatten() - .filter_map(|v| v.as_str()) - .filter_map(|v| OrderedVersion::parse_in(v, ecosystem)) - .collect(); - let named = - |v: &OrderedVersion| listed.iter().any(|l| v.compare(l) == Some(Ordering::Equal)); - if named(here) { - exits.push(first_above(releases, here, |r| !named(r))); + let listed = enumerated(ecosystem, range); + if is_listed(&listed, here) { + exits.push(known.next_above(here, |r| !is_listed(&listed, r), withheld)); } } let mut best: Option = None; @@ -268,34 +389,67 @@ fn exit(ecosystem: &str, here: &OrderedVersion, ranges: &[Range], releases: &[Re /// nothing with a way out does, so there is nowhere to go), or an advisory /// cannot be evaluated against a version on the path (no range data, /// unparseable): no target is then a claim 4DA can stand behind. +/// +/// No registry facts: [`walk_target_in`] with none. pub fn walk_target( ecosystem: &str, installed: &str, ranges: &[&Option], ) -> Option { + walk_target_in(ecosystem, installed, ranges, &RegistryReleases::default()).target +} + +/// [`walk_target`] with what the registry says about the package's releases: +/// every release it lists can be a step, and none it withdrew (yanked, +/// deprecated) can. A window whose every exit is withdrawn has no way out, +/// like an open-ended one, and the outcome names the withdrawn releases. +pub fn walk_target_in( + ecosystem: &str, + installed: &str, + ranges: &[&Option], + registry: &RegistryReleases, +) -> WalkOutcome { + let mut withheld = Vec::new(); + match walk(ecosystem, installed, ranges, registry, &mut withheld) { + Ok(target) => WalkOutcome { target, withheld }, + Err(Unreadable) => WalkOutcome::default(), + } +} + +/// The walk met an advisory it cannot evaluate on its path. +struct Unreadable; + +fn walk( + ecosystem: &str, + installed: &str, + ranges: &[&Option], + registry: &RegistryReleases, + withheld: &mut Vec, +) -> Result, Unreadable> { let parsed: Vec>> = ranges .iter() .map(|r| r.as_deref().and_then(|j| serde_json::from_str(j).ok())) .collect(); - let releases = known_releases(ecosystem, &parsed); + let known = Known::new(ecosystem, &parsed, registry); let start = installed.trim(); let mut current = start.to_string(); for _ in 0..MAX_HOPS { - let here = OrderedVersion::parse_in(¤t, ecosystem)?; + let here = OrderedVersion::parse_in(¤t, ecosystem).ok_or(Unreadable)?; let mut next: Option = None; let mut stuck = false; for (json, advisory) in ranges.iter().zip(&parsed) { match check_version_affected(ecosystem, Some(¤t), json) { (false, _) => continue, - (true, false) => return None, + (true, false) => return Err(Unreadable), (true, true) => {} } - match exit(ecosystem, &here, advisory.as_deref()?, &releases) { + let advisory = advisory.as_deref().ok_or(Unreadable)?; + match exit(ecosystem, &here, advisory, &known, withheld) { Exit::Stuck => stuck = true, Exit::To(step) => { let replace = match next.as_ref() { None => true, - Some(best) => outranks(&step, best)?, + Some(best) => outranks(&step, best).ok_or(Unreadable)?, }; if replace { next = Some(step); @@ -306,14 +460,58 @@ pub fn walk_target( match next { Some((_, raw)) => current = raw, None => { - return (current != start).then_some(WalkTarget { + return Ok((current != start).then_some(WalkTarget { version: current, clears_all: !stuck, - }) + })) } } } - None + Err(Unreadable) +} + +/// A fix bound (a line fix, or an advisory's listed fix) as a release the +/// registry still offers: `fix` itself, or, when the registry withdrew it, +/// the lowest allowed release above it that this advisory (`ranges_json`) +/// does not affect. `fix: None` with `withheld` set: the fix was published +/// and withdrawn, and no installable release past it is known. +pub fn installable_fix( + ecosystem: &str, + fix: &str, + ranges_json: &Option, + registry: &RegistryReleases, +) -> LineFix { + let bound = OrderedVersion::parse_in(fix, ecosystem); + let Some(bound) = bound.filter(|b| is_listed(®istry.excluded_in(ecosystem), b)) else { + return LineFix { + fix: Some(fix.to_string()), + withheld: Vec::new(), + }; + }; + let parsed: [Option>; 1] = [ranges_json + .as_deref() + .and_then(|j| serde_json::from_str(j).ok())]; + let ranges: &[Range] = parsed[0].as_deref().unwrap_or_default(); + let known = Known::new(ecosystem, &parsed, registry); + let mut withheld = vec![fix.trim().to_string()]; + let next = known.next_above(&bound, |v| !affects(ecosystem, ranges, v), &mut withheld); + LineFix { + fix: next.map(|(_, raw)| raw), + withheld, + } +} + +/// [`fix_for_version`] through [`installable_fix`]. `None` fix and no +/// `withheld`: the line has no fix at all. +pub fn installable_line_fix( + ecosystem: &str, + user_version: &str, + affected_ranges_json: &Option, + registry: &RegistryReleases, +) -> LineFix { + fix_for_version(ecosystem, user_version, affected_ranges_json) + .map(|fix| installable_fix(ecosystem, &fix, affected_ranges_json, registry)) + .unwrap_or_default() } /// The lowest version reachable from `installed` that NO known advisory @@ -359,3 +557,7 @@ pub fn upgrade_type(installed: &str, target: &str) -> Option { UpgradeType::Patch }) } + +#[cfg(test)] +#[path = "fix_target_tests.rs"] +mod tests; diff --git a/crates/fourda-depengine/src/fix_target_tests.rs b/crates/fourda-depengine/src/fix_target_tests.rs new file mode 100644 index 000000000..0da2c1bc3 --- /dev/null +++ b/crates/fourda-depengine/src/fix_target_tests.rs @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: FSL-1.1-Apache-2.0 +//! Withdrawn releases in the fix-target walk (FX-H5). The app's +//! `osv::fix_target_tests` and the shared rule vectors cover the rest. + +use super::*; + +/// One advisory's stored ranges JSON (`Some`, as the mirror stores it). +#[allow(clippy::unnecessary_wraps)] +fn ranges(events: &str) -> Option { + Some(format!(r#"[{{"type":"SEMVER","events":{events}}}]"#)) +} + +fn facts(published: &[&str], excluded: &[&str]) -> RegistryReleases { + RegistryReleases { + published: published.iter().map(ToString::to_string).collect(), + excluded: excluded.iter().map(ToString::to_string).collect(), + } +} + +fn walk_in(installed: &str, all: &[Option], registry: &RegistryReleases) -> WalkOutcome { + let refs: Vec<&Option> = all.iter().collect(); + walk_target_in("crates.io", installed, &refs, registry) +} + +/// h2 0.3.17 fixed RUSTSEC-2023-0034 and was yanked; 0.3.18 is the release. +#[test] +fn a_yanked_fix_steps_to_the_next_release_the_registry_offers() { + let h2 = [ranges(r#"[{"introduced":"0"},{"fixed":"0.3.17"}]"#)]; + let none = walk_in("0.3.16", &h2, &RegistryReleases::default()); + assert_eq!(none.target.map(|t| t.version).as_deref(), Some("0.3.17")); + + let registry = facts(&["0.3.16", "0.3.17", "0.3.18", "0.3.19"], &["0.3.17"]); + let out = walk_in("0.3.16", &h2, ®istry); + assert_eq!( + out.target, + Some(WalkTarget { + version: "0.3.18".into(), + clears_all: true + }) + ); + assert_eq!(out.withheld, ["0.3.17"]); +} + +#[test] +fn every_withdrawn_release_on_the_way_is_named_and_skipped() { + let a = [ranges(r#"[{"introduced":"0"},{"fixed":"1.0.1"}]"#)]; + let registry = facts(&["1.0.0", "1.0.1", "1.0.2", "1.0.3"], &["1.0.1", "1.0.2"]); + let out = walk_in("1.0.0", &a, ®istry); + assert_eq!(out.target.map(|t| t.version).as_deref(), Some("1.0.3")); + assert_eq!(out.withheld, ["1.0.1", "1.0.2"]); +} + +#[test] +fn when_every_clearing_release_is_withdrawn_there_is_no_target_and_it_says_why() { + let a = [ranges(r#"[{"introduced":"0"},{"fixed":"1.0.1"}]"#)]; + let out = walk_in("1.0.0", &a, &facts(&["1.0.0", "1.0.1"], &["1.0.1"])); + assert_eq!(out.target, None); + assert_eq!(out.withheld, ["1.0.1"]); + + // Another advisory still has a way out: the walk takes it, and does not + // claim to clear the one whose only fix was withdrawn. + let both = [ + ranges(r#"[{"introduced":"0"},{"fixed":"2.0.0"}]"#), + ranges(r#"[{"introduced":"0"},{"fixed":"1.1.0"}]"#), + ]; + let registry = facts(&["1.0.0", "1.1.0", "2.0.0"], &["2.0.0"]); + let out = walk_in("1.0.0", &both, ®istry); + assert_eq!( + out.target, + Some(WalkTarget { + version: "1.1.0".into(), + clears_all: false + }) + ); + assert_eq!(out.withheld, ["2.0.0"]); +} + +#[test] +fn a_withdrawn_release_above_a_last_affected_bound_is_passed_over() { + let a = [ranges(r#"[{"introduced":"0"},{"last_affected":"2.0.0"}]"#)]; + let registry = facts(&["2.0.0", "2.0.1", "2.0.2"], &["2.0.1"]); + let out = walk_in("1.5.0", &a, ®istry); + assert_eq!(out.target.map(|t| t.version).as_deref(), Some("2.0.2")); + assert_eq!(out.withheld, ["2.0.1"]); +} + +#[test] +fn published_releases_let_the_walk_name_one_no_advisory_mentions() { + // Without the registry list nothing above the last_affected bound is + // known; with it, the first release past it is. + let a = [ranges(r#"[{"introduced":"0"},{"last_affected":"2.0.0"}]"#)]; + assert_eq!( + walk_in("1.5.0", &a, &RegistryReleases::default()).target, + None + ); + let out = walk_in("1.5.0", &a, &facts(&["2.0.0", "2.1.0-rc.1", "2.1.0"], &[])); + assert_eq!(out.target.map(|t| t.version).as_deref(), Some("2.1.0")); + assert!(out.withheld.is_empty()); +} + +#[test] +fn an_unreadable_advisory_withholds_nothing() { + let all = [ranges(r#"[{"introduced":"0"},{"fixed":"1.0.1"}]"#), None]; + let out = walk_in("1.0.0", &all, &facts(&["1.0.1", "1.0.2"], &["1.0.1"])); + assert_eq!(out, WalkOutcome::default()); +} + +#[test] +fn installable_fix_skips_a_release_the_same_advisory_still_affects() { + // Withdrawn 1.0.1 ends the first window; 1.0.2 opens a second one. + let a = ranges( + r#"[{"introduced":"0"},{"fixed":"1.0.1"},{"introduced":"1.0.2"},{"fixed":"1.0.3"}]"#, + ); + let registry = facts(&["1.0.1", "1.0.2", "1.0.3"], &["1.0.1"]); + let fix = installable_line_fix("npm", "1.0.0", &a, ®istry); + assert_eq!(fix.fix.as_deref(), Some("1.0.3")); + assert_eq!(fix.withheld, ["1.0.1"]); + // Not withdrawn: the line fix as before. + let fix = installable_line_fix("npm", "1.0.0", &a, &RegistryReleases::default()); + assert_eq!(fix.fix.as_deref(), Some("1.0.1")); + assert!(fix.withheld.is_empty()); + // No line fix at all: nothing, and nothing withheld. + let open = ranges(r#"[{"introduced":"0"}]"#); + assert_eq!( + installable_line_fix("npm", "1.0.0", &open, ®istry), + LineFix::default() + ); +} diff --git a/crates/fourda-depengine/src/lib.rs b/crates/fourda-depengine/src/lib.rs index bca2dabf7..aa6ab3cdc 100644 --- a/crates/fourda-depengine/src/lib.rs +++ b/crates/fourda-depengine/src/lib.rs @@ -20,7 +20,9 @@ //! - [`matching`]: `match_inventory(&[Package], &[AdvisoryRecord])`, the //! findings per installed copy with each copy's fix targets; //! - [`exposure`]: the same check for one advisory (the scorer's verdict); -//! - [`lines`]: the per-project, per-release-line upgrade targets. +//! - [`lines`]: the per-project, per-release-line upgrade targets; +//! - [`registry`]: the registry's releases and the ones it withdrew (yanked +//! or deprecated), which no target may name. //! //! The app (`src-tauri`) keeps the DB and network halves and re-exports these //! under their old `crate::osv::...` paths. @@ -33,6 +35,7 @@ pub mod lines; pub mod matching; pub mod osv; pub mod range; +pub mod registry; pub mod version_order; #[cfg(target_arch = "wasm32")] diff --git a/crates/fourda-depengine/src/lines.rs b/crates/fourda-depengine/src/lines.rs index 1c1cb9dd6..5685da7d8 100644 --- a/crates/fourda-depengine/src/lines.rs +++ b/crates/fourda-depengine/src/lines.rs @@ -187,6 +187,7 @@ mod tests { fixed_version: fixed.map(str::to_string), clean_version: clean.map(str::to_string), target_version: walked.map(str::to_string), + withheld: Vec::new(), } } diff --git a/crates/fourda-depengine/src/matching.rs b/crates/fourda-depengine/src/matching.rs index ffae29eb0..79f479680 100644 --- a/crates/fourda-depengine/src/matching.rs +++ b/crates/fourda-depengine/src/matching.rs @@ -17,9 +17,10 @@ use std::collections::{HashMap, HashSet}; use serde::{Deserialize, Serialize}; use crate::ecosystem::normalize as normalize_ecosystem; -use crate::fix_target::{fix_for_version, walk_target, WalkTarget}; +use crate::fix_target::{installable_fix, installable_line_fix, walk_target_in, WalkOutcome}; use crate::osv::AdvisoryRecord; use crate::range::{check_version_affected, parse_version}; +use crate::registry::{RegistryIndex, RegistryReleases}; /// Installed copies by (project, package key, OSV ecosystem): each copy's /// (version, is_direct, is_dev). @@ -84,6 +85,12 @@ pub struct MatchedDependency { /// could not evaluate an advisory. #[serde(default)] pub target_version: Option, + /// Releases the registry withdrew (yanked, deprecated) that this copy's + /// line fix or walk would otherwise have named; none of the fields above + /// ever holds one. With no target, a non-empty list means the fix exists + /// but was withdrawn ("no installable fix", not "no fix published"). + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub withheld: Vec, } /// One advisory row that affects at least one installed copy. @@ -122,9 +129,10 @@ pub fn inventory_problem(inventory: &[Package]) -> Option { }) } -/// [`match_inventory_with`] without a host hook: every affected copy counts. +/// [`match_inventory_with`] without registry facts or a host hook: every +/// affected copy counts. pub fn match_inventory(inventory: &[Package], records: &[AdvisoryRecord]) -> Vec { - match_inventory_with(inventory, records, |_, _, _| {}) + match_inventory_with(inventory, records, &RegistryIndex::new(), |_, _, _| {}) } /// Match `records` against `inventory`. Findings come in `records` order (one @@ -133,12 +141,18 @@ pub fn match_inventory(inventory: &[Package], records: &[AdvisoryRecord]) -> Vec /// versions inside parseable windows; a copy with no version, or an advisory /// whose window cannot be read, is a conservative (unconfirmed) match. /// +/// `registry` holds what each package's registry says about its releases +/// (keyed by [`package_key`] and OSV ecosystem); no copy's target, line fix +/// or finding fix is ever a release it withdrew. A package with no entry is +/// matched on the advisory data alone. +/// /// `retain(index, record, copies)` sees each advisory's affected copies /// (`index` into `records`) before they are grouped and may drop some (the /// app's reachability filter). pub fn match_inventory_with( inventory: &[Package], records: &[AdvisoryRecord], + registry: &RegistryIndex, mut retain: F, ) -> Vec where @@ -190,7 +204,8 @@ where .or_default() .push(&record.affected_ranges); } - let mut walk_cache: HashMap<(String, String, String), Option> = HashMap::new(); + let mut walk_cache: HashMap<(String, String, String), WalkOutcome> = HashMap::new(); + let no_facts = RegistryReleases::default(); let mut findings: Vec = Vec::new(); let mut seen = HashSet::new(); @@ -203,6 +218,7 @@ where let Some(dep_entries) = dep_index.get(&key) else { continue; }; + let facts = registry.get(&key).unwrap_or(&no_facts); let mut copies = Vec::new(); for dep in dep_entries { @@ -225,37 +241,33 @@ where // Targets are facts about THIS copy: its own line's fix, and // where the walk over every advisory of the package lands // (clean, or clearing every advisory that has a way out). - let (fixed_version, walk) = match version.as_deref() { + let targets = match version.as_deref() { Some(v) if confirmed => { let walk = walk_cache .entry((key.0.clone(), key.1.clone(), v.to_string())) .or_insert_with(|| { package_ranges .get(&key) - .and_then(|ranges| walk_target(&record.ecosystem, v, ranges)) + .map(|ranges| { + walk_target_in(&record.ecosystem, v, ranges, facts) + }) + .unwrap_or_default() }) .clone(); - ( - fix_for_version(&record.ecosystem, v, &record.affected_ranges), - walk, - ) + CopyTargets::new(record, v, walk, facts) } - _ => (None, None), + _ => CopyTargets::default(), }; - let clean_version = walk - .as_ref() - .filter(|w| w.clears_all) - .map(|w| w.version.clone()); - let target_version = walk.map(|w| w.version); copies.push(MatchedDependency { project_path: normalize_project_path(&dep.project), installed_version: version, is_direct, is_dev, is_version_confirmed: confirmed, - fixed_version, - clean_version, - target_version, + fixed_version: targets.fixed, + clean_version: targets.clean, + target_version: targets.target, + withheld: targets.withheld, }); } } @@ -314,7 +326,11 @@ where .filter_map(|fix| parse_version(&fix).map(|parsed| (parsed, fix))) .max_by(|a, b| a.0.cmp(&b.0)) .map(|(_, fix)| fix) - .or_else(|| highest_listed_fix(&record.fixed_versions)); + .or_else(|| { + highest_listed_fix(&record.fixed_versions).and_then(|fix| { + installable_fix(&record.ecosystem, &fix, &record.affected_ranges, facts).fix + }) + }); findings.push(Finding { record: index, @@ -328,6 +344,44 @@ where findings } +/// One confirmed copy's targets, none of them a release the registry +/// withdrew. +#[derive(Default)] +struct CopyTargets { + fixed: Option, + clean: Option, + target: Option, + withheld: Vec, +} + +impl CopyTargets { + fn new( + record: &AdvisoryRecord, + installed: &str, + walk: WalkOutcome, + facts: &RegistryReleases, + ) -> Self { + let line = + installable_line_fix(&record.ecosystem, installed, &record.affected_ranges, facts); + let mut withheld = walk.withheld; + for w in line.withheld { + if !withheld.contains(&w) { + withheld.push(w); + } + } + CopyTargets { + fixed: line.fix, + clean: walk + .target + .as_ref() + .filter(|w| w.clears_all) + .map(|w| w.version.clone()), + target: walk.target.map(|w| w.version), + withheld, + } + } +} + /// The versions checked for one resolved row: every installed copy of that /// (project, package), deduped by version (first wins, with its own /// direct/dev labels), then the resolved row's own version unless a copy diff --git a/crates/fourda-depengine/src/matching_tests.rs b/crates/fourda-depengine/src/matching_tests.rs index 5c6d9fda4..a915d300a 100644 --- a/crates/fourda-depengine/src/matching_tests.rs +++ b/crates/fourda-depengine/src/matching_tests.rs @@ -70,10 +70,89 @@ fn no_resolved_row_means_no_audit_and_the_hook_can_drop_copies() { let inventory = vec![pkg("npm", "/p", "a", Some("1.0.0"), true)]; assert_eq!(match_inventory(&inventory, &records).len(), 1); - let dropped = match_inventory_with(&inventory, &records, |_, _, copies| copies.clear()); + let dropped = match_inventory_with( + &inventory, + &records, + &RegistryIndex::new(), + |_, _, copies| copies.clear(), + ); assert!(dropped.is_empty()); } +fn facts(published: &[&str], excluded: &[&str]) -> RegistryReleases { + RegistryReleases { + published: published.iter().map(ToString::to_string).collect(), + excluded: excluded.iter().map(ToString::to_string).collect(), + } +} + +/// FX-H5: lodash 4.17.21's advisories are all fixed in 4.18.0, which npm +/// marks deprecated ("Bad release"). No field may name it. +#[test] +fn a_withdrawn_fix_is_never_a_copy_target_or_a_finding_fix() { + let inventory = vec![pkg("npm", "/p", "lodash", Some("4.17.21"), true)]; + let ranges = r#"[{"type":"SEMVER","events":[{"introduced":"0"},{"fixed":"4.18.0"}]}]"#; + let records = vec![record( + "GHSA-1", + "npm", + "lodash", + ranges, + Some(r#"["4.18.0"]"#), + )]; + let mut registry = RegistryIndex::new(); + registry.insert( + ("lodash".into(), "npm".into()), + facts(&["4.17.21", "4.17.23", "4.18.0", "4.18.1"], &["4.18.0"]), + ); + let f = &match_inventory_with(&inventory, &records, ®istry, |_, _, _| {})[0]; + let copy = &f.dependency_instances[0]; + assert_eq!(copy.fixed_version.as_deref(), Some("4.18.1")); + assert_eq!(copy.clean_version.as_deref(), Some("4.18.1")); + assert_eq!(copy.target_version.as_deref(), Some("4.18.1")); + assert_eq!(copy.withheld, ["4.18.0"]); + assert_eq!(f.fixed_version.as_deref(), Some("4.18.1")); + + // Without registry facts: today's answer, unchanged. + let copy = &match_inventory(&inventory, &records)[0].dependency_instances[0]; + assert_eq!(copy.target_version.as_deref(), Some("4.18.0")); + assert!(copy.withheld.is_empty()); + + // Every clearing release withdrawn: no target, and it says why. + registry.insert( + ("lodash".into(), "npm".into()), + facts(&["4.17.21", "4.18.0"], &["4.18.0"]), + ); + let f = &match_inventory_with(&inventory, &records, ®istry, |_, _, _| {})[0]; + let copy = &f.dependency_instances[0]; + assert_eq!(copy.fixed_version, None); + assert_eq!(copy.target_version, None); + assert_eq!(copy.clean_version, None); + assert_eq!(copy.withheld, ["4.18.0"]); + assert_eq!(f.fixed_version, None); +} + +/// A copy no window could place still never reports a withdrawn listed fix. +#[test] +fn an_unconfirmed_finding_skips_a_withdrawn_listed_fix() { + let inventory = vec![pkg("npm", "/p", "lodash", None, true)]; + let ranges = r#"[{"type":"SEMVER","events":[{"introduced":"0"},{"fixed":"4.18.0"}]}]"#; + let records = vec![record( + "GHSA-1", + "npm", + "lodash", + ranges, + Some(r#"["4.18.0"]"#), + )]; + let mut registry = RegistryIndex::new(); + registry.insert( + ("lodash".into(), "npm".into()), + facts(&["4.18.0", "4.18.1"], &["4.18.0"]), + ); + let f = &match_inventory_with(&inventory, &records, ®istry, |_, _, _| {})[0]; + assert!(!f.is_version_confirmed); + assert_eq!(f.fixed_version.as_deref(), Some("4.18.1")); +} + #[test] fn a_version_less_row_is_a_conservative_match_and_a_repeat_row_reports_once() { let inventory = vec![pkg("PyPI", "/p", "Typing_Extensions", None, true)]; diff --git a/crates/fourda-depengine/src/registry.rs b/crates/fourda-depengine/src/registry.rs new file mode 100644 index 000000000..c4b8c29e3 --- /dev/null +++ b/crates/fourda-depengine/src/registry.rs @@ -0,0 +1,87 @@ +// SPDX-License-Identifier: FSL-1.1-Apache-2.0 +//! What a package's registry says about its releases, beyond the advisory +//! data: which releases exist, and which the registry has withdrawn or marked +//! unfit (crates.io and PyPI `yanked`, npm `deprecated`). The host fetches +//! these; the engine only reads them. +//! +//! Live 2026-10-16 (FX-H5): every advisory lodash 4.17.21 still had was fixed +//! in 4.18.0, so 4DA said "update lodash to >= 4.18.0". npm marks 4.18.0 +//! deprecated ("Bad release. Please use lodash@4.17.21 instead."); 4.18.1 is +//! the release to take. The walk now steps over a withdrawn release exactly +//! as it steps over a pre-release, and names what it stepped over. + +use std::cmp::Ordering; +use std::collections::HashMap; + +use serde::{Deserialize, Serialize}; + +use crate::version_order::OrderedVersion; + +/// One package's releases as its registry reports them. Both lists empty is +/// "no registry facts": the engine then knows only the releases the advisory +/// data names, as it did before this input existed. +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct RegistryReleases { + /// Every release the registry lists (any order; pre-releases are left out + /// of targets as always). Lets the walk name a release no advisory + /// mentions, such as the one after a withdrawn fix. + #[serde(default)] + pub published: Vec, + /// Releases the registry withdrew or marked unfit. Never a target. + #[serde(default)] + pub excluded: Vec, +} + +impl RegistryReleases { + /// No facts at all. + pub fn is_empty(&self) -> bool { + self.published.is_empty() && self.excluded.is_empty() + } + + /// The excluded releases, parsed in `ecosystem`. Unparseable spellings + /// are dropped (they cannot equal a version the walk can name). + pub fn excluded_in(&self, ecosystem: &str) -> Vec { + self.excluded + .iter() + .filter_map(|v| OrderedVersion::parse_in(v, ecosystem)) + .collect() + } + + /// Whether the registry withdrew `version` (equal as versions, so build + /// metadata such as Go's `+incompatible` does not hide it). + pub fn excludes(&self, ecosystem: &str, version: &str) -> bool { + OrderedVersion::parse_in(version, ecosystem) + .is_some_and(|v| is_listed(&self.excluded_in(ecosystem), &v)) + } +} + +/// Whether `version` equals any of `listed`. +pub(crate) fn is_listed(listed: &[OrderedVersion], version: &OrderedVersion) -> bool { + listed + .iter() + .any(|l| version.compare(l) == Some(Ordering::Equal)) +} + +/// Registry facts per package, keyed by (package key as +/// [`crate::matching::package_key`] forms it, OSV ecosystem), the key the +/// matcher groups a package's advisories by. +pub type RegistryIndex = HashMap<(String, String), RegistryReleases>; + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn excludes_compares_as_versions() { + let r = RegistryReleases { + published: vec![], + excluded: vec!["4.18.0".into(), "v25.0.13+incompatible".into()], + }; + assert!(r.excludes("npm", "4.18.0")); + assert!(!r.excludes("npm", "4.18.1")); + assert!(r.excludes("Go", "v25.0.13")); + assert!(!r.excludes("npm", "not a version")); + assert!(!r.is_empty()); + assert!(RegistryReleases::default().is_empty()); + } +} diff --git a/src-tauri/src/evidence/upgrade_plan.rs b/src-tauri/src/evidence/upgrade_plan.rs index b3be478a2..2a0f05a10 100644 --- a/src-tauri/src/evidence/upgrade_plan.rs +++ b/src-tauri/src/evidence/upgrade_plan.rs @@ -179,6 +179,10 @@ struct PackageGroup<'a> { /// "Upgrade" (2026-09-06: 14 of 30 live plan items said "Upgrade gtk — /// clears 1 advisory" with nothing to upgrade to). has_fix: bool, + /// With no fix: the releases that fixed it but that the registry withdrew + /// (yanked, deprecated; FX-H5). Non-empty means "no installable fix", + /// never "no fix published". + withheld: Vec, /// Every advisory is a maintenance notice (unmaintained / deprecated: no /// fix, no CVSS). Folded into ONE Watch item, never an upgrade step. informational: bool, @@ -246,7 +250,16 @@ fn package_group<'a>( // (no fix exists) read "Fixable now via a direct dependency bump" // on the live plan (2026-09-06). let fixable_now = has_fix && instances().any(|d| d.is_direct); - let informational = !has_fix && advisories.iter().all(|a| is_informational(a)); + let mut withheld: Vec = Vec::new(); + if !has_fix { + for w in instances().flat_map(|d| d.withheld.iter()) { + if !withheld.contains(w) { + withheld.push(w.clone()); + } + } + } + let informational = + !has_fix && withheld.is_empty() && advisories.iter().all(|a| is_informational(a)); // The inputs of the ONE scope rule (AD-046): version-confirmed installs in // the projects this row names — the set the alert path reads too. let scope = crate::osv::identity::ExposureScope::of(&advisories, &projects); @@ -287,6 +300,7 @@ fn package_group<'a>( lines, targets, has_fix, + withheld, informational, cohort_key, routes: crate::osv::parent_hint::TransitiveRoutes::default(), @@ -383,7 +397,12 @@ impl PackageGroup<'_> { ) } else { format!( - "No fix published for {pkg} — {n} {adv} across {m} {proj}", + "{none} for {pkg} — {n} {adv} across {m} {proj}", + none = if self.withheld.is_empty() { + "No fix published" + } else { + "No installable fix" + }, pkg = self.package, n = n, adv = plural(n, "advisory", "advisories"), @@ -392,7 +411,13 @@ impl PackageGroup<'_> { ) }); - let scope_note = if !self.has_fix { + let scope_note = if !self.has_fix && !self.withheld.is_empty() { + format!( + "The release that fixes it ({}) was withdrawn by its registry (yanked or \ + deprecated), and no later release is known to clear it — pin, patch, or replace", + self.withheld.join(", ") + ) + } else if !self.has_fix { "No fix has been published — pin, patch, or replace; there is nothing to upgrade to yet" .to_string() } else if self.fixable_now { diff --git a/src-tauri/src/evidence/upgrade_plan_tests.rs b/src-tauri/src/evidence/upgrade_plan_tests.rs index 33011d6ef..33ee3ae45 100644 --- a/src-tauri/src/evidence/upgrade_plan_tests.rs +++ b/src-tauri/src/evidence/upgrade_plan_tests.rs @@ -495,6 +495,7 @@ fn matched(cvss: f64, installs: &[(bool, bool, bool)]) -> crate::osv::types::Mat fixed_version: None, clean_version: None, target_version: None, + withheld: Vec::new(), }, ) .collect(), @@ -1262,3 +1263,64 @@ fn advisory_citation_age_comes_from_the_publish_date() { assert_eq!(advisory_age_days(Some("")), UNKNOWN_AGE_DAYS); assert_eq!(advisory_age_days(Some("not a date")), UNKNOWN_AGE_DAYS); } + +// --------------------------------------------------------------------------- +// FX-H5 (2026-10-16): lodash 4.17.21 was told "Upgrade lodash to >= 4.18.0"; +// npm marks 4.18.0 deprecated ("Bad release"). Neither the title nor the work +// order may name a release its registry withdrew. +// --------------------------------------------------------------------------- + +fn registry_facts(db: &Database, eco: &str, name: &str, published: &str, excluded: &str) { + let fetched_at = chrono::Utc::now().to_rfc3339(); + let doc = format!( + r#"[{{"ecosystem":"{eco}","key":"{name}","name":"{name}","fetched_at":"{fetched_at}","published":{published},"excluded":{excluded}}}]"# + ); + db.set_kv("osv_registry_releases", &doc).unwrap(); +} + +#[test] +fn a_deprecated_fix_release_is_never_the_plan_target() { + let db = test_db(); + db.store_dependency("/proj/a", "lodash", Some("4.17.21"), "npm", false, None) + .unwrap(); + advisory(&db, "GHSA-lodash-2", "lodash", "npm", "4.18.0", 7.5); + registry_facts( + &db, + "npm", + "lodash", + r#"["4.17.21","4.17.23","4.18.0","4.18.1"]"#, + r#"["4.18.0"]"#, + ); + let built = super::build_upgrade_plan(&db); + assert_eq!(built.items.len(), 1); + assert!( + built.items[0].title.contains(">= 4.18.1"), + "title: {}", + built.items[0].title + ); + let all = serde_json::to_string(&(&built.items, &built.steps)).unwrap(); + assert!(!all.contains("4.18.0"), "a withdrawn release leaked: {all}"); +} + +#[test] +fn every_fixing_release_withdrawn_reads_no_installable_fix() { + let db = test_db(); + db.store_dependency("/proj/a", "h2", Some("0.3.16"), "rust", false, None) + .unwrap(); + advisory(&db, "RUSTSEC-2023-0034", "h2", "crates.io", "0.3.17", 7.5); + registry_facts( + &db, + "crates.io", + "h2", + r#"["0.3.16","0.3.17"]"#, + r#"["0.3.17"]"#, + ); + let built = super::build_upgrade_plan(&db); + assert_eq!(built.items.len(), 1); + let title = &built.items[0].title; + assert!( + title.starts_with("No installable fix for h2"), + "title: {title}" + ); + assert!(!title.contains("0.3.17"), "title: {title}"); +} diff --git a/src-tauri/src/osv/fix_target_tests.rs b/src-tauri/src/osv/fix_target_tests.rs index 18bc2579f..310c53f7e 100644 --- a/src-tauri/src/osv/fix_target_tests.rs +++ b/src-tauri/src/osv/fix_target_tests.rs @@ -132,6 +132,7 @@ fn copy(project: &str, version: &str, fix: Option<&str>, clean: Option<&str>) -> fixed_version: fix.map(str::to_string), clean_version: clean.map(str::to_string), target_version: clean.map(str::to_string), + withheld: Vec::new(), } } diff --git a/src-tauri/src/osv/identity.rs b/src-tauri/src/osv/identity.rs index 920324824..f527bb28e 100644 --- a/src-tauri/src/osv/identity.rs +++ b/src-tauri/src/osv/identity.rs @@ -732,6 +732,7 @@ mod tests { fixed_version: None, clean_version: None, target_version: None, + withheld: Vec::new(), }; let mut sandbox = adv("GHSA-s", "sandbox", None, "s", &[], Some(9.8), None); sandbox.dependency_instances = vec![ diff --git a/src-tauri/src/osv/matching.rs b/src-tauri/src/osv/matching.rs index 054f44f4d..189478619 100644 --- a/src-tauri/src/osv/matching.rs +++ b/src-tauri/src/osv/matching.rs @@ -116,7 +116,9 @@ pub(crate) fn get_matched_advisories_with_not_compiled( let records: Vec = advisories.iter().map(advisory_record).collect(); let mut reach = ReachFilter::default(); - let findings = match_inventory_with(&inventory, &records, |index, _, copies| { + // Releases each package's registry withdrew (FX-H5): never a target. + let registry = super::release_facts::load(db); + let findings = match_inventory_with(&inventory, &records, ®istry, |index, _, copies| { reach.retain_compiled(db, &advisories[index], copies); }); diff --git a/src-tauri/src/osv/mod.rs b/src-tauri/src/osv/mod.rs index cd93fb4ef..3e9135cd9 100644 --- a/src-tauri/src/osv/mod.rs +++ b/src-tauri/src/osv/mod.rs @@ -18,6 +18,7 @@ pub(crate) mod matching; pub(crate) mod parent_hint; pub(crate) mod reachability; pub(crate) mod readiness; +pub(crate) mod release_facts; pub(crate) mod sync; pub(crate) mod types; #[cfg(test)] diff --git a/src-tauri/src/osv/reachability_tests.rs b/src-tauri/src/osv/reachability_tests.rs index 07aa6c53b..c168f76d1 100644 --- a/src-tauri/src/osv/reachability_tests.rs +++ b/src-tauri/src/osv/reachability_tests.rs @@ -192,6 +192,7 @@ fn instance(project: &str, version: &str, confirmed: bool) -> MatchedDependency fixed_version: Some("2.0.0".to_string()), clean_version: None, target_version: None, + withheld: Vec::new(), } } diff --git a/src-tauri/src/osv/release_facts.rs b/src-tauri/src/osv/release_facts.rs new file mode 100644 index 000000000..90255914c --- /dev/null +++ b/src-tauri/src/osv/release_facts.rs @@ -0,0 +1,309 @@ +// SPDX-License-Identifier: FSL-1.1-Apache-2.0 +//! Registry release facts for the fix-target walk (FX-H5): which releases a +//! package has, and which its registry withdrew. A fix target must never be +//! one of those: lodash 4.17.21 was told "update to >= 4.18.0", a release npm +//! marks deprecated ("Bad release"), and a yanked crate named by +//! `cargo update --precise` fails outright. +//! +//! What counts as withdrawn, per registry: +//! - **crates.io:** `yanked` on the version (`/api/v1/crates/{crate}`). +//! - **PyPI:** a release whose every file is `yanked` (PEP 592; pip skips it +//! unless pinned exactly), from `/pypi/{package}/json`. +//! - **npm:** `deprecated` on the version, from the abbreviated packument +//! (`GET /{package}`, `Accept: application/vnd.npm.install-v1+json`), but +//! only while the package's `latest` release is NOT deprecated. When `latest` +//! is deprecated too, the whole package is retired ("no longer supported"), +//! not one release; excluding every release would hide the version that +//! clears the advisory, so nothing is excluded then. +//! - **Go:** not covered. A module's `retract` directives live in the +//! `go.mod` of its newest version, which 4DA does not read; Go targets are +//! computed from the advisory data alone, as before. +//! +//! Fetched only for packages with a version-confirmed advisory, after each +//! OSV sync, from hosts the registry adapters already contact (NETWORK.md +//! §2c). Kept in `kv_store` and refreshed when older than a day. Offline, or +//! for a package never fetched, the engine gets no facts and behaves exactly +//! as it did before this module: targets come from the advisory data. + +use std::collections::{BTreeSet, HashMap}; + +use chrono::{DateTime, Duration, Utc}; +use fourda_depengine::registry::{RegistryIndex, RegistryReleases}; +use serde::{Deserialize, Serialize}; +use tracing::{debug, info, warn}; + +use crate::db::Database; + +/// `kv_store` key holding every package's facts as one JSON document. +const KV_KEY: &str = "osv_registry_releases"; +/// Facts younger than this are not re-fetched. +const FRESH_HOURS: i64 = 24; +/// Facts older than this are dropped rather than used. +const KEEP_DAYS: i64 = 30; +/// Registry reads per sync, so a large vulnerable set spreads over syncs. +const MAX_FETCHES_PER_SYNC: usize = 60; +/// Consecutive failures that end a pass (offline, or a registry down). +const MAX_FAILURES: usize = 3; +const USER_AGENT: &str = "4DA-Developer-OS/1.0 (https://4da.ai)"; +const NPM_ABBREVIATED: &str = "application/vnd.npm.install-v1+json"; + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +struct StoredPackage { + /// OSV ecosystem (`npm`, `crates.io`, `PyPI`). + ecosystem: String, + /// The matcher's package key. + key: String, + /// The registry's own spelling, used for the next fetch. + name: String, + fetched_at: DateTime, + #[serde(flatten)] + releases: RegistryReleases, +} + +fn read_stored(db: &Database) -> Vec { + db.get_kv(KV_KEY) + .ok() + .flatten() + .and_then(|text| serde_json::from_str(&text).ok()) + .unwrap_or_default() +} + +/// Every package's facts for the matcher, keyed as it groups advisories. +/// Empty (and so exactly the advisory-data-only behaviour) when nothing was +/// ever fetched. +pub(crate) fn load(db: &Database) -> RegistryIndex { + index(read_stored(db), Utc::now()) +} + +fn index(stored: Vec, now: DateTime) -> RegistryIndex { + stored + .into_iter() + .filter(|p| now - p.fetched_at < Duration::days(KEEP_DAYS)) + .map(|p| ((p.key, p.ecosystem), p.releases)) + .collect() +} + +/// The OSV ecosystem this module reads a registry for, if any. +fn covered(ecosystem: &str) -> Option<&'static str> { + match crate::ecosystem::Ecosystem::parse(ecosystem)?.osv_name() { + "npm" => Some("npm"), + "crates.io" => Some("crates.io"), + "PyPI" => Some("PyPI"), + _ => None, + } +} + +/// (OSV ecosystem, name) of every covered package with a version-confirmed +/// advisory: the only packages whose targets the facts can change. +fn wanted(db: &Database) -> Vec<(String, String)> { + let matches = super::matching::get_matched_advisories(db).unwrap_or_default(); + let mut out: BTreeSet<(String, String)> = BTreeSet::new(); + for m in matches.iter().filter(|m| m.is_version_confirmed) { + if let Some(eco) = covered(&m.ecosystem) { + out.insert((eco.to_string(), m.package_name.clone())); + } + } + out.into_iter().collect() +} + +/// Fetch facts for the packages that need them (missing or older than a +/// day), at most [`MAX_FETCHES_PER_SYNC`]. Best effort: a failed read keeps +/// whatever was stored. Returns how many packages were refreshed. +pub(crate) async fn refresh(db: &Database) -> usize { + let now = Utc::now(); + let mut stored: HashMap<(String, String), StoredPackage> = read_stored(db) + .into_iter() + .filter(|p| now - p.fetched_at < Duration::days(KEEP_DAYS)) + .map(|p| ((p.key.clone(), p.ecosystem.clone()), p)) + .collect(); + let due = due(wanted(db), &stored, now); + if due.is_empty() { + return 0; + } + let fetched = fetch_all(due, &mut stored).await; + save(db, stored); + info!(target: "4da::osv", fetched, "registry release facts refreshed"); + fetched +} + +/// The wanted packages with no facts, or facts older than a day, capped. +fn due( + wanted: Vec<(String, String)>, + stored: &HashMap<(String, String), StoredPackage>, + now: DateTime, +) -> Vec<(String, String)> { + wanted + .into_iter() + .filter(|(eco, name)| { + let key = (super::matching::package_key(name, eco), eco.clone()); + stored + .get(&key) + .is_none_or(|p| now - p.fetched_at >= Duration::hours(FRESH_HOURS)) + }) + .take(MAX_FETCHES_PER_SYNC) + .collect() +} + +fn save(db: &Database, stored: HashMap<(String, String), StoredPackage>) { + let mut all: Vec = stored.into_values().collect(); + all.sort_by(|a, b| (&a.ecosystem, &a.key).cmp(&(&b.ecosystem, &b.key))); + match serde_json::to_string(&all) { + Ok(text) => { + if let Err(e) = db.set_kv(KV_KEY, &text) { + warn!(target: "4da::osv", error = %e, "registry release facts: store failed"); + } + } + Err(e) => warn!(target: "4da::osv", error = %e, "registry release facts: encode failed"), + } +} + +/// Read each due package's registry; a success replaces its stored facts. +async fn fetch_all( + due: Vec<(String, String)>, + stored: &mut HashMap<(String, String), StoredPackage>, +) -> usize { + let client = reqwest::Client::builder() + .user_agent(USER_AGENT) + .timeout(std::time::Duration::from_secs(30)) + .redirect(crate::http_client::ssrf_guarded_redirect_policy()) + .build() + .unwrap_or_default(); + let (mut fetched, mut failures) = (0usize, 0usize); + for (eco, name) in due { + match fetch(&client, &eco, &name).await { + Ok(releases) => { + failures = 0; + fetched += 1; + let key = super::matching::package_key(&name, &eco); + stored.insert( + (key.clone(), eco.clone()), + StoredPackage { + ecosystem: eco, + key, + name, + fetched_at: Utc::now(), + releases, + }, + ); + } + Err(e) => { + debug!(target: "4da::osv", package = %name, error = %e, "registry release facts unavailable"); + failures += 1; + if failures >= MAX_FAILURES { + warn!(target: "4da::osv", "registry release facts: stopping after {failures} failures"); + break; + } + } + } + // crates.io asks for at most one request per second. + tokio::time::sleep(std::time::Duration::from_millis(1100)).await; + } + fetched +} + +async fn fetch( + client: &reqwest::Client, + eco: &str, + name: &str, +) -> Result { + let (url, accept) = match eco { + "npm" => ( + format!("https://registry.npmjs.org/{}", name.replace('/', "%2f")), + NPM_ABBREVIATED, + ), + "crates.io" => ( + format!("https://crates.io/api/v1/crates/{name}"), + "application/json", + ), + "PyPI" => ( + format!("https://pypi.org/pypi/{name}/json"), + "application/json", + ), + other => return Err(format!("no registry read for {other}")), + }; + let reply = client + .get(&url) + .header(reqwest::header::ACCEPT, accept) + .send() + .await + .map_err(|e| e.to_string())?; + if !reply.status().is_success() { + return Err(format!("HTTP {}", reply.status())); + } + let doc: serde_json::Value = reply.json().await.map_err(|e| e.to_string())?; + let releases = match eco { + "npm" => parse_npm(&doc), + "crates.io" => parse_crates(&doc), + _ => parse_pypi(&doc), + }; + releases.ok_or_else(|| "unreadable registry answer".to_string()) +} + +/// npm abbreviated packument: every version, and the deprecated ones unless +/// `latest` is deprecated too (a retired package, see the module notes). +fn parse_npm(doc: &serde_json::Value) -> Option { + let versions = doc.get("versions")?.as_object()?; + let deprecated = |v: &serde_json::Value| { + v.get("deprecated") + .is_some_and(|d| d.as_str().is_some_and(|s| !s.is_empty()) || d.as_bool() == Some(true)) + }; + let retired = doc + .get("dist-tags") + .and_then(|t| t.get("latest")) + .and_then(|l| l.as_str()) + .and_then(|l| versions.get(l)) + .is_some_and(deprecated); + Some(RegistryReleases { + published: versions.keys().cloned().collect(), + excluded: if retired { + Vec::new() + } else { + versions + .iter() + .filter(|(_, v)| deprecated(v)) + .map(|(k, _)| k.clone()) + .collect() + }, + }) +} + +/// crates.io crate document: every version's `num`, the yanked ones excluded. +fn parse_crates(doc: &serde_json::Value) -> Option { + let versions = doc.get("versions")?.as_array()?; + let mut out = RegistryReleases::default(); + for v in versions { + let Some(num) = v.get("num").and_then(|n| n.as_str()) else { + continue; + }; + out.published.push(num.to_string()); + if v.get("yanked").and_then(serde_json::Value::as_bool) == Some(true) { + out.excluded.push(num.to_string()); + } + } + Some(out) +} + +/// PyPI JSON API: every release; one whose files are all yanked is excluded. +/// A release with no files at all cannot be installed and is not listed. +fn parse_pypi(doc: &serde_json::Value) -> Option { + let releases = doc.get("releases")?.as_object()?; + let mut out = RegistryReleases::default(); + for (version, files) in releases { + let files = files.as_array().map(Vec::as_slice).unwrap_or_default(); + if files.is_empty() { + continue; + } + out.published.push(version.clone()); + if files + .iter() + .all(|f| f.get("yanked").and_then(serde_json::Value::as_bool) == Some(true)) + { + out.excluded.push(version.clone()); + } + } + Some(out) +} + +#[cfg(test)] +#[path = "release_facts_tests.rs"] +mod tests; diff --git a/src-tauri/src/osv/release_facts_tests.rs b/src-tauri/src/osv/release_facts_tests.rs new file mode 100644 index 000000000..de7b900b2 --- /dev/null +++ b/src-tauri/src/osv/release_facts_tests.rs @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: FSL-1.1-Apache-2.0 +//! Registry answers trimmed from the live registries (2026-10-16). + +use serde_json::json; + +use super::*; + +#[test] +fn npm_excludes_a_deprecated_release_while_latest_is_live() { + // registry.npmjs.org/lodash, abbreviated packument. + let doc = json!({ + "dist-tags": {"latest": "4.18.1"}, + "versions": { + "4.17.21": {}, + "4.17.23": {}, + "4.18.0": {"deprecated": "Bad release. Please use lodash@4.17.21 instead."}, + "4.18.1": {} + } + }); + let r = parse_npm(&doc).unwrap_or_default(); + assert_eq!(r.published.len(), 4); + assert_eq!(r.excluded, ["4.18.0"]); +} + +#[test] +fn npm_excludes_nothing_when_the_whole_package_is_retired() { + let doc = json!({ + "dist-tags": {"latest": "2.88.2"}, + "versions": { + "2.88.0": {"deprecated": "request has been deprecated"}, + "2.88.2": {"deprecated": "request has been deprecated"} + } + }); + let r = parse_npm(&doc).unwrap_or_default(); + assert_eq!(r.published.len(), 2); + assert!(r.excluded.is_empty()); + assert!(parse_npm(&json!({"error": "not found"})).is_none()); +} + +#[test] +fn crates_excludes_yanked_versions() { + // crates.io/api/v1/crates/h2: 0.3.17 is yanked. + let doc = json!({"versions": [ + {"num": "0.3.18", "yanked": false}, + {"num": "0.3.17", "yanked": true}, + {"num": "0.3.16", "yanked": false} + ]}); + let r = parse_crates(&doc).unwrap_or_default(); + assert_eq!(r.published, ["0.3.18", "0.3.17", "0.3.16"]); + assert_eq!(r.excluded, ["0.3.17"]); +} + +#[test] +fn pypi_excludes_a_release_whose_every_file_is_yanked() { + let doc = json!({"releases": { + "1.0.0": [{"yanked": false}], + "1.0.1": [{"yanked": true}, {"yanked": true}], + "1.0.2": [{"yanked": true}, {"yanked": false}], + "1.0.3": [] + }}); + let r = parse_pypi(&doc).unwrap_or_default(); + assert_eq!(r.published, ["1.0.0", "1.0.1", "1.0.2"]); + assert_eq!(r.excluded, ["1.0.1"]); +} + +#[test] +fn the_index_keys_like_the_matcher_and_drops_month_old_facts() { + let now = Utc::now(); + let pkg = |key: &str, age_days: i64| StoredPackage { + ecosystem: "npm".into(), + key: key.into(), + name: key.into(), + fetched_at: now - Duration::days(age_days), + releases: RegistryReleases { + published: vec!["1.0.0".into()], + excluded: vec![], + }, + }; + let idx = index(vec![pkg("lodash", 1), pkg("old", 40)], now); + assert!(idx.contains_key(&("lodash".to_string(), "npm".to_string()))); + assert!(!idx.contains_key(&("old".to_string(), "npm".to_string()))); + + // A stored document round-trips with the facts flattened in. + let text = serde_json::to_string(&vec![pkg("lodash", 0)]).unwrap_or_default(); + assert!(text.contains("\"published\":[\"1.0.0\"]")); + let back: Vec = serde_json::from_str(&text).unwrap_or_default(); + assert_eq!(back.len(), 1); +} + +#[test] +fn only_registries_with_release_facts_are_covered() { + assert_eq!(covered("javascript"), Some("npm")); + assert_eq!(covered("rust"), Some("crates.io")); + assert_eq!(covered("pip"), Some("PyPI")); + assert_eq!(covered("go"), None); +} diff --git a/src-tauri/src/osv/sync.rs b/src-tauri/src/osv/sync.rs index 52cbb383b..defb7be19 100644 --- a/src-tauri/src/osv/sync.rs +++ b/src-tauri/src/osv/sync.rs @@ -355,6 +355,11 @@ pub async fn sync(db: &Database) -> Result { info!(target: "4da::osv", resolved_alerts, "Retired patched dependency alerts"); } + // Which releases the registries withdrew (yanked, deprecated), for the + // packages that now have a confirmed advisory: no fix target may name + // one. Best effort; without it targets come from the advisory data. + super::release_facts::refresh(db).await; + // Count matched advisories result.advisories_matched = super::matching::count_matches(db).unwrap_or(0); result.duration_ms = start.elapsed().as_millis() as u64; diff --git a/src-tauri/src/osv/vectors_tests.rs b/src-tauri/src/osv/vectors_tests.rs index 22d0e7090..9befb55f6 100644 --- a/src-tauri/src/osv/vectors_tests.rs +++ b/src-tauri/src/osv/vectors_tests.rs @@ -31,8 +31,9 @@ use serde_json::Value; // version_order, osv_range and fix_target run against the pure engine crate // directly (single-engine C0.1); refresh_verdict and fix_command are still the // app's `fix_path` (C0.4), and records are merged by the app's `osv::sync`. -use fourda_depengine::fix_target::{fix_for_version, walk_target}; +use fourda_depengine::fix_target::{installable_line_fix, walk_target_in}; use fourda_depengine::range::{check_version_affected, parse_version}; +use fourda_depengine::registry::RegistryReleases; use fourda_depengine::version_order::OrderedVersion; use super::fix_path::{refresh_verdict, Basis, Manager, Refresh, Requirement}; @@ -175,20 +176,37 @@ fn same_version(eco: &str, a: Option<&str>, b: Option<&str>) -> bool { /// The copy's target as `fix_target::line_targets` forms it: where the walk /// lands (`clears_all_known` when it clears everything), else, when the walk /// cannot run, the highest own-line fix among the advisories that affect the -/// copy. -fn engine_target(eco: &str, installed: &str, ranges: &[Option]) -> (Option, bool) { +/// copy. Neither is ever a release `registry` withdrew; the third value names +/// the withdrawn ones the walk stepped over. +fn engine_target( + eco: &str, + installed: &str, + ranges: &[Option], + registry: &RegistryReleases, +) -> (Option, bool, Vec) { let refs: Vec<&Option> = ranges.iter().collect(); - if let Some(walk) = walk_target(eco, installed, &refs) { - return (Some(walk.version), walk.clears_all); + let outcome = walk_target_in(eco, installed, &refs, registry); + if let Some(walk) = outcome.target { + return (Some(walk.version), walk.clears_all, outcome.withheld); } let partial = ranges .iter() .filter(|r| check_version_affected(eco, Some(installed), r) == (true, true)) - .filter_map(|r| fix_for_version(eco, installed, r)) + .filter_map(|r| installable_line_fix(eco, installed, r, registry).fix) .filter_map(|f| parse_version(&f).map(|p| (p, f))) .max_by(|a, b| a.0.cmp(&b.0)) .map(|(_, f)| f); - (partial, false) + (partial, false, outcome.withheld) +} + +/// A string-array field of a vector line (absent reads as empty). +fn strings(v: &Value, key: &str) -> Vec { + v.get(key) + .and_then(Value::as_array) + .into_iter() + .flatten() + .filter_map(|s| s.as_str().map(str::to_string)) + .collect() } fn check_fix_target(v: &Value, records: &mut Records) -> Result<(), String> { @@ -202,7 +220,11 @@ fn check_fix_target(v: &Value, records: &mut Records) -> Result<(), String> { { ranges.push(records.ranges(id.as_str().unwrap_or_default(), eco, name)?); } - let (target, clears) = engine_target(eco, installed, &ranges); + let registry = RegistryReleases { + published: strings(v, "published"), + excluded: strings(v, "excluded"), + }; + let (target, clears, withheld) = engine_target(eco, installed, &ranges, ®istry); let want_target = opt_s(v, "target"); let want_clears = v.get("clears").and_then(Value::as_bool).unwrap_or(false); if !same_version(eco, target.as_deref(), want_target) { @@ -212,6 +234,13 @@ fn check_fix_target(v: &Value, records: &mut Records) -> Result<(), String> { if want_target.is_some() && clears != want_clears { return Err(format!("clears: got {clears}, want {want_clears}")); } + // `withheld` is checked only on lines that state it. + if v.get("withheld").is_some() && withheld != strings(v, "withheld") { + return Err(format!( + "withheld: got {withheld:?}, want {:?}", + strings(v, "withheld") + )); + } Ok(()) } diff --git a/src-tauri/src/preemption.rs b/src-tauri/src/preemption.rs index 8e2ef3608..b3453700f 100644 --- a/src-tauri/src/preemption.rs +++ b/src-tauri/src/preemption.rs @@ -3117,6 +3117,7 @@ mod tests { fixed_version: None, clean_version: None, target_version: None, + withheld: Vec::new(), }, crate::osv::types::MatchedDependency { project_path: "/direct".into(), @@ -3127,6 +3128,7 @@ mod tests { fixed_version: None, clean_version: None, target_version: None, + withheld: Vec::new(), }, ], }; @@ -3168,6 +3170,7 @@ mod tests { fixed_version: Some(fix.into()), clean_version: Some(fix.into()), target_version: Some(fix.into()), + withheld: Vec::new(), }; let advisory = crate::osv::types::MatchedAdvisory { advisory_id: "GHSA-q2hr-2g5m-vwhr".into(), @@ -3244,6 +3247,7 @@ mod tests { fixed_version: None, clean_version: None, target_version: None, + withheld: Vec::new(), }], }; let projects = vec!["/paddle-webhook".to_string()]; diff --git a/src-tauri/src/scoring/security_verdict.rs b/src-tauri/src/scoring/security_verdict.rs index 2cf1f7c56..20429491e 100644 --- a/src-tauri/src/scoring/security_verdict.rs +++ b/src-tauri/src/scoring/security_verdict.rs @@ -288,13 +288,34 @@ pub(crate) fn ranges_verdict(installed: Option<&str>, ranges: &[String]) -> Opti Some(reqs?.iter().any(|req| req.matches(&version))) } -fn highest_fix(advisory: &StoredAdvisory) -> Option { +/// The advisory's highest listed fix, as a release its registry still offers +/// (FX-H5): a yanked or deprecated fix gives way to the next allowed release +/// the advisory does not affect, or to nothing. +fn highest_fix(db: &Database, advisory: &StoredAdvisory) -> Option { let versions: Vec = serde_json::from_str(advisory.fixed_versions.as_deref()?).ok()?; - versions + let fix = versions .into_iter() .filter_map(|v| super::release_version::lenient_semver(&v, None).map(|parsed| (parsed, v))) .max_by(|a, b| a.0.cmp(&b.0)) - .map(|(_, v)| v) + .map(|(_, v)| v)?; + let eco = crate::ecosystem::Ecosystem::parse(&advisory.ecosystem) + .map_or(advisory.ecosystem.as_str(), |e| e.osv_name()); + let key = ( + crate::osv::matching::package_key(&advisory.package_name, eco), + eco.to_string(), + ); + match crate::osv::release_facts::load(db).get(&key) { + Some(facts) => { + fourda_depengine::fix_target::installable_fix( + eco, + &fix, + &advisory.affected_ranges, + facts, + ) + .fix + } + None => Some(fix), + } } /// Run the Preemption matcher for every mirror row of the advisory the item @@ -331,7 +352,7 @@ pub(crate) fn matcher_verdict(db: &Database, ids: &[String]) -> Option` | PEP 440, SemVer 2.0.0 and the Go Modules Reference spec examples. Every line is cross-checked when written: PyPI lines with python `packaging` 25.0, the others with node `semver` 7 (Go with the `v` stripped). | | `osv_range.jsonl` | `{record_id, eco, name, version, affected}` | OSV's own answer: `/v1/querybatch` at the exact version, recorded in `../expected/.json` (records pinned 2026-10-10 in `../osv/`). `affected: true` is used only when the finding's alias group holds exactly one pinned record for that package, so OSV returned exactly that record. | -| `fix_target.jsonl` | `{eco, name, installed, record_ids[], target, clears, evidence}` | ft-001..ft-022: the 22 cases of the 2026-10-10 fix-path oracle (the action was applied, re-resolved and re-scanned, and the target checked minimal against OSV per published version), re-checked 2026-10-12 against the pinned ids. ft-023..: OSV `/v1/querybatch` per published stable release (2026-10-12), restricted to the pinned `record_ids`. `evidence` holds the OSV ids at `installed` and at `target`. | +| `fix_target.jsonl` | `{eco, name, installed, record_ids[], target, clears, evidence}`, optionally `published[]`, `excluded[]`, `withheld[]` | ft-001..ft-022: the 22 cases of the 2026-10-10 fix-path oracle (the action was applied, re-resolved and re-scanned, and the target checked minimal against OSV per published version), re-checked 2026-10-12 against the pinned ids. ft-023..: OSV `/v1/querybatch` per published stable release (2026-10-12), restricted to the pinned `record_ids`. `evidence` holds the OSV ids at `installed` and at `target`. | | `refresh_verdict.jsonl` | `{eco, name, installed, target, requirements[], verdict, via, inferred}` | The 16 transitive oracle cases (the refresh was applied: it reached the target, or it did not move the copy), each with its parents' requirements read from the case lockfile (`package-lock.json` `requires`, `yarn.lock` `dependencies`) or the parent's published manifest (npm registry for pnpm, crates.io for Cargo). Each case also appears with `requirements: []` to pin the semver fallback. The rest apply AD-055 ¶1 by hand. | | `fix_command.jsonl` | `{tool, eco, pkg, installed, target, workspace_root, command}` | The command text of AD-055 ¶4. Where the oracle applied that tool's refresh, the `source` names the case. | | `range_check.jsonl` | `{eco, name, constraint, from, releases[], record_ids[], expected}` | `@4da/mcp-server` 6.2.0 recorded fixtures (`src/__tests__/fixtures/dependency-check/recorded/`, origin/main `bae421b`): OSV's querybatch answer per release of ws, Jinja2 and golang.org/x/net. Constraint membership: node `semver` (npm, Go without `v`; `v0` is the v0 line), python `packaging` `SpecifierSet` (PyPI). | @@ -46,7 +46,20 @@ Field conventions: unions, spaces AND) or `cargo` (a bare version is a caret, commas AND). - A `fix_target` `target` is compared as a version (build metadata ignored). `clears: false` means the target clears only the advisories that have a - fix. `target: null` means no advisory on the copy has one. + fix. `target: null` means no advisory on the copy has one, or (with + `excluded`) every release that has one was withdrawn. +- Optional `fix_target` registry fields (FX-H5, 2026-10-16). Lines without + them run on the advisory data alone, as before: + - `published[]`: the registry's releases above `installed`, as the + registry listed them on the date in `source`. They join the releases the + advisory data names, so the walk can step to a release no advisory + mentions. + - `excluded[]`: releases the registry withdrew (crates.io and PyPI + `yanked`; npm `deprecated` while the package's `latest` is not). No + target may be one; the walk steps over them as it steps over + pre-releases. + - `withheld[]` (expected, checked only when present): the withdrawn + releases the walk would otherwise have named, in the order met. - `range_check` `releases` are the stable releases that have a recorded OSV answer. `record_ids` are the pinned records in `../osv/`; they are the same set as the MCP's recorded `/v1/query` answer for each package. A release is @@ -118,9 +131,10 @@ form. > fix"). - The 22 oracle cases: ft-001..ft-022 (openssl 0.10.80: ft-006; braces `last_affected`: ft-004) -- OSV per published release, all four registries: ft-023..ft-061 +- OSV per published release, all four registries: ft-023..ft-061, ft-062, ft-063 - No clean version, so the target clears only the fixed advisories (`clears: false`): ft-004, ft-029, ft-041, ft-042, ft-055, ft-060, ft-061 - A `last_affected` window that later releases leave (the walk steps past the bound and keeps checking every advisory): ft-043 (pyjwt), ft-052 (mistune) +- A release the registry withdrew is never the target (FX-H5): npm deprecated, ft-007 (lodash 4.18.0, "Bad release", so 4.18.1) and ft-063 (brace-expansion 3.0.3, so 3.0.4); crates.io yanked, ft-062 (h2 0.3.17, so 0.3.18); every clearing release withdrawn, so no target and `withheld` names it, ft-064 (by hand) ### Refresh verdict — AD-055 ¶1, ¶2, ¶3 @@ -181,7 +195,9 @@ form. - golang.org/x/net (Go), including `v0` -> v0.60.0 / v0.61.0: rc-023..rc-028 The release-age hold and withdrawn or deprecated releases are not part of -these vectors. They depend on the clock and on registry metadata. +these `range_check` vectors. They depend on the clock and on registry +metadata. (Withdrawn releases as fix targets are pinned in `fix_target.jsonl` +with the registry state written into the line, above.) ## `engines_skip.json` — the shrink-only gate diff --git a/src-tauri/tests/conformance/vectors/fix_target.jsonl b/src-tauri/tests/conformance/vectors/fix_target.jsonl index be0e16713..1f78d186f 100644 --- a/src-tauri/tests/conformance/vectors/fix_target.jsonl +++ b/src-tauri/tests/conformance/vectors/fix_target.jsonl @@ -4,7 +4,7 @@ {"id":"ft-004","rule":"AD-055 ¶5: a last_affected window with no later fix declines a clean claim","source":"fix-path oracle 2026-10-10 (applied, re-resolved, re-scanned; target minimal vs OSV /v1/query per published version); re-checked 2026-10-12 against the pinned record ids; case U04-braces-pnpm-refresh","eco":"npm","name":"braces","installed":"3.0.2","record_ids":["GHSA-cwfw-4gq5-mrqx","GHSA-grv7-fg5c-xmjg","GHSA-vfj7-8cjw-p6xm"],"target":"3.0.3","clears":false,"evidence":{"osv_ids_at_installed":["GHSA-grv7-fg5c-xmjg","GHSA-vfj7-8cjw-p6xm"],"osv_ids_at_target":["GHSA-vfj7-8cjw-p6xm"]}} {"id":"ft-005","rule":"AD-055 ¶5: the smallest version no known advisory affects","source":"fix-path oracle 2026-10-10 (applied, re-resolved, re-scanned; target minimal vs OSV /v1/query per published version); re-checked 2026-10-12 against the pinned record ids; case U05-mio-cargo-refresh","eco":"crates.io","name":"mio","installed":"0.8.0","record_ids":["GHSA-pf3p-x6qj-6j7q","GHSA-r8w9-5wcg-vfj7","RUSTSEC-2020-0081","RUSTSEC-2024-0019"],"target":"0.8.11","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-r8w9-5wcg-vfj7","RUSTSEC-2024-0019"],"osv_ids_at_target":[]}} {"id":"ft-006","rule":"AD-055 ¶5: the smallest version no known advisory affects","source":"fix-path oracle 2026-10-10 (applied, re-resolved, re-scanned; target minimal vs OSV /v1/query per published version); re-checked 2026-10-12 against the pinned record ids; case U06-openssl-cargo-refresh","eco":"crates.io","name":"openssl","installed":"0.10.38","record_ids":["GHSA-34p9-f4q3-c4r7","GHSA-3gxf-9r58-2ghg","GHSA-4fcv-w3qc-ppgg","GHSA-6hcf-g6gr-hhcr","GHSA-8c75-8mhr-p7r9","GHSA-9qwg-crg9-m2vc","GHSA-ghm9-cr32-g9qj","GHSA-hppc-g8h3-xhp3","GHSA-phqj-4mhp-q6mq","GHSA-pqf5-4pqq-29f5","GHSA-q445-7m23-qrmw","GHSA-rpmj-rpgj-qmpm","GHSA-xcf7-rvmh-g6q4","GHSA-xjxc-vfw2-cg96","GHSA-xmgf-hq76-4vx2","GHSA-xp3w-r5p5-63rr","GHSA-xphf-cx8h-7q9g","GHSA-xv59-967r-8726","RUSTSEC-2016-0001","RUSTSEC-2018-0010","RUSTSEC-2023-0022","RUSTSEC-2023-0023","RUSTSEC-2023-0024","RUSTSEC-2023-0044","RUSTSEC-2023-0072","RUSTSEC-2024-0357","RUSTSEC-2025-0004","RUSTSEC-2025-0022"],"target":"0.10.80","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-3gxf-9r58-2ghg","GHSA-6hcf-g6gr-hhcr","GHSA-8c75-8mhr-p7r9","GHSA-9qwg-crg9-m2vc","GHSA-hppc-g8h3-xhp3","GHSA-pqf5-4pqq-29f5","GHSA-q445-7m23-qrmw","GHSA-rpmj-rpgj-qmpm","GHSA-xcf7-rvmh-g6q4","GHSA-xmgf-hq76-4vx2","GHSA-xp3w-r5p5-63rr","GHSA-xphf-cx8h-7q9g","GHSA-xv59-967r-8726","RUSTSEC-2023-0022","RUSTSEC-2023-0023","RUSTSEC-2023-0024","RUSTSEC-2023-0044","RUSTSEC-2023-0072","RUSTSEC-2024-0357","RUSTSEC-2025-0004"],"osv_ids_at_target":[]}} -{"id":"ft-007","rule":"AD-055 ¶5: the smallest version no known advisory affects","source":"fix-path oracle 2026-10-10 (applied, re-resolved, re-scanned; target minimal vs OSV /v1/query per published version); re-checked 2026-10-12 against the pinned record ids; case U07-lodash-pnpm-bump","eco":"npm","name":"lodash","installed":"4.17.21","record_ids":["GHSA-29mw-wpgm-hmr9","GHSA-35jh-r3h4-6jhm","GHSA-4xc9-xhrj-v574","GHSA-f23m-r3pf-42rh","GHSA-fvqr-27wr-82fm","GHSA-jf85-cpcp-j695","GHSA-p6mc-m468-83gw","GHSA-r5fr-rjxr-66jc","GHSA-x5rq-j2xg-h7qm","GHSA-xxjr-mmjv-4gpg"],"target":"4.18.0","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-f23m-r3pf-42rh","GHSA-r5fr-rjxr-66jc","GHSA-xxjr-mmjv-4gpg"],"osv_ids_at_target":[]}} +{"id":"ft-007","rule":"AD-055 ¶5 + FX-H5: the smallest release no known advisory affects that its registry has not withdrawn (crates.io/PyPI yanked, npm deprecated)","source":"fix-path oracle 2026-10-10 (applied, re-resolved, re-scanned; target minimal vs OSV /v1/query per published version); re-checked 2026-10-12 against the pinned record ids; case U07-lodash-pnpm-bump; FX-H5 2026-10-16: npm marks 4.18.0 deprecated (\"Bad release. Please use lodash@4.17.21 instead.\", abbreviated packument); 4.18.1 is the next release, not deprecated, and OSV /v1/query at 4.18.1 returns none of the pinned ids","eco":"npm","name":"lodash","installed":"4.17.21","record_ids":["GHSA-29mw-wpgm-hmr9","GHSA-35jh-r3h4-6jhm","GHSA-4xc9-xhrj-v574","GHSA-f23m-r3pf-42rh","GHSA-fvqr-27wr-82fm","GHSA-jf85-cpcp-j695","GHSA-p6mc-m468-83gw","GHSA-r5fr-rjxr-66jc","GHSA-x5rq-j2xg-h7qm","GHSA-xxjr-mmjv-4gpg"],"target":"4.18.1","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-f23m-r3pf-42rh","GHSA-r5fr-rjxr-66jc","GHSA-xxjr-mmjv-4gpg"],"osv_ids_at_target":[]},"published":["4.17.23","4.18.0","4.18.1"],"excluded":["4.18.0"]} {"id":"ft-008","rule":"AD-055 ¶5: the smallest version no known advisory affects","source":"fix-path oracle 2026-10-10 (applied, re-resolved, re-scanned; target minimal vs OSV /v1/query per published version); re-checked 2026-10-12 against the pinned record ids; case U08-nanoid-pnpm-refresh","eco":"npm","name":"nanoid","installed":"3.3.4","record_ids":["GHSA-28wg-ghj8-5hjv","GHSA-2v37-7h3g-55p8","GHSA-mwcw-c2x4-8c55","GHSA-qrpm-p2h7-hrv2","GHSA-xwg4-73v4-xw9w"],"target":"3.3.18","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-28wg-ghj8-5hjv","GHSA-2v37-7h3g-55p8","GHSA-mwcw-c2x4-8c55","GHSA-xwg4-73v4-xw9w"],"osv_ids_at_target":[]}} {"id":"ft-009","rule":"AD-055 ¶5: the smallest version no known advisory affects","source":"fix-path oracle 2026-10-10 (applied, re-resolved, re-scanned; target minimal vs OSV /v1/query per published version); re-checked 2026-10-12 against the pinned record ids; case U09b-jsonwebtoken-pnpm-upstream","eco":"npm","name":"jsonwebtoken","installed":"8.5.1","record_ids":["GHSA-8cf7-32gw-wr33","GHSA-c7hr-j4mj-j2w6","GHSA-hjrf-2m68-5959","GHSA-qwph-4952-7xr6"],"target":"9.0.0","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-8cf7-32gw-wr33","GHSA-hjrf-2m68-5959","GHSA-qwph-4952-7xr6"],"osv_ids_at_target":[]}} {"id":"ft-010","rule":"AD-055 ¶5: the smallest version no known advisory affects","source":"fix-path oracle 2026-10-10 (applied, re-resolved, re-scanned; target minimal vs OSV /v1/query per published version); re-checked 2026-10-12 against the pinned record ids; case U10-picomatch-pnpm-refresh","eco":"npm","name":"picomatch","installed":"2.3.1","record_ids":["GHSA-3v7f-55p6-f55p","GHSA-c2c7-rcm5-vvqj"],"target":"2.3.2","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-3v7f-55p6-f55p","GHSA-c2c7-rcm5-vvqj"],"osv_ids_at_target":[]}} @@ -59,3 +59,6 @@ {"id":"ft-059","rule":"AD-055 ¶5: the smallest version no known advisory affects","source":"OSV /v1/querybatch per published stable release (2026-10-12), restricted to the pinned record ids: smallest release above `installed` that no advisory a later release fixes affects","eco":"Go","name":"google.golang.org/protobuf","installed":"1.28.0","record_ids":["GHSA-8r3f-844c-mc37","GHSA-hw7c-3rfg-p46j","GO-2023-1631","GO-2024-2611"],"target":"1.33.0","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-8r3f-844c-mc37","GO-2024-2611"],"osv_ids_at_target":[],"stable_releases_checked":26}} {"id":"ft-060","rule":"AD-055 ¶5: a window with no later fix declines a clean claim; the target clears the advisories that have a fix","source":"OSV /v1/querybatch per published stable release (2026-10-12), restricted to the pinned record ids: smallest release above `installed` that no advisory a later release fixes affects","eco":"Go","name":"github.com/jackc/pgx/v4","installed":"4.14.0","record_ids":["GHSA-j88v-2chj-qfwx","GHSA-m7wr-2xf7-cm9p","GHSA-mrww-27vc-gghv","GO-2024-2605","GO-2024-2606","GO-2026-5004"],"target":"4.18.2","clears":false,"evidence":{"osv_ids_at_installed":["GHSA-j88v-2chj-qfwx","GHSA-m7wr-2xf7-cm9p","GHSA-mrww-27vc-gghv","GO-2024-2605","GO-2024-2606","GO-2026-5004"],"osv_ids_at_target":["GHSA-j88v-2chj-qfwx","GO-2026-5004"],"stable_releases_checked":11}} {"id":"ft-061","rule":"AD-055 ¶5: a window with no later fix declines a clean claim; the target clears the advisories that have a fix","source":"OSV /v1/querybatch per published stable release (2026-10-12), restricted to the pinned record ids: smallest release above `installed` that no advisory a later release fixes affects","eco":"Go","name":"github.com/docker/docker","installed":"20.10.7+incompatible","record_ids":["GHSA-232p-vwff-86mp","GHSA-2mm7-x5h6-5pvq","GHSA-33pg-m6jh-5237","GHSA-3fwx-pjgw-3558","GHSA-44gg-pmqr-4669","GHSA-4vq8-7jfc-9cvp","GHSA-5qgp-p5jc-w2rm","GHSA-6hwg-w5jg-9c6x","GHSA-6wrf-mxfj-pf5p","GHSA-8fvr-5rqf-3wwh","GHSA-8w94-cf6g-c8mg","GHSA-997c-fj8j-rq5h","GHSA-g44j-7vp3-68cv","GHSA-g7v2-2qxx-wjrw","GHSA-j249-ghv5-7mxv","GHSA-jq35-85cj-fj4p","GHSA-mq39-4gv4-mvpx","GHSA-pxq6-2prw-chj9","GHSA-qmmc-jppf-32wv","GHSA-qrqr-3x5j-2xw9","GHSA-rc4r-wh2q-q6c4","GHSA-rg2x-37c3-w2rh","GHSA-v23v-6jw2-98fq","GHSA-v2cv-wwxq-qq97","GHSA-v4h8-794j-g8mm","GHSA-v994-f8vw-g7j4","GHSA-vj3f-3286-r4pf","GHSA-vp35-85q5-9f25","GHSA-vp62-88p7-qqf5","GHSA-wxj3-qwv4-cvfm","GHSA-x4rx-4gw3-53p4","GHSA-x84c-p2g9-rqv9","GHSA-x86f-5xw2-fm2r","GHSA-xmmx-7jpf-fx42","GHSA-xw73-rw38-6vjc","GO-2022-0390","GO-2022-0625","GO-2022-0630","GO-2022-0636","GO-2022-0638","GO-2022-0640","GO-2022-0647","GO-2022-0649","GO-2022-0705","GO-2022-0708","GO-2022-0751","GO-2022-0752","GO-2022-0985","GO-2022-1107","GO-2023-1699","GO-2023-1700","GO-2023-1701","GO-2024-2500","GO-2024-2512","GO-2024-2521","GO-2024-2659","GO-2024-2737","GO-2024-2913","GO-2024-2914","GO-2024-3005","GO-2025-3829","GO-2025-3830","GO-2026-4883","GO-2026-4887","GO-2026-5617","GO-2026-5668","GO-2026-5746"],"target":"25.0.13+incompatible","clears":false,"evidence":{"osv_ids_at_installed":["GHSA-232p-vwff-86mp","GHSA-2mm7-x5h6-5pvq","GHSA-33pg-m6jh-5237","GHSA-3fwx-pjgw-3558","GHSA-4vq8-7jfc-9cvp","GHSA-6wrf-mxfj-pf5p","GHSA-jq35-85cj-fj4p","GHSA-mq39-4gv4-mvpx","GHSA-pxq6-2prw-chj9","GHSA-rc4r-wh2q-q6c4","GHSA-rg2x-37c3-w2rh","GHSA-v23v-6jw2-98fq","GHSA-v994-f8vw-g7j4","GHSA-vp35-85q5-9f25","GHSA-vp62-88p7-qqf5","GHSA-x86f-5xw2-fm2r","GHSA-xmmx-7jpf-fx42","GHSA-xw73-rw38-6vjc","GO-2022-0390","GO-2022-0985","GO-2022-1107","GO-2023-1699","GO-2023-1700","GO-2023-1701","GO-2024-2500","GO-2024-2512","GO-2024-2913","GO-2024-2914","GO-2024-3005","GO-2025-3829","GO-2026-4883","GO-2026-4887","GO-2026-5617","GO-2026-5668","GO-2026-5746"],"osv_ids_at_target":["GHSA-pxq6-2prw-chj9","GHSA-rg2x-37c3-w2rh","GHSA-vp62-88p7-qqf5","GHSA-x86f-5xw2-fm2r","GO-2026-4883","GO-2026-4887","GO-2026-5617","GO-2026-5668","GO-2026-5746"],"stable_releases_checked":110}} +{"id":"ft-062","rule":"AD-055 ¶5 + FX-H5: the smallest release no known advisory affects that its registry has not withdrawn (crates.io/PyPI yanked, npm deprecated)","source":"crates.io /api/v1/crates/h2/versions (2026-10-16): 0.3.17 is yanked, the only yanked release above 0.3.16; OSV /v1/query (2026-10-16) at 0.3.16 returns both pinned ids, at 0.3.18 neither. Restricted to the pinned record ids (RUSTSEC-2023-0034 and its GHSA twin, both fixed 0.3.17)","eco":"crates.io","name":"h2","installed":"0.3.16","record_ids":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"published":["0.3.17","0.3.18","0.3.19","0.3.20","0.3.21","0.3.22","0.3.23","0.3.24","0.3.25","0.3.26","0.3.27","0.4.0","0.4.1","0.4.2","0.4.3","0.4.4","0.4.5","0.4.6","0.4.7","0.4.8","0.4.9","0.4.10","0.4.11","0.4.12","0.4.13","0.4.14","0.4.15","0.4.16","0.4.17","0.4.18","0.4.19","0.4.20"],"excluded":["0.3.17"],"target":"0.3.18","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"osv_ids_at_target":[]}} +{"id":"ft-063","rule":"AD-055 ¶5 + FX-H5: the smallest release no known advisory affects that its registry has not withdrawn (crates.io/PyPI yanked, npm deprecated)","source":"npm abbreviated packument for brace-expansion (2026-10-16): 3.0.3 is deprecated (\"contains bloat files\"), the only deprecated release above 3.0.2, and 5.0.12 (latest) is not; OSV /v1/query (2026-10-16) at 3.0.2 returns GHSA-mh99-v99m-4gvg (fixed 3.0.3 on the 3.x line), at 3.0.4 it does not. Restricted to that pinned record","eco":"npm","name":"brace-expansion","installed":"3.0.2","record_ids":["GHSA-mh99-v99m-4gvg"],"published":["3.0.3","3.0.4","3.0.5","3.0.6","3.0.7","3.0.8","3.0.9","4.0.0","4.0.1","5.0.2","5.0.3","5.0.4","5.0.5","5.0.6","5.0.7","5.0.8","5.0.9","5.0.10","5.0.11","5.0.12"],"excluded":["3.0.3"],"target":"3.0.4","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-mh99-v99m-4gvg"],"osv_ids_at_target":[]}} +{"id":"ft-064","rule":"AD-055 ¶5 + FX-H5: the smallest release no known advisory affects that its registry has not withdrawn (crates.io/PyPI yanked, npm deprecated); when every release that clears an advisory is withdrawn there is no target, and the withdrawn ones are named","source":"by hand from ft-062: the same pinned records with a stated registry where 0.3.17 is the last release (published/excluded are given, not observed). Every exit is withdrawn, so no target; `withheld` names 0.3.17","eco":"crates.io","name":"h2","installed":"0.3.16","record_ids":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"published":["0.3.17"],"excluded":["0.3.17"],"target":null,"clears":false,"withheld":["0.3.17"],"evidence":{"osv_ids_at_installed":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"osv_ids_at_target":[]}} From 1562c31364a0ca88db999c8ee40d2a65624199d0 Mon Sep 17 00:00:00 2001 From: antony Date: Sat, 17 Oct 2026 10:05:57 +1000 Subject: [PATCH 2/3] fix(fix-target): npm deprecation excludes only a superseded release; review of #976 B1: npm `deprecated` is often a whole-major "please upgrade" notice, not a withdrawn release (glob deprecates 160 of 168 versions; tar every 6.x). Excluding them all sent glob 10.4.5 (GHSA-5j98-mcp5-4vw2) to 12.0.0 and tar 6.1.11 to 7.5.8. A deprecated npm release is now excluded only when a later, non-deprecated, stable release exists on its caret line (same major; same minor for 0.x); a package whose `latest` is deprecated excludes nothing. lodash 4.18.0 and brace-expansion 3.0.3 stay excluded. New vectors ft-065 (glob 10.4.5 -> 10.5.0) and ft-066 (tar 6.1.11 -> 6.2.1) from live data, 2026-10-17. S2: morning audit (G4) accepts a version the stored registry facts list, so a registry-only target such as lodash 4.18.1 is not a version fault. S3: each registry is read only while its source (npm_registry, crates_io, pypi) is enabled; NETWORK.md says so. S4: only crates.io requests are paced (1.1 s apart). S5: 90 s request budget for large documents (next ~26 MB, boto3 ~3.3 MB). Nits: no default client on a builder error; OSV live feed text annotates a withdrawn "Fixed in" release with the one to use; the security verdict reads registry facts once per verdict. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7 --- NETWORK.md | 10 +- src-tauri/src/morning_audit.rs | 5 +- src-tauri/src/osv/release_facts.rs | 149 ++++++++++++++---- src-tauri/src/osv/release_facts_tests.rs | 119 +++++++++++++- src-tauri/src/scoring/security_verdict.rs | 12 +- src-tauri/src/sources/osv_live.rs | 71 ++++++++- src-tauri/tests/conformance/vectors/README.md | 11 +- .../conformance/vectors/fix_target.jsonl | 2 + 8 files changed, 339 insertions(+), 40 deletions(-) diff --git a/NETWORK.md b/NETWORK.md index ad044ce9a..08b7f91f5 100644 --- a/NETWORK.md +++ b/NETWORK.md @@ -283,16 +283,18 @@ module proxy (§1b, §1e) receive them too; §4 has the complete list. | `api.osv.dev` | `POST /v1/querybatch` | Your discovered package names + ecosystems, batched (≤1000/batch) | | `osv-vulnerabilities.storage.googleapis.com` | `GET /{ecosystem}/all.zip` | Plain `GET` — downloads the public advisory mirror (no user data) | | `registry.npmjs.org` | `GET /{package}` (abbreviated packument, `Accept: application/vnd.npm.install-v1+json`) | The package name only | -| `crates.io` | `GET /api/v1/crates/{crate}` | The crate name only. User-Agent `4DA-Developer-OS/1.0 (https://4da.ai)`, 1.1 s apart | +| `crates.io` | `GET /api/v1/crates/{crate}` | The crate name only. User-Agent `4DA-Developer-OS/1.0 (https://4da.ai)` | | `pypi.org` | `GET /pypi/{package}/json` | The package name only | - **Registry release check** (the last three rows, `src-tauri/src/osv/release_facts.rs`): after each sync, for packages that have a **version-confirmed advisory** only, 4DA reads which releases exist and which the registry withdrew (crates.io and PyPI *yanked*, npm *deprecated*), so a fix target never names one. At most 60 packages per sync, each re-read at most once a day; the answer is - kept in the local database. **Your installed version is never sent.** Go modules are not - checked (their `retract` directives are not read). Offline, targets come from the advisory data - alone, as before. + kept in the local database. **Your installed version is never sent.** Each registry is read only + while its source (npm / crates.io / PyPI, Settings → Sources) is enabled; a disabled source makes + no call here either. Requests to crates.io are spaced 1.1 s apart. Go modules are not checked + (their `retract` directives are not read). Offline, targets come from the advisory data alone, + as before. - **Trigger:** background, at startup, **only if dependencies exist** and the local mirror was last synced **> 6 hours ago** (`app_setup.rs`). - **Disable:** if 4DA has discovered no dependencies, nothing is sent. Removing/disabling ACE project diff --git a/src-tauri/src/morning_audit.rs b/src-tauri/src/morning_audit.rs index 5a321a5ed..4ea8df4f9 100644 --- a/src-tauri/src/morning_audit.rs +++ b/src-tauri/src/morning_audit.rs @@ -55,7 +55,8 @@ impl MorningAudit { } /// Is `version` a real version of `package`: installed in some project, a -/// fixed version OSV lists for it, or a release the registry feed carries? +/// fixed version OSV lists for it, a release the registry feed carries, or a +/// release the stored registry facts list (`osv::release_facts`)? /// The G4 replay's check, shared so the gate and the replay agree. pub(crate) fn version_is_true(conn: &Connection, package: &str, version: &str) -> bool { let q = |sql: &str| -> bool { @@ -66,6 +67,8 @@ pub(crate) fn version_is_true(conn: &Connection, package: &str, version: &str) - || q("SELECT 1 FROM osv_advisories WHERE LOWER(package_name) = LOWER(?1) AND fixed_versions LIKE '%\"' || ?2 || '\"%' LIMIT 1") || q("SELECT 1 FROM source_items WHERE title LIKE '%' || ?1 || ' v' || ?2 || '%' LIMIT 1") + // A release the registry lists (FX-H5: a target past a withdrawn fix). + || crate::osv::release_facts::registry_lists(conn, package, version) } /// Every `(package, version, what)` the facts carry, in render order. diff --git a/src-tauri/src/osv/release_facts.rs b/src-tauri/src/osv/release_facts.rs index 90255914c..1a9995e91 100644 --- a/src-tauri/src/osv/release_facts.rs +++ b/src-tauri/src/osv/release_facts.rs @@ -11,10 +11,18 @@ //! unless pinned exactly), from `/pypi/{package}/json`. //! - **npm:** `deprecated` on the version, from the abbreviated packument //! (`GET /{package}`, `Accept: application/vnd.npm.install-v1+json`), but -//! only while the package's `latest` release is NOT deprecated. When `latest` -//! is deprecated too, the whole package is retired ("no longer supported"), -//! not one release; excluding every release would hide the version that -//! clears the advisory, so nothing is excluded then. +//! only when a LATER, non-deprecated, stable release exists in the same +//! caret range (same major; same minor for `0.x`). npm's `deprecated` is +//! often a whole-major "please upgrade" notice, not a withdrawn release: +//! glob deprecates 160 of 168 versions ("Old versions of glob are not +//! supported"), 10.5.0 included, and 12.0.0 is its first clean release. +//! Excluding them all sent glob 10.4.5 (GHSA-5j98-mcp5-4vw2, fixed 10.5.0) +//! to 12.0.0, a major jump that turned "a lockfile refresh is enough" into +//! "the parent must upgrade" (review of #976, 2026-10-17). lodash 4.18.0 is +//! still excluded: 4.18.1 is clean in 4.x. A package whose `latest` is +//! deprecated (retired whole) excludes nothing. +//! - Each ecosystem is read only while its registry source (`npm_registry`, +//! `crates_io`, `pypi`) is enabled: a disabled source makes no calls. //! - **Go:** not covered. A module's `retract` directives live in the //! `go.mod` of its newest version, which 4DA does not read; Go targets are //! computed from the advisory data alone, as before. @@ -33,6 +41,7 @@ use serde::{Deserialize, Serialize}; use tracing::{debug, info, warn}; use crate::db::Database; +use crate::osv::matching::parse_version; /// `kv_store` key holding every package's facts as one JSON document. const KV_KEY: &str = "osv_registry_releases"; @@ -44,6 +53,10 @@ const KEEP_DAYS: i64 = 30; const MAX_FETCHES_PER_SYNC: usize = 60; /// Consecutive failures that end a pass (offline, or a registry down). const MAX_FAILURES: usize = 3; +/// Whole-request budget; see `fetch_all` for the document sizes behind it. +const REQUEST_TIMEOUT_SECS: u64 = 90; +/// Gap between two crates.io reads (its crawler policy: one per second). +const CRATES_PAUSE_MS: u64 = 1100; const USER_AGENT: &str = "4DA-Developer-OS/1.0 (https://4da.ai)"; const NPM_ABBREVIATED: &str = "application/vnd.npm.install-v1+json"; @@ -93,8 +106,19 @@ fn covered(ecosystem: &str) -> Option<&'static str> { } } +/// The registry source whose toggle governs reads for an OSV ecosystem. +fn source_type(ecosystem: &str) -> Option<&'static str> { + match ecosystem { + "npm" => Some("npm_registry"), + "crates.io" => Some("crates_io"), + "PyPI" => Some("pypi"), + _ => None, + } +} + /// (OSV ecosystem, name) of every covered package with a version-confirmed -/// advisory: the only packages whose targets the facts can change. +/// advisory whose registry source is enabled: the only packages whose +/// targets the facts can change, and the only registries 4DA may contact. fn wanted(db: &Database) -> Vec<(String, String)> { let matches = super::matching::get_matched_advisories(db).unwrap_or_default(); let mut out: BTreeSet<(String, String)> = BTreeSet::new(); @@ -103,7 +127,42 @@ fn wanted(db: &Database) -> Vec<(String, String)> { out.insert((eco.to_string(), m.package_name.clone())); } } - out.into_iter().collect() + gate(out.into_iter().collect(), |source| { + db.is_source_enabled(source) + }) +} + +/// Drop the packages whose registry source is disabled. +fn gate(wanted: Vec<(String, String)>, enabled: impl Fn(&str) -> bool) -> Vec<(String, String)> { + wanted + .into_iter() + .filter(|(eco, _)| source_type(eco).is_some_and(&enabled)) + .collect() +} + +/// Whether the stored registry facts list `version` as a published release +/// of `package` (any covered ecosystem). Read straight from `kv_store` on +/// `conn`, for the morning audit (G4): a target only the registry names, +/// such as lodash 4.18.1, is a real version. +pub(crate) fn registry_lists(conn: &rusqlite::Connection, package: &str, version: &str) -> bool { + let text: Option = conn + .query_row( + "SELECT CAST(value AS TEXT) FROM kv_store WHERE key = ?1", + rusqlite::params![KV_KEY], + |row| row.get(0), + ) + .ok(); + let stored: Vec = text + .and_then(|t| serde_json::from_str(&t).ok()) + .unwrap_or_default(); + stored.iter().any(|p| { + (p.name.eq_ignore_ascii_case(package) + || p.key == super::matching::package_key(package, &p.ecosystem)) + && p.releases + .published + .iter() + .any(|v| v.trim() == version.trim()) + }) } /// Fetch facts for the packages that need them (missing or older than a @@ -162,14 +221,33 @@ async fn fetch_all( due: Vec<(String, String)>, stored: &mut HashMap<(String, String), StoredPackage>, ) -> usize { - let client = reqwest::Client::builder() + // Large documents are normal here: `next`'s abbreviated packument is + // ~26 MB and boto3's PyPI JSON ~3.3 MB (2026-10-17), so the budget is + // per-request generous rather than the 30 s the small reads use. + let client = match reqwest::Client::builder() .user_agent(USER_AGENT) - .timeout(std::time::Duration::from_secs(30)) + .connect_timeout(std::time::Duration::from_secs(10)) + .timeout(std::time::Duration::from_secs(REQUEST_TIMEOUT_SECS)) .redirect(crate::http_client::ssrf_guarded_redirect_policy()) .build() - .unwrap_or_default(); + { + Ok(client) => client, + Err(e) => { + warn!(target: "4da::osv", error = %e, "registry release facts: no HTTP client"); + return 0; + } + }; let (mut fetched, mut failures) = (0usize, 0usize); + let mut crates_before = false; for (eco, name) in due { + // crates.io asks for at most one request per second; npm and PyPI + // are not paced. + if eco == "crates.io" { + if crates_before { + tokio::time::sleep(std::time::Duration::from_millis(CRATES_PAUSE_MS)).await; + } + crates_before = true; + } match fetch(&client, &eco, &name).await { Ok(releases) => { failures = 0; @@ -195,8 +273,6 @@ async fn fetch_all( } } } - // crates.io asks for at most one request per second. - tokio::time::sleep(std::time::Duration::from_millis(1100)).await; } fetched } @@ -239,31 +315,50 @@ async fn fetch( releases.ok_or_else(|| "unreadable registry answer".to_string()) } -/// npm abbreviated packument: every version, and the deprecated ones unless -/// `latest` is deprecated too (a retired package, see the module notes). +fn npm_deprecated(manifest: &serde_json::Value) -> bool { + manifest + .get("deprecated") + .is_some_and(|d| d.as_str().is_some_and(|s| !s.is_empty()) || d.as_bool() == Some(true)) +} + +/// Same caret range (`^a` admits `b`'s line): same major, and for `0.x` the +/// same minor. +fn same_caret_line(a: &semver::Version, b: &semver::Version) -> bool { + a.major == b.major && (a.major > 0 || a.minor == b.minor) +} + +/// npm abbreviated packument: every version, and a deprecated one excluded +/// only when a later, non-deprecated, stable release exists on its caret line +/// (see the module notes). A package whose `latest` is deprecated (retired +/// whole) excludes nothing. fn parse_npm(doc: &serde_json::Value) -> Option { let versions = doc.get("versions")?.as_object()?; - let deprecated = |v: &serde_json::Value| { - v.get("deprecated") - .is_some_and(|d| d.as_str().is_some_and(|s| !s.is_empty()) || d.as_bool() == Some(true)) - }; let retired = doc .get("dist-tags") .and_then(|t| t.get("latest")) .and_then(|l| l.as_str()) .and_then(|l| versions.get(l)) - .is_some_and(deprecated); + .is_some_and(npm_deprecated); + let clean: Vec = versions + .iter() + .filter(|(_, m)| !npm_deprecated(m)) + .filter_map(|(v, _)| parse_version(v)) + .filter(|v| v.pre.is_empty()) + .collect(); + let superseded = |v: &semver::Version| clean.iter().any(|c| same_caret_line(v, c) && c > v); + let excluded = if retired { + Vec::new() + } else { + versions + .iter() + .filter(|(_, m)| npm_deprecated(m)) + .filter(|(v, _)| parse_version(v).is_some_and(|p| superseded(&p))) + .map(|(v, _)| v.clone()) + .collect() + }; Some(RegistryReleases { published: versions.keys().cloned().collect(), - excluded: if retired { - Vec::new() - } else { - versions - .iter() - .filter(|(_, v)| deprecated(v)) - .map(|(k, _)| k.clone()) - .collect() - }, + excluded, }) } diff --git a/src-tauri/src/osv/release_facts_tests.rs b/src-tauri/src/osv/release_facts_tests.rs index de7b900b2..7340a4cb9 100644 --- a/src-tauri/src/osv/release_facts_tests.rs +++ b/src-tauri/src/osv/release_facts_tests.rs @@ -1,5 +1,5 @@ // SPDX-License-Identifier: FSL-1.1-Apache-2.0 -//! Registry answers trimmed from the live registries (2026-10-16). +//! Registry answers trimmed from the live registries (2026-10-16/17). use serde_json::json; @@ -37,6 +37,123 @@ fn npm_excludes_nothing_when_the_whole_package_is_retired() { assert!(parse_npm(&json!({"error": "not found"})).is_none()); } +const OLD_GLOB: &str = "Old versions of glob are not supported, and contain widely publicized \ + security vulnerabilities, which have been fixed in the current version."; + +/// Review of #976 (2026-10-17): glob deprecates every release below 12 with +/// one "please upgrade" notice. That is not a withdrawn release: 10.5.0 (the +/// fix for GHSA-5j98-mcp5-4vw2 on the 10.x line) must stay a target. +#[test] +fn npm_keeps_a_release_whose_whole_major_is_deprecated() { + let doc = json!({ + "dist-tags": {"latest": "13.0.6"}, + "versions": { + "10.4.5": {"deprecated": OLD_GLOB}, + "10.5.0": {"deprecated": OLD_GLOB}, + "11.1.0": {"deprecated": OLD_GLOB}, + "12.0.0": {}, + "13.0.6": {} + } + }); + let r = parse_npm(&doc).unwrap_or_default(); + assert!(r.excluded.is_empty(), "{:?}", r.excluded); +} + +/// tar deprecates 7.0.0..7.5.7 and every 6.x; 7.5.8 is clean. Only the 7.x +/// releases with a clean later 7.x are withdrawn; 6.2.1 stays. +#[test] +fn npm_excludes_a_deprecated_release_only_when_its_line_has_a_clean_successor() { + let doc = json!({ + "dist-tags": {"latest": "7.5.22"}, + "versions": { + "6.2.1": {"deprecated": "Old versions of tar are not supported"}, + "7.5.7": {"deprecated": "Old versions of tar are not supported"}, + "7.5.8": {}, + "7.5.22": {}, + "8.0.0-rc.1": {} + } + }); + let r = parse_npm(&doc).unwrap_or_default(); + assert_eq!(r.excluded, ["7.5.7"]); +} + +#[test] +fn npm_caret_line_for_0x_is_the_minor_and_prereleases_do_not_count() { + let doc = json!({ + "dist-tags": {"latest": "0.3.0"}, + "versions": { + "0.2.1": {"deprecated": "bad"}, + "0.2.2-beta.1": {}, + "0.3.0": {}, + "0.4.1": {"deprecated": "bad"}, + "0.4.2": {} + } + }); + let r = parse_npm(&doc).unwrap_or_default(); + // 0.2.1 has no clean stable 0.2.x after it (0.3.0 is another caret line, + // 0.2.2-beta.1 a pre-release); 0.4.1 has 0.4.2. + assert_eq!(r.excluded, ["0.4.1"]); +} + +#[test] +fn a_disabled_registry_source_is_never_contacted() { + let wanted = vec![ + ("npm".to_string(), "lodash".to_string()), + ("crates.io".to_string(), "h2".to_string()), + ("PyPI".to_string(), "jinja2".to_string()), + ]; + let kept = gate(wanted.clone(), |s| s != "npm_registry"); + assert_eq!(kept, wanted[1..].to_vec()); + assert!(gate(wanted.clone(), |_| false).is_empty()); + assert_eq!(gate(wanted.clone(), |_| true), wanted); +} + +#[test] +fn a_disabled_source_row_gates_the_database_path_too() { + let db = crate::test_utils::test_db(); + { + let conn = db.conn.lock(); + conn.execute( + "INSERT INTO sources (source_type, name, enabled) VALUES ('crates_io', 'crates.io', 0) + ON CONFLICT(source_type) DO UPDATE SET enabled = 0", + [], + ) + .unwrap(); + } + let wanted = vec![ + ("crates.io".to_string(), "h2".to_string()), + ("npm".to_string(), "lodash".to_string()), + ]; + let kept = gate(wanted, |s| db.is_source_enabled(s)); + assert_eq!(kept, [("npm".to_string(), "lodash".to_string())]); +} + +/// Review of #976 (S2): the morning audit (G4) must accept a target only +/// the registry names (lodash 4.18.1 is in no advisory and no feed title). +#[test] +fn the_morning_audit_accepts_a_release_the_registry_lists() { + let db = crate::test_utils::test_db(); + let fetched_at = Utc::now().to_rfc3339(); + db.set_kv( + KV_KEY, + &format!( + r#"[{{"ecosystem":"npm","key":"lodash","name":"lodash","fetched_at":"{fetched_at}","published":["4.18.0","4.18.1"],"excluded":["4.18.0"]}}]"# + ), + ) + .unwrap(); + let conn = db.conn.lock(); + assert!(registry_lists(&conn, "lodash", "4.18.1")); + assert!(registry_lists(&conn, "Lodash", "4.18.1")); + assert!(!registry_lists(&conn, "lodash", "4.19.0")); + assert!(!registry_lists(&conn, "underscore", "4.18.1")); + assert!(crate::morning_audit::version_is_true( + &conn, "lodash", "4.18.1" + )); + assert!(!crate::morning_audit::version_is_true( + &conn, "lodash", "4.19.0" + )); +} + #[test] fn crates_excludes_yanked_versions() { // crates.io/api/v1/crates/h2: 0.3.17 is yanked. diff --git a/src-tauri/src/scoring/security_verdict.rs b/src-tauri/src/scoring/security_verdict.rs index 20429491e..6d85f84f4 100644 --- a/src-tauri/src/scoring/security_verdict.rs +++ b/src-tauri/src/scoring/security_verdict.rs @@ -291,7 +291,10 @@ pub(crate) fn ranges_verdict(installed: Option<&str>, ranges: &[String]) -> Opti /// The advisory's highest listed fix, as a release its registry still offers /// (FX-H5): a yanked or deprecated fix gives way to the next allowed release /// the advisory does not affect, or to nothing. -fn highest_fix(db: &Database, advisory: &StoredAdvisory) -> Option { +fn highest_fix( + registry: &fourda_depengine::registry::RegistryIndex, + advisory: &StoredAdvisory, +) -> Option { let versions: Vec = serde_json::from_str(advisory.fixed_versions.as_deref()?).ok()?; let fix = versions .into_iter() @@ -304,7 +307,7 @@ fn highest_fix(db: &Database, advisory: &StoredAdvisory) -> Option { crate::osv::matching::package_key(&advisory.package_name, eco), eco.to_string(), ); - match crate::osv::release_facts::load(db).get(&key) { + match registry.get(&key) { Some(facts) => { fourda_depengine::fix_target::installable_fix( eco, @@ -336,6 +339,8 @@ pub(crate) fn matcher_verdict(db: &Database, ids: &[String]) -> Option = None; let mut any_tier: Option<&'static str> = None; + // Registry facts, read at most once per verdict (FX-H5). + let mut registry: Option = None; for row in &rows { let exposure = exposure_for_advisory(db, row); if exposure.copies_checked == 0 { @@ -352,7 +357,8 @@ pub(crate) fn matcher_verdict(db: &Database, ids: &[String]) -> Option = std::collections::HashSet::new(); let mut items = Vec::new(); + // Releases the registries withdrew (FX-H5), so "Fixed in" never reads as + // advice to take a yanked or deprecated release. + let registry = crate::osv::release_facts::load(db); for ((eco, name, ver), ids) in deps.iter().zip(per_dep_ids.iter()) { + let facts = registry.get(&(crate::osv::matching::package_key(name, eco), eco.clone())); for id in ids { let Some(vuln) = hydrated.get(id) else { continue; @@ -145,7 +149,11 @@ async fn query_live_version_matched( if !seen.insert((id.clone(), name.to_lowercase(), eco.clone())) { continue; } - items.push(live_vuln_to_grounded_item(vuln, name, eco, ver)); + let mut item = live_vuln_to_grounded_item(vuln, name, eco, ver); + if let Some(note) = withdrawn_note(vuln, name, eco, facts) { + item.content.push_str(¬e); + } + items.push(item); } } Ok(items) @@ -280,6 +288,41 @@ fn live_vuln_to_grounded_item( .with_metadata(metadata) } +/// When the registry withdrew the advisory's "Fixed in" release (yanked or +/// deprecated, FX-H5), the note appended to that line: the release to take +/// instead, or that none past it is known. `None` when it was not withdrawn +/// or no registry facts are held. +fn withdrawn_note( + vuln: &OsvVulnerability, + package: &str, + ecosystem: &str, + facts: Option<&fourda_depengine::registry::RegistryReleases>, +) -> Option { + let facts = facts?; + let fixed = fixed_version_for(vuln, package, ecosystem)?; + if !facts.excludes(ecosystem, &fixed) { + return None; + } + let ranges: Vec<&OsvRange> = vuln + .affected + .iter() + .flatten() + .filter(|a| { + a.package.as_ref().is_some_and(|p| { + p.name.eq_ignore_ascii_case(package) && p.ecosystem.eq_ignore_ascii_case(ecosystem) + }) + }) + .flat_map(|a| a.ranges.iter().flatten()) + .collect(); + let ranges_json = serde_json::to_string(&ranges).ok(); + let next = + fourda_depengine::fix_target::installable_fix(ecosystem, &fixed, &ranges_json, facts).fix; + Some(match next { + Some(next) => format!(" (withdrawn by the registry; use {next})"), + None => " (withdrawn by the registry; no later release is known to fix it)".to_string(), + }) +} + /// First concrete fixed version for `package` in this advisory (ECOSYSTEM/SEMVER ranges only; /// GIT commit hashes and "-NA" unknown bounds are skipped). fn fixed_version_for(vuln: &OsvVulnerability, package: &str, ecosystem: &str) -> Option { @@ -493,6 +536,32 @@ mod tests { ); } + /// FX-H5: "Fixed in: 4.18.0" for lodash, a release npm deprecated. + #[test] + fn a_withdrawn_fixed_in_release_is_annotated() { + let v = vuln_with(vec![OsvAffected { + package: Some(pkg("lodash", "npm")), + ranges: Some(vec![semver("0", "4.18.0")]), + }]); + let facts = fourda_depengine::registry::RegistryReleases { + published: vec!["4.18.0".into(), "4.18.1".into()], + excluded: vec!["4.18.0".into()], + }; + assert_eq!( + withdrawn_note(&v, "lodash", "npm", Some(&facts)).as_deref(), + Some(" (withdrawn by the registry; use 4.18.1)") + ); + let only = fourda_depengine::registry::RegistryReleases { + published: vec!["4.18.0".into()], + excluded: vec!["4.18.0".into()], + }; + assert!(withdrawn_note(&v, "lodash", "npm", Some(&only)) + .is_some_and(|n| n.contains("no later release"))); + let live = fourda_depengine::registry::RegistryReleases::default(); + assert_eq!(withdrawn_note(&v, "lodash", "npm", Some(&live)), None); + assert_eq!(withdrawn_note(&v, "lodash", "npm", None), None); + } + #[test] fn test_live_vuln_to_grounded_item_title_and_fields() { let v = vuln_with(vec![OsvAffected { diff --git a/src-tauri/tests/conformance/vectors/README.md b/src-tauri/tests/conformance/vectors/README.md index 1fe7bb2dc..125d6069f 100644 --- a/src-tauri/tests/conformance/vectors/README.md +++ b/src-tauri/tests/conformance/vectors/README.md @@ -54,8 +54,12 @@ Field conventions: registry listed them on the date in `source`. They join the releases the advisory data names, so the walk can step to a release no advisory mentions. - - `excluded[]`: releases the registry withdrew (crates.io and PyPI - `yanked`; npm `deprecated` while the package's `latest` is not). No + - `excluded[]`: releases the registry withdrew. crates.io and PyPI: + `yanked`. npm: `deprecated`, but only when a later, non-deprecated, + stable release exists on the same caret line (same major; same minor for + `0.x`), and never when the package's `latest` is deprecated; a + whole-major "please upgrade" notice is not a withdrawal. The line holds + the app's rule applied to the live registry on the `source` date. No target may be one; the walk steps over them as it steps over pre-releases. - `withheld[]` (expected, checked only when present): the withdrawn @@ -131,10 +135,11 @@ form. > fix"). - The 22 oracle cases: ft-001..ft-022 (openssl 0.10.80: ft-006; braces `last_affected`: ft-004) -- OSV per published release, all four registries: ft-023..ft-061, ft-062, ft-063 +- OSV per published release, all four registries: ft-023..ft-061, ft-062, ft-063, ft-065, ft-066 - No clean version, so the target clears only the fixed advisories (`clears: false`): ft-004, ft-029, ft-041, ft-042, ft-055, ft-060, ft-061 - A `last_affected` window that later releases leave (the walk steps past the bound and keeps checking every advisory): ft-043 (pyjwt), ft-052 (mistune) - A release the registry withdrew is never the target (FX-H5): npm deprecated, ft-007 (lodash 4.18.0, "Bad release", so 4.18.1) and ft-063 (brace-expansion 3.0.3, so 3.0.4); crates.io yanked, ft-062 (h2 0.3.17, so 0.3.18); every clearing release withdrawn, so no target and `withheld` names it, ft-064 (by hand) +- A whole-major npm deprecation notice is not a withdrawal, so the target stays on the installed line (review of #976): ft-065 (glob 10.4.5 -> 10.5.0, not 12.0.0), ft-066 (tar 6.1.11 -> 6.2.1, while tar 7.0.0..7.5.7 are excluded because 7.5.8 is clean) ### Refresh verdict — AD-055 ¶1, ¶2, ¶3 diff --git a/src-tauri/tests/conformance/vectors/fix_target.jsonl b/src-tauri/tests/conformance/vectors/fix_target.jsonl index 1f78d186f..2ace909db 100644 --- a/src-tauri/tests/conformance/vectors/fix_target.jsonl +++ b/src-tauri/tests/conformance/vectors/fix_target.jsonl @@ -62,3 +62,5 @@ {"id":"ft-062","rule":"AD-055 ¶5 + FX-H5: the smallest release no known advisory affects that its registry has not withdrawn (crates.io/PyPI yanked, npm deprecated)","source":"crates.io /api/v1/crates/h2/versions (2026-10-16): 0.3.17 is yanked, the only yanked release above 0.3.16; OSV /v1/query (2026-10-16) at 0.3.16 returns both pinned ids, at 0.3.18 neither. Restricted to the pinned record ids (RUSTSEC-2023-0034 and its GHSA twin, both fixed 0.3.17)","eco":"crates.io","name":"h2","installed":"0.3.16","record_ids":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"published":["0.3.17","0.3.18","0.3.19","0.3.20","0.3.21","0.3.22","0.3.23","0.3.24","0.3.25","0.3.26","0.3.27","0.4.0","0.4.1","0.4.2","0.4.3","0.4.4","0.4.5","0.4.6","0.4.7","0.4.8","0.4.9","0.4.10","0.4.11","0.4.12","0.4.13","0.4.14","0.4.15","0.4.16","0.4.17","0.4.18","0.4.19","0.4.20"],"excluded":["0.3.17"],"target":"0.3.18","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"osv_ids_at_target":[]}} {"id":"ft-063","rule":"AD-055 ¶5 + FX-H5: the smallest release no known advisory affects that its registry has not withdrawn (crates.io/PyPI yanked, npm deprecated)","source":"npm abbreviated packument for brace-expansion (2026-10-16): 3.0.3 is deprecated (\"contains bloat files\"), the only deprecated release above 3.0.2, and 5.0.12 (latest) is not; OSV /v1/query (2026-10-16) at 3.0.2 returns GHSA-mh99-v99m-4gvg (fixed 3.0.3 on the 3.x line), at 3.0.4 it does not. Restricted to that pinned record","eco":"npm","name":"brace-expansion","installed":"3.0.2","record_ids":["GHSA-mh99-v99m-4gvg"],"published":["3.0.3","3.0.4","3.0.5","3.0.6","3.0.7","3.0.8","3.0.9","4.0.0","4.0.1","5.0.2","5.0.3","5.0.4","5.0.5","5.0.6","5.0.7","5.0.8","5.0.9","5.0.10","5.0.11","5.0.12"],"excluded":["3.0.3"],"target":"3.0.4","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-mh99-v99m-4gvg"],"osv_ids_at_target":[]}} {"id":"ft-064","rule":"AD-055 ¶5 + FX-H5: the smallest release no known advisory affects that its registry has not withdrawn (crates.io/PyPI yanked, npm deprecated); when every release that clears an advisory is withdrawn there is no target, and the withdrawn ones are named","source":"by hand from ft-062: the same pinned records with a stated registry where 0.3.17 is the last release (published/excluded are given, not observed). Every exit is withdrawn, so no target; `withheld` names 0.3.17","eco":"crates.io","name":"h2","installed":"0.3.16","record_ids":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"published":["0.3.17"],"excluded":["0.3.17"],"target":null,"clears":false,"withheld":["0.3.17"],"evidence":{"osv_ids_at_installed":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"osv_ids_at_target":[]}} +{"id":"ft-065","rule":"AD-055 ¶5 + FX-H5: an npm `deprecated` notice excludes a release only when a later, non-deprecated stable release exists in its caret range (same major; same minor for 0.x). A whole-major \"please upgrade\" notice is not a withdrawn release, so the target stays on the installed line","source":"npm abbreviated packument for glob (2026-10-17): 10.5.0 is deprecated (\"Old versions of glob are not supported, ... Please update.\", a whole-package notice) but no later 10.x release is free of a deprecation notice, so it is not excluded; the excluded list is that rule applied to every release above 10.4.5. OSV /v1/query (2026-10-17) at 10.4.5: [\"GHSA-5j98-mcp5-4vw2\"], at 10.5.0: []. Restricted to the pinned record ids","eco":"npm","name":"glob","installed":"10.4.5","record_ids":["GHSA-5j98-mcp5-4vw2"],"published":["10.5.0","11.0.0","11.0.1","11.0.2","11.0.3","11.1.0","12.0.0","13.0.0","13.0.1","13.0.2","13.0.3","13.0.4","13.0.5","13.0.6"],"excluded":[],"target":"10.5.0","clears":true,"withheld":[],"evidence":{"osv_ids_at_installed":["GHSA-5j98-mcp5-4vw2"],"osv_ids_at_target":[]}} +{"id":"ft-066","rule":"AD-055 ¶5 + FX-H5: an npm `deprecated` notice excludes a release only when a later, non-deprecated stable release exists in its caret range (same major; same minor for 0.x). A whole-major \"please upgrade\" notice is not a withdrawn release, so the target stays on the installed line","source":"npm abbreviated packument for tar (2026-10-17): 6.2.1 is deprecated (\"Old versions of tar are not supported, ... Please update.\", a whole-package notice) but no later 6.x release is free of a deprecation notice, so it is not excluded; the excluded list is that rule applied to every release above 6.1.11. OSV /v1/query (2026-10-17) at 6.1.11: [\"GHSA-f5x3-32g6-xq36\"], at 6.2.1: []. Restricted to the pinned record ids","eco":"npm","name":"tar","installed":"6.1.11","record_ids":["GHSA-f5x3-32g6-xq36"],"published":["6.1.12","6.1.13","6.1.14","6.1.15","6.2.0","6.2.1","7.0.0","7.0.1","7.1.0","7.2.0","7.3.0","7.4.0","7.4.1","7.4.2","7.4.3","7.4.4","7.5.0","7.5.1","7.5.2","7.5.3","7.5.4","7.5.5","7.5.6","7.5.7","7.5.8","7.5.9","7.5.10","7.5.11","7.5.12","7.5.13","7.5.14","7.5.15","7.5.16","7.5.17","7.5.18","7.5.19","7.5.20","7.5.21","7.5.22"],"excluded":["7.0.0","7.0.1","7.1.0","7.2.0","7.3.0","7.4.0","7.4.1","7.4.2","7.4.3","7.4.4","7.5.0","7.5.1","7.5.2","7.5.3","7.5.4","7.5.5","7.5.6","7.5.7"],"target":"6.2.1","clears":true,"withheld":[],"evidence":{"osv_ids_at_installed":["GHSA-f5x3-32g6-xq36"],"osv_ids_at_target":[]}} From 78cf9b246d3f79560b6002e968dbb1064cd7fbd3 Mon Sep 17 00:00:00 2001 From: antony Date: Sat, 17 Oct 2026 13:34:39 +1000 Subject: [PATCH 3/3] test(vectors): renumber the glob and tar fix_target lines ft-067/ft-068 (#972 holds ft-065/066) Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7 --- src-tauri/tests/conformance/vectors/README.md | 4 ++-- src-tauri/tests/conformance/vectors/fix_target.jsonl | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src-tauri/tests/conformance/vectors/README.md b/src-tauri/tests/conformance/vectors/README.md index 125d6069f..d47f81e4e 100644 --- a/src-tauri/tests/conformance/vectors/README.md +++ b/src-tauri/tests/conformance/vectors/README.md @@ -135,11 +135,11 @@ form. > fix"). - The 22 oracle cases: ft-001..ft-022 (openssl 0.10.80: ft-006; braces `last_affected`: ft-004) -- OSV per published release, all four registries: ft-023..ft-061, ft-062, ft-063, ft-065, ft-066 +- OSV per published release, all four registries: ft-023..ft-061, ft-062, ft-063, ft-067, ft-068 - No clean version, so the target clears only the fixed advisories (`clears: false`): ft-004, ft-029, ft-041, ft-042, ft-055, ft-060, ft-061 - A `last_affected` window that later releases leave (the walk steps past the bound and keeps checking every advisory): ft-043 (pyjwt), ft-052 (mistune) - A release the registry withdrew is never the target (FX-H5): npm deprecated, ft-007 (lodash 4.18.0, "Bad release", so 4.18.1) and ft-063 (brace-expansion 3.0.3, so 3.0.4); crates.io yanked, ft-062 (h2 0.3.17, so 0.3.18); every clearing release withdrawn, so no target and `withheld` names it, ft-064 (by hand) -- A whole-major npm deprecation notice is not a withdrawal, so the target stays on the installed line (review of #976): ft-065 (glob 10.4.5 -> 10.5.0, not 12.0.0), ft-066 (tar 6.1.11 -> 6.2.1, while tar 7.0.0..7.5.7 are excluded because 7.5.8 is clean) +- A whole-major npm deprecation notice is not a withdrawal, so the target stays on the installed line (review of #976): ft-067 (glob 10.4.5 -> 10.5.0, not 12.0.0), ft-068 (tar 6.1.11 -> 6.2.1, while tar 7.0.0..7.5.7 are excluded because 7.5.8 is clean) ### Refresh verdict — AD-055 ¶1, ¶2, ¶3 diff --git a/src-tauri/tests/conformance/vectors/fix_target.jsonl b/src-tauri/tests/conformance/vectors/fix_target.jsonl index 2ace909db..617111f9e 100644 --- a/src-tauri/tests/conformance/vectors/fix_target.jsonl +++ b/src-tauri/tests/conformance/vectors/fix_target.jsonl @@ -62,5 +62,5 @@ {"id":"ft-062","rule":"AD-055 ¶5 + FX-H5: the smallest release no known advisory affects that its registry has not withdrawn (crates.io/PyPI yanked, npm deprecated)","source":"crates.io /api/v1/crates/h2/versions (2026-10-16): 0.3.17 is yanked, the only yanked release above 0.3.16; OSV /v1/query (2026-10-16) at 0.3.16 returns both pinned ids, at 0.3.18 neither. Restricted to the pinned record ids (RUSTSEC-2023-0034 and its GHSA twin, both fixed 0.3.17)","eco":"crates.io","name":"h2","installed":"0.3.16","record_ids":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"published":["0.3.17","0.3.18","0.3.19","0.3.20","0.3.21","0.3.22","0.3.23","0.3.24","0.3.25","0.3.26","0.3.27","0.4.0","0.4.1","0.4.2","0.4.3","0.4.4","0.4.5","0.4.6","0.4.7","0.4.8","0.4.9","0.4.10","0.4.11","0.4.12","0.4.13","0.4.14","0.4.15","0.4.16","0.4.17","0.4.18","0.4.19","0.4.20"],"excluded":["0.3.17"],"target":"0.3.18","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"osv_ids_at_target":[]}} {"id":"ft-063","rule":"AD-055 ¶5 + FX-H5: the smallest release no known advisory affects that its registry has not withdrawn (crates.io/PyPI yanked, npm deprecated)","source":"npm abbreviated packument for brace-expansion (2026-10-16): 3.0.3 is deprecated (\"contains bloat files\"), the only deprecated release above 3.0.2, and 5.0.12 (latest) is not; OSV /v1/query (2026-10-16) at 3.0.2 returns GHSA-mh99-v99m-4gvg (fixed 3.0.3 on the 3.x line), at 3.0.4 it does not. Restricted to that pinned record","eco":"npm","name":"brace-expansion","installed":"3.0.2","record_ids":["GHSA-mh99-v99m-4gvg"],"published":["3.0.3","3.0.4","3.0.5","3.0.6","3.0.7","3.0.8","3.0.9","4.0.0","4.0.1","5.0.2","5.0.3","5.0.4","5.0.5","5.0.6","5.0.7","5.0.8","5.0.9","5.0.10","5.0.11","5.0.12"],"excluded":["3.0.3"],"target":"3.0.4","clears":true,"evidence":{"osv_ids_at_installed":["GHSA-mh99-v99m-4gvg"],"osv_ids_at_target":[]}} {"id":"ft-064","rule":"AD-055 ¶5 + FX-H5: the smallest release no known advisory affects that its registry has not withdrawn (crates.io/PyPI yanked, npm deprecated); when every release that clears an advisory is withdrawn there is no target, and the withdrawn ones are named","source":"by hand from ft-062: the same pinned records with a stated registry where 0.3.17 is the last release (published/excluded are given, not observed). Every exit is withdrawn, so no target; `withheld` names 0.3.17","eco":"crates.io","name":"h2","installed":"0.3.16","record_ids":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"published":["0.3.17"],"excluded":["0.3.17"],"target":null,"clears":false,"withheld":["0.3.17"],"evidence":{"osv_ids_at_installed":["GHSA-f8vr-r385-rh5r","RUSTSEC-2023-0034"],"osv_ids_at_target":[]}} -{"id":"ft-065","rule":"AD-055 ¶5 + FX-H5: an npm `deprecated` notice excludes a release only when a later, non-deprecated stable release exists in its caret range (same major; same minor for 0.x). A whole-major \"please upgrade\" notice is not a withdrawn release, so the target stays on the installed line","source":"npm abbreviated packument for glob (2026-10-17): 10.5.0 is deprecated (\"Old versions of glob are not supported, ... Please update.\", a whole-package notice) but no later 10.x release is free of a deprecation notice, so it is not excluded; the excluded list is that rule applied to every release above 10.4.5. OSV /v1/query (2026-10-17) at 10.4.5: [\"GHSA-5j98-mcp5-4vw2\"], at 10.5.0: []. Restricted to the pinned record ids","eco":"npm","name":"glob","installed":"10.4.5","record_ids":["GHSA-5j98-mcp5-4vw2"],"published":["10.5.0","11.0.0","11.0.1","11.0.2","11.0.3","11.1.0","12.0.0","13.0.0","13.0.1","13.0.2","13.0.3","13.0.4","13.0.5","13.0.6"],"excluded":[],"target":"10.5.0","clears":true,"withheld":[],"evidence":{"osv_ids_at_installed":["GHSA-5j98-mcp5-4vw2"],"osv_ids_at_target":[]}} -{"id":"ft-066","rule":"AD-055 ¶5 + FX-H5: an npm `deprecated` notice excludes a release only when a later, non-deprecated stable release exists in its caret range (same major; same minor for 0.x). A whole-major \"please upgrade\" notice is not a withdrawn release, so the target stays on the installed line","source":"npm abbreviated packument for tar (2026-10-17): 6.2.1 is deprecated (\"Old versions of tar are not supported, ... Please update.\", a whole-package notice) but no later 6.x release is free of a deprecation notice, so it is not excluded; the excluded list is that rule applied to every release above 6.1.11. OSV /v1/query (2026-10-17) at 6.1.11: [\"GHSA-f5x3-32g6-xq36\"], at 6.2.1: []. Restricted to the pinned record ids","eco":"npm","name":"tar","installed":"6.1.11","record_ids":["GHSA-f5x3-32g6-xq36"],"published":["6.1.12","6.1.13","6.1.14","6.1.15","6.2.0","6.2.1","7.0.0","7.0.1","7.1.0","7.2.0","7.3.0","7.4.0","7.4.1","7.4.2","7.4.3","7.4.4","7.5.0","7.5.1","7.5.2","7.5.3","7.5.4","7.5.5","7.5.6","7.5.7","7.5.8","7.5.9","7.5.10","7.5.11","7.5.12","7.5.13","7.5.14","7.5.15","7.5.16","7.5.17","7.5.18","7.5.19","7.5.20","7.5.21","7.5.22"],"excluded":["7.0.0","7.0.1","7.1.0","7.2.0","7.3.0","7.4.0","7.4.1","7.4.2","7.4.3","7.4.4","7.5.0","7.5.1","7.5.2","7.5.3","7.5.4","7.5.5","7.5.6","7.5.7"],"target":"6.2.1","clears":true,"withheld":[],"evidence":{"osv_ids_at_installed":["GHSA-f5x3-32g6-xq36"],"osv_ids_at_target":[]}} +{"id":"ft-067","rule":"AD-055 ¶5 + FX-H5: an npm `deprecated` notice excludes a release only when a later, non-deprecated stable release exists in its caret range (same major; same minor for 0.x). A whole-major \"please upgrade\" notice is not a withdrawn release, so the target stays on the installed line","source":"npm abbreviated packument for glob (2026-10-17): 10.5.0 is deprecated (\"Old versions of glob are not supported, ... Please update.\", a whole-package notice) but no later 10.x release is free of a deprecation notice, so it is not excluded; the excluded list is that rule applied to every release above 10.4.5. OSV /v1/query (2026-10-17) at 10.4.5: [\"GHSA-5j98-mcp5-4vw2\"], at 10.5.0: []. Restricted to the pinned record ids","eco":"npm","name":"glob","installed":"10.4.5","record_ids":["GHSA-5j98-mcp5-4vw2"],"published":["10.5.0","11.0.0","11.0.1","11.0.2","11.0.3","11.1.0","12.0.0","13.0.0","13.0.1","13.0.2","13.0.3","13.0.4","13.0.5","13.0.6"],"excluded":[],"target":"10.5.0","clears":true,"withheld":[],"evidence":{"osv_ids_at_installed":["GHSA-5j98-mcp5-4vw2"],"osv_ids_at_target":[]}} +{"id":"ft-068","rule":"AD-055 ¶5 + FX-H5: an npm `deprecated` notice excludes a release only when a later, non-deprecated stable release exists in its caret range (same major; same minor for 0.x). A whole-major \"please upgrade\" notice is not a withdrawn release, so the target stays on the installed line","source":"npm abbreviated packument for tar (2026-10-17): 6.2.1 is deprecated (\"Old versions of tar are not supported, ... Please update.\", a whole-package notice) but no later 6.x release is free of a deprecation notice, so it is not excluded; the excluded list is that rule applied to every release above 6.1.11. OSV /v1/query (2026-10-17) at 6.1.11: [\"GHSA-f5x3-32g6-xq36\"], at 6.2.1: []. Restricted to the pinned record ids","eco":"npm","name":"tar","installed":"6.1.11","record_ids":["GHSA-f5x3-32g6-xq36"],"published":["6.1.12","6.1.13","6.1.14","6.1.15","6.2.0","6.2.1","7.0.0","7.0.1","7.1.0","7.2.0","7.3.0","7.4.0","7.4.1","7.4.2","7.4.3","7.4.4","7.5.0","7.5.1","7.5.2","7.5.3","7.5.4","7.5.5","7.5.6","7.5.7","7.5.8","7.5.9","7.5.10","7.5.11","7.5.12","7.5.13","7.5.14","7.5.15","7.5.16","7.5.17","7.5.18","7.5.19","7.5.20","7.5.21","7.5.22"],"excluded":["7.0.0","7.0.1","7.1.0","7.2.0","7.3.0","7.4.0","7.4.1","7.4.2","7.4.3","7.4.4","7.5.0","7.5.1","7.5.2","7.5.3","7.5.4","7.5.5","7.5.6","7.5.7"],"target":"6.2.1","clears":true,"withheld":[],"evidence":{"osv_ids_at_installed":["GHSA-f5x3-32g6-xq36"],"osv_ids_at_target":[]}}