feat: add controlled vulnerability database and signed seed bootstrap

.coderisktools-baseline.json

@@ -18,6 +18,7 @@
     "sha256:317334d25e7f63f731739102c0f3324e149163dc9aa89fd062cb56a002c38e90",
     "sha256:32003d386f4c13bbd8e39554ecf8e790165bf5d8027978c3fd53eedadc18a6a3",
     "sha256:3e8c5ab6e930af06663435256e3205c0dcc548c366b128138504e64ad0a35340",
+    "sha256:3fa4350533e8d0b945c36390441343258bcf356aa7f90ffcec205723a48acfbb",
     "sha256:41a2d2511602221b2e73923b12ec4ed17126a52a1e4eefc457bad128e6cd4582",
     "sha256:43fbf8cbca5bb69e800d6bad8d2e93e5b0c9782cfd77b1a24e01d7dcaa23cb43",
     "sha256:44fff6f8123baedbb3805ab5510e8e75ef98479060a1db6814a23c2cadad7f72",
@@ -51,7 +52,6 @@
     "sha256:95f4fb0457e624269f2e6db5ad54c88942b2d7fe1092c79343b9c2ba0ae0e4f6",
     "sha256:9912c72fe03e28b66d5b4667c666714d1ba43cc93124187b65a453927d239cbb",
     "sha256:9f33a910547b5beef79ce1c0aba074f306a966846d1ca608d4a9fb72a2b2c601",
-    "sha256:a02ba68f0be4d9b78998ae99f3ee9197b26de0762b701a55986d33879d71064c",
     "sha256:a052f7f69ccc1e3c69286d4b4be5a8b64c951345afdd7a8c3a1568efff22e87a",
     "sha256:a153a81f7c9c09b188669b04c2e94fb1ab97c6cb257713180938b64bc24292fa",
     "sha256:a422a63976b0052a6999f2d10c8ad7b270d79c4abcb54b3e87c1b575aec957bc",

CHANGELOG.md

@@ -4,17 +4,27 @@ All notable changes to `coderisktools-scanner` are documented here.
 
 ## [Unreleased]
 
-### Vulnerability enrichment
+## [3.1.0] — 2026-07-23
 
-- versioned the local vulnerability database schema with explicit initialization, migration, current-state reporting and fail-closed future-version handling;
-- preserved NVD configuration logic instead of flattening it: node `AND`/`OR`, `negate`, nested children and a legacy flat CPE projection are all available;
-- preserved NVD references, source tags and CVE change history through parser, SQLite enrichment and normalized reports;
-- added deterministic readback tests and documentation for the V5x–V5z contracts.
+### Added
 
-### Scanner evidence boundary
+- added the opt-in, read-only vulnerability inventory and matching pipeline with explicit local SQLite selection;
+- added controlled, user-triggered staging/update, verification, reconciliation, rollback and retention operations;
+- added bounded public feed adapters and explicit provenance/quality reports without claiming full-feed coverage;
+- added a real partial `seed` snapshot with 187 advisories, 378 affected-package rows and seven represented OSV ecosystems;
+- added signed, pinned seed bootstrap and a separate explicit `--profile seed --apply` activation command.
 
-- documented the required distinction between runtime remediation evidence and publication-safe redaction;
-- runtime evidence preservation remains an open formatter/pipeline audit finding; checked-in/public artifacts remain synthetic or explicitly redacted.
+### Fixed
+
+- directory self-scan now skips SQLite database artifacts, preventing the real seed from tripping the scanner byte cap in CI;
+- bootstrap now verifies the detached Ed25519 manifest envelope, exact database SHA-256, SQLite integrity, foreign keys, snapshot identity and manifest counts before atomic installation.
+
+### Seed boundary
+
+- the seed is `completeness=partial` and `production_full_database=false`;
+- 1,688 unresolved KEV/EPSS enrichments and 129 exact-alias conflicts are retained and disclosed rather than heuristically merged;
+- an empty seed scan is not evidence that a project has no vulnerabilities;
+- Core/Full activation remains separate and is never replaced automatically by seed bootstrap.
 
 ## [3.0.1] — 2026-07-20
 

README.md

@@ -500,7 +500,7 @@ Valid agents: `generic`, `codex`, `claude-code`. This is an explicit input contr
 ```yaml
 repos:
   - repo: https://github.com/9batalion/coderisktools-scanner
-    rev: v3.0.1
+    rev: v3.1.0
     hooks:
       - id: coderisktools-secret-scan
 ```
@@ -522,7 +522,7 @@ jobs:
       - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
         with:
           fetch-depth: 0
-      - uses: 9batalion/coderisktools-scanner@v3.0.1
+      - uses: 9batalion/coderisktools-scanner@v3.1.0
         with:
           profile: balanced
 ```

data/vulnerability-seed/README.md

@@ -0,0 +1,63 @@
+# Verified vulnerability seed
+
+This directory contains a real, bounded starter snapshot for local development and integration tests.
+
+## Scope
+
+- profile: `seed`
+- completeness: `partial`
+- `production_full_database`: `false`
+- advisories: 187
+- affected packages: 378
+- SQLite size: approximately 5.6 MB
+- OSV: 87 imported advisories from selected PyPI, npm, Go, crates.io, Maven, NuGet and Packagist records
+- CISA KEV: 1,653 source records processed; 2 exact enrichments and 1,651 unresolved records retained
+- GHSA: bounded 100-record batch, 100 imported
+- EPSS: 54 targeted records; 17 exact enrichments and 37 unresolved records retained
+- exact alias index: 129 ambiguous alias conflicts are reported and are not heuristically merged
+
+This is not a complete Core database and must not be used to interpret an empty scan as proof of no vulnerabilities. The seed intentionally preserves unresolved enrichment and alias conflicts instead of making unsafe automatic merge decisions.
+
+## Verify
+
+From the repository root:
+
+```bash
+sha256sum -c data/vulnerability-seed/seed-vulndb.sqlite.sha256
+python3 scripts/verify_seed_vulndb.py \
+  --database data/vulnerability-seed/seed-vulndb.sqlite \
+  --manifest data/vulnerability-seed/manifest.json
+```
+
+The verifier checks the SQLite integrity, foreign-key integrity, database digest, and manifest counts.
+
+## Pinned release bootstrap
+
+The `v3.1.0` release publishes the exact SQLite, detached manifest, SHA-256 sidecar, signed manifest envelope and public keyring. Bootstrap installs the database as **staged** only:
+
+```bash
+secret-scanner vuln-db bootstrap \
+  --asset-url https://github.com/9batalion/coderisktools-scanner/releases/download/v3.1.0/coderisktools-vulndb-seed-2026-07-23.sqlite \
+  --manifest-url https://github.com/9batalion/coderisktools-scanner/releases/download/v3.1.0/coderisktools-vulndb-seed-2026-07-23.manifest.json \
+  --signature-url https://github.com/9batalion/coderisktools-scanner/releases/download/v3.1.0/coderisktools-vulndb-seed-2026-07-23.sig \
+  --keyring data/vulnerability-seed/release-keyring.json \
+  --destination "$HOME/.local/share/coderisktools/vuln-db/seed.sqlite"
+```
+
+Activation is a separate explicit operation and is limited to `--profile seed`:
+
+```bash
+secret-scanner vuln-db activate \
+  --database "$HOME/.local/share/coderisktools/vuln-db/seed.sqlite" \
+  --manifest data/vulnerability-seed/manifest.json \
+  --profile seed \
+  --apply
+```
+
+A seed activation never replaces a Core/Full pointer automatically.
+
+## Provenance
+
+The snapshot was built from verified staged CISA KEV and GHSA source artifacts, targeted EPSS data from FIRST, and OSV vulnerability records obtained through the public OSV API using explicit package probes. Raw source feeds are not committed to this repository.
+
+A larger Core snapshot belongs in a GitHub Release Asset, not in Git history. Any future replacement must update the manifest, SHA-256 sidecar, signed envelope, and independent verification evidence together.

data/vulnerability-seed/manifest.json

@@ -0,0 +1,68 @@
+{
+  "advisory_count": 187,
+  "affected_package_count": 378,
+  "alias_conflict_count": 129,
+  "completeness": "partial",
+  "content_digest": "sha256:8c122da9ffbbe70f677503068382416ad0d9d31adcdff3ceb1a9adf50de48f04",
+  "correlation": "exact-alias-index-with-unresolved-conflicts",
+  "database": {
+    "advisory_count": 187,
+    "affected_package_count": 378,
+    "content_digest": "sha256:8c122da9ffbbe70f677503068382416ad0d9d31adcdff3ceb1a9adf50de48f04",
+    "path": "seed-vulndb.sqlite",
+    "sha256": "sha256:2bf63ca969a56a12551c491a956746d504409f13ea1c006abcf19e52ff0e6b04"
+  },
+  "database_sha256": "sha256:2bf63ca969a56a12551c491a956746d504409f13ea1c006abcf19e52ff0e6b04",
+  "production_full_database": false,
+  "profile": "seed",
+  "provenance": {
+    "built_at": "2026-07-23",
+    "cache_reuse": "CISA KEV and GHSA verified staging cache; EPSS targeted CVEs from FIRST API",
+    "osv_transport": "OSV API per-package probes plus OSV vulnerability records"
+  },
+  "release": {
+    "asset": "coderisktools-vulndb-seed-2026-07-23.sqlite",
+    "key_id": "coderisktools-seed-2026",
+    "manifest": "coderisktools-vulndb-seed-2026-07-23.manifest.json",
+    "sha256": "coderisktools-vulndb-seed-2026-07-23.sha256",
+    "signature": "coderisktools-vulndb-seed-2026-07-23.sig",
+    "tag": "v3.1.0"
+  },
+  "schema": "coderisktools.vulnerability.seed-manifest.v1",
+  "snapshot_id": "seed-8c122da9ffbbe70f677503068382416a",
+  "source_digest": "sha256:a9499f0d50184de077690431a98027dbb7c52fe916e1dc8879db77f2a2ac4ea9",
+  "sources": {
+    "cisa-kev": {
+      "imported": 2,
+      "records": 1653,
+      "status": "complete",
+      "unresolved": 1651
+    },
+    "epss": {
+      "imported": 17,
+      "records": 54,
+      "status": "bounded",
+      "unresolved": 37
+    },
+    "ghsa": {
+      "imported": 100,
+      "records": 100,
+      "status": "bounded"
+    },
+    "osv": {
+      "failed_ecosystems": [],
+      "imported": 87,
+      "records_per_ecosystem": 500,
+      "status": "partial",
+      "successful_ecosystems": [
+        "PyPI",
+        "npm",
+        "Go",
+        "crates.io",
+        "Maven",
+        "NuGet",
+        "Packagist"
+      ]
+    }
+  }
+}

data/vulnerability-seed/release-keyring.json

@@ -0,0 +1,7 @@
+{
+  "keys": {
+    "coderisktools-seed-2026": "64d79d903860fc16b2adb99d9a8ebe6a05540b9a9a2437d0062ba1c552a380a1"
+  },
+  "schema": "coderisktools.rule-keyring",
+  "version": 1
+}

data/vulnerability-seed/seed-vulndb.sqlite.sha256

@@ -0,0 +1 @@
+2bf63ca969a56a12551c491a956746d504409f13ea1c006abcf19e52ff0e6b04  seed-vulndb.sqlite

docs/EXTERNAL-EVIDENCE-COMPARISON.md

@@ -0,0 +1,24 @@
+# External evidence comparison
+
+The benchmark comparison fixture is intentionally offline and synthetic. It
+compares identifier sets representing an internal result and explicitly
+supplied OSV-Scanner, Trivy and Grype evidence. It does not execute any of the
+external tools and does not merge their findings into `Finding`, `ScanResult`
+or the local vulnerability database.
+
+## Difference meanings
+
+- `aligned`: the identifier is present in both result sets;
+- `external_only`: the supplied external evidence contains an identifier not
+  present in the internal result;
+- `internal_only`: the internal result contains an identifier absent from the
+  supplied external evidence.
+
+Differences are not proof that one tool is correct. They can result from
+package identity normalization, version/range semantics, advisory database
+freshness, severity policy, ignored paths, transitive dependency handling or
+source-specific matching rules. Investigations must use the original evidence
+and provenance rather than copying an external result.
+
+The fixture is a contract test for comparison/reporting behavior, not an
+accuracy ranking of OSV-Scanner, Trivy or Grype.

docs/FULL-FEED-COVERAGE-PLAN.md

@@ -0,0 +1,87 @@
+# Full feed coverage plan
+
+## Objective
+
+Build a reproducible local vulnerability database from declared sources without
+claiming universal ecosystem coverage. A feed becomes `ready` only after its
+adapter, fixture corpus, provenance, license/terms review, quality gate,
+rebuild determinism and rollback path pass.
+
+## Source waves
+
+### Wave 1 — core advisory and exploitation sources
+
+- OSV
+- NVD
+- CISA KEV
+- EPSS
+- GitHub Advisory Database
+
+### Wave 2 — Linux distributions
+
+- Debian Security
+- Ubuntu Security
+- Red Hat Security
+- SUSE Security
+- Alpine Security
+
+### Wave 3 — CSAF providers
+
+- Generic CSAF 2.0
+- Provider-specific product trees
+- Provider-specific status and remediation semantics
+
+### Wave 4 — package ecosystems
+
+- PyPI
+- npm
+- crates.io
+- Maven Central
+- NuGet
+- remaining ecosystem adapters
+
+## Per-feed implementation contract
+
+1. Declare endpoint, source identity, terms/license status and update cadence.
+2. Fetch only through the existing HTTPS allowlist, bounded streaming and
+   conditional request policy.
+3. Verify content type, size, digest and source provenance.
+4. Parse into the existing OSV-shaped import boundary without changing
+   `Finding`, `ScanResult` or fingerprint contracts.
+5. Preserve source-native identifiers, aliases, references and raw status.
+6. Run quality metrics and reject invalid snapshots before activation.
+7. Test malformed, oversized, duplicate, stale and semantically ambiguous data.
+8. Rebuild twice and compare artifact digests.
+9. Verify rollback and air-gap export/import.
+10. Mark the feed `ready` only when all evidence is stored.
+
+The OSV adapter now supports a separate staging operation: an allowlisted HTTPS
+URL is streamed to an atomically replaced file and then passed to the existing
+bounded OSV importer. Activation remains explicit (`activate=False` by default),
+and the adapter reports the downloaded payload digest. Live feed acceptance,
+license verification and a verified full snapshot are still required before OSV
+can become `ready`.
+
+NVD now has a bounded API 2.0 file ingestion contract with source digest,
+strict CVE parsing, exact advisory correlation, partial-record errors and
+explicit activation. Live NVD feed acceptance and terms verification remain
+required before `ready`.
+
+KEV, EPSS and GitHub Advisory now have bounded local feed orchestration with
+source digest, delegation to existing database importers, partial-error
+reporting and explicit activation. Their live feed, terms and full snapshot
+evidence remain required before `ready`.
+
+When network access is unavailable, `evaluate_feed_artifact()` provides the
+acceptance path for an externally acquired artifact from CI, cache or an
+air-gapped transfer. It verifies JSON envelope, size/record bounds and SHA-256
+without downloading or activating the artifact; this is not equivalent to live
+transport acceptance.
+
+## Non-goals
+
+- No assertion that all advisories worldwide are covered.
+- No package-manager execution.
+- No execution of scanned repository code.
+- No automatic activation after download.
+- No redistribution of source data before terms/license verification.

docs/KNOWN-LIMITATIONS.md

@@ -0,0 +1,53 @@
+# Known limitations
+
+This document is part of the production snapshot boundary. It intentionally
+lists limitations instead of implying complete vulnerability coverage.
+
+## Vulnerability data
+
+- Feed adapters are bounded contracts, not a claim of complete live-feed
+  ingestion or complete historical coverage.
+- OSV-shaped records are supported locally; source-specific fields outside the
+  bounded adapter contract may be preserved only as metadata or omitted.
+- Matching can be `indeterminate` when a package, version, range or ecosystem
+  cannot be compared safely.
+- Enrichment and correlation evidence is additive and provenance-aware; it is
+  not a replacement for vendor advisory review.
+
+## Ecosystem coverage
+
+- Python, npm, Cargo/ crates.io and Go have the strongest local inventory path.
+- Linux distribution, Maven, NuGet, RubyGems, Swift, Dart, Elixir, Haskell,
+  R, Conan and vcpkg adapters are bounded and fixture-driven.
+- CSAF support is bounded to the documented product/PURL, advisory, provider,
+  remediation, vendor-status and quality-gate contract.
+- The project must not be described as providing full support for all package
+  managers, distributions, feeds or CSAF provider extensions.
+
+## Benchmarking
+
+- Public fixtures are deterministic but intentionally small; they are not a
+  statistically complete vulnerability corpus.
+- Precision/recall results are fixture results, not a security certification.
+- Latency depends on hardware, Python version and database state. Absolute
+  cross-platform baselines require separate measurement.
+- Comparisons with OSV-Scanner, Trivy and Grype are external-evidence work and
+  must not be represented as copied or authoritative results.
+
+## Production and recovery
+
+- Manifest signing verifies supplied keys and bytes; key custody and rotation
+  remain deployment responsibilities.
+- Air-gap import restores a verified database but does not activate it.
+- Rollback planning is non-destructive until an explicit apply operation.
+- Release readiness reports validate supplied metadata; they cannot prove the
+  integrity of an upstream build system or hosting provider.
+
+## Scanner boundary
+
+- A clean result is not proof that code is secure.
+- False positives and false negatives remain possible.
+- The scanner is not a security audit, certification, legal opinion or
+  compliance guarantee.
+- The scanner does not execute target-project code and ordinary scans do not
+  require network access.