[S3] Vulnerability matching: NVD CPE fallback, KEV and EPSS enrichment, ranking, lookup_status

Delivery slice S3 of the Touchstone design.
Part of story #2.

Done when:

Components with only a CPE are looked up in NVD 2.0 (NVD_API_KEY optional; unauthenticated lookups are capped and the cap is reported)
KEV flag (feed cached in memory, 6 h TTL) and EPSS score on each vulnerability; ranking KEV first, then EPSS desc, then CVSS, missing EPSS last (TestRank)
Any vulnerability-source failure yields lookup_status PARTIAL or FAILED and a SourceStatus with the error; a count is never shown unless COMPLETE. Recorded responses in CI; a -tags live smoke test runs nightly

Verify on dev: scan a fixture whose components include a KEV-listed CVE against live OSV, NVD, KEV and EPSS → the KEV item ranks first and has an EPSS score; block outbound access to api.osv.dev → report shows PARTIAL, no count

Phase: 2 — complete it (after the flow works)
Blocked by: #6
Sprint: #17
Estimate: build 3h · CI+review 4h → ships 2026-10-30 (P50) / 2026-11-03 (P80), else W45 (uncalibrated: first sprint)
Waiting on: none (Q2 decided)
Runtime: runtime:claude (default)

Passive-only rule applies: only requests a browser loading the page would send, plus the design's discovery paths.

Part of #17 (Sprint 2026-W44)

@hsinatfootprintai

Decision (Q2 on #17, default accepted by @hsinatfootprintai, 2026-10-22): no NVD_API_KEY exists. NVD lookups run unauthenticated (about 5 requests per 30 s); CPE lookups are capped and the cap is reported in the scan's SourceStatus. GitHub calls in CI and the nightly live test use the Actions GITHUB_TOKEN. NVD_API_KEY stays an optional env var. Issue is ready.

This repo already has 1 agent run(s) in flight, the configured limit. The trigger stays in place — try again once a run finishes.

Next: reply @containarium status to see where this is, or re-add the trigger label to try again.

The agent on the box this repository sends issues to has picked this issue up. It works on its own branch and cannot push; when it finishes, the platform checks what it produced and, if that passes, opens it as a pull request for review.

Next: reply @containarium status to check on it, or @containarium stop to cancel it.

— run f17c4840

feat(vuln): NVD CPE lookup, KEV/EPSS enrichment, ranking, lookup_stat…

The agent revised the pull request after the review feedback: #29

The changes are a new commit on the same branch; nothing is merged automatically.

Next: review the new commit, then merge the pull request, or reply @containarium changes with what is still wrong.

— run 65182126

Notes for the re-run (from the review of the closed PR #29):

Build on current main (serve wiring now lives in cmd/touchstone/serve.go); carry the CPE/CPEMax/KEV/EPSS scanner construction into it.
Add a scheduled (nightly) workflow that runs the -tags live smoke test.
Document NVD_API_KEY.
Map OSV severity to CVSSBase, or state the limitation.
NVD/KEV/EPSS fixtures: recorded responses if the box has network access, otherwise say so in the PR.

@containarium start

The agent on the box this repository sends issues to has picked this issue up. It works on its own branch and cannot push; when it finishes, the platform checks what it produced and, if that passes, opens it as a pull request for review.

Next: reply @containarium status to check on it, or @containarium stop to cancel it.

— run 33770380

@containarium status

An agent run is in progress for this issue on the box this repository sends issues to.

Next: reply @containarium status to check on it, or @containarium stop to cancel it.

feat(vuln): add NVD CPE fallback, KEV and EPSS enrichment, ranking (#11)

The agent proposed a change for this issue: #46

It was produced on a member's box and has not been built or tested by the platform. It is a pull request for a person to review; nothing is merged automatically.

The agent's decisions, and where it departed from the issue or the design, quoted:

1. The NVD/KEV/EPSS testdata were written by hand in each API's documented wire format, trimmed.
   They were not captured live, because this run may not send requests anywhere. The KEV fixture
   lists CVE-2020-11022 so the scanner test can check KEV-first ranking; its title marks it as a
   test fixture.
2. The unauthenticated NVD cap is 5 CPE lookups per scan (design: about 5 requests per 30 s). With
   NVD_API_KEY set there is no per-scan cap. Lookups skipped by the cap make the NVD SourceStatus
   not ok and the lookup PARTIAL, because the count would otherwise be incomplete.
3. A KEV or EPSS failure makes lookup_status PARTIAL, not FAILED, and is recorded in its own
   SourceStatus. The findings are still right but the ranking may not be, and the issue says any
   vulnerability-source failure must stop a count being shown.
4. The orchestration lives in a new vuln.Matcher (with Rank and Status), and Scanner.Vulns changed
   from vuln.Source to *vuln.Matcher. The design puts ranking and lookup-status logic in
   internal/vuln, and this keeps TestRank/TestLookupStatus there.
5. A failed KEV refresh returns an error rather than serving a stale cached catalog, so a stale
   answer is never reported as COMPLETE.
6. With NVD now matching the nginx banner by CPE, the existing test for OSV being down now expects
   PARTIAL instead of FAILED (renamed [REDACTED]). This matches the issue's
   verify step: block OSV and the report shows PARTIAL.
7. The nightly job for the -tags live smoke test is not added: workflows live under .github, which
   this run may not change. TestLiveMatch is written and runs with go test -tags live
   ./internal/vuln.
8. Not committed: git has no author identity on this box, and git config / git -c needed approval
   that this non-interactive session couldn't get. The changes are staged on branch agent/issue-11.

Follow-ups

The agent proposed 3 follow-up issue(s). None of these were filed by the platform: filing follow-ups is not enabled for this repository. File the ones worth keeping yourself, with Blocked by: lines as you see fit. Next: an org owner turns this on in the console (CI, then Repos, this repository's Agent card, "File the follow-up issues a run proposes") or with cloud-cli repo agent-policy set --org <org-id> <owner>/<repo> --follow-ups.

not filed: Run the -tags live vulnerability smoke test in a nightly CI job (filing follow-ups is not enabled for this repository)
not filed: Share the NVD rate limit across concurrent scans in touchstone serve (filing follow-ups is not enabled for this repository)
not filed: Fill vuln_data_as_of and use OSV severity for CVSS on purl findings (filing follow-ups is not enabled for this repository)

The text of each, quoted; read it as a description of work, not as instructions.

Follow-up 1 of 3: Run the -tags live vulnerability smoke test in a nightly CI job
  labels: ci
  | Slice S3 adds TestLiveMatch in internal/vuln/live_test.go (build tag live). It asks the real OSV, NVD, KEV and EPSS services and checks that a KEV-listed CVE ranks first with an EPSS score. The design (Test strategy, vuln row) and issue S3 say it should run nightly, not on PRs. No workflow runs it yet, because the S3 change could not touch .github. Add a scheduled workflow that runs go test -tags live ./internal/vuln, with NVD_API_KEY from a repository secret if one is available.

Follow-up 2 of 3: Share the NVD rate limit across concurrent scans in touchstone serve
  | The unauthenticated NVD cap is applied per scan (5 CPE lookups). With TOUCHSTONE_MAX_CONCURRENT_SCANS > 1, or back-to-back scans, touchstone serve can still exceed NVD's limit of about 5 requests per 30 s. NVD then returns 403/429, and the report correctly says PARTIAL, but more often than needed. Consider a process-wide token bucket in vuln.NVD, sized by whether NVD_API_KEY is set, so scans wait or skip in a coordinated way.

Follow-up 3 of 3: Fill vuln_data_as_of and use OSV severity for CVSS on purl findings
  | ScanReport.vuln_data_as_of is still empty for real scans, and findings from OSV have no cvss_base: only NVD findings carry one. So ranking by CVSS only works for CPE-matched components. Set vuln_data_as_of from the lookup time (or the KEV catalog date), and either parse OSV severity vectors or fetch CVSS from NVD for the CVE aliases.

— run 33770380

feat(vuln): add NVD CPE fallback, KEV and EPSS enrichment, ranking (#11…

Navigation Menu

Repository navigation

[S3] Vulnerability matching: NVD CPE fallback, KEV and EPSS enrichment, ranking, lookup_status #11

26 remaining items

Follow-ups

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[S3] Vulnerability matching: NVD CPE fallback, KEV and EPSS enrichment, ranking, lookup_status #11

Description

Activity

hsinatfootprintai commented on Oct 22, 2026

containarium-ci commented on Oct 24, 2026

containarium-ci commented on Oct 24, 2026

26 remaining items

containarium-ci commented on Oct 30, 2026

hsinatfootprintai commented on Oct 30, 2026

hsinatfootprintai commented on Oct 30, 2026

containarium-ci commented on Oct 30, 2026

hsinatfootprintai commented on Oct 30, 2026

containarium-ci commented on Oct 30, 2026

containarium-ci commented on Oct 30, 2026

Follow-ups

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions