Add vuln data source docs and initial air-gap guide

docs/concepts/.pages

@@ -2,6 +2,7 @@ title: Concepts
 nav:
   - index.md
   - About access control: access-control.md
+  - About vulnerability data sources: about-vulnerability-data-sources.md
   - About notifications: notifications.md
   - About time series metrics: time-series-metrics.md
   - About component policies: component-policies.md

docs/concepts/about-vulnerability-data-sources.md

@@ -0,0 +1,145 @@
+# About vulnerability data sources
+
+A vulnerability data source is an upstream feed of known security defects that Dependency-Track uses to find
+vulnerabilities in your components. No single feed covers every ecosystem, identifier scheme, or disclosure path, so
+Dependency-Track integrates with several. You pick which ones to enable based on what you ship and where the relevant
+advisories are published. None of them are required, and enabling all of them is rarely the right answer.
+
+This page explains what each source contributes, how the data reaches your findings, and the trade-offs you accept when
+you enable a source or turn one off. For the procedure to enable sources, see [Configuring vulnerability
+sources](../guides/administration/configuring-vulnerability-sources.md).
+
+## Mirrored sources
+
+Dependency-Track can mirror three public data sources into its own database:
+
+- **[National Vulnerability Database (NVD)](https://nvd.nist.gov/)** is the canonical CVE feed maintained by NIST. It
+  contains over 200,000 CVE records spanning the early nineties to the present and describes affected products as
+  Common Platform Enumeration (CPE) names. NVD is the broadest source for non-package software (operating systems,
+  firmware, network equipment), but its CPE-based product model is a poor fit for open source packages.
+- **[GitHub advisories](https://github.com/advisories)** is GitHub's curated advisory database (GHSA). It covers open
+  source packages on npm, Maven, PyPI, NuGet, Go, RubyGems, and others. Advisories use Package URL (PURL) keys, often
+  appear before the corresponding NVD entry, and frequently carry more ecosystem-specific detail than the NVD record.
+- **[OSV](https://osv.dev/)** is Google's open source vulnerability database. It aggregates advisories from GitHub,
+  PyPA, RustSec, the Go team, distribution security teams, and others into one PURL-keyed dataset, and it covers
+  ecosystems and distributions that NVD and GHSA do not (for example crates.io, Hex, Pub, Alpine, and Debian).
+
+### Picking sources
+
+The three feeds overlap heavily for open source ecosystems. Most operators do not need all three:
+
+- **GHSA and OSV cover much of the same ground.** OSV ingests GitHub advisories and adds advisories from other curators
+  (PyPA, RustSec, Go, distribution security teams). OSV also lets you mirror per ecosystem, which is finer-grained
+  control than GHSA offers. If you want one source for open source packages, prefer OSV.
+- **GHSA** is useful on its own if you stay close to the GitHub ecosystem or want the data exactly as GitHub publishes
+  it, without the aggregation layer that OSV adds.
+- **NVD** is the only mirrored source that meaningfully covers non-package software identified by CPE. Enable it when
+  your BOMs include such components, or when you specifically need CVE coverage independent of curator-specific feeds.
+
+## How Dependency-Track uses the data
+
+Mirroring is a background task. On a configurable schedule and at instance startup, Dependency-Track downloads new and
+changed records from each enabled source and writes them into its own database.
+
+Vulnerability analysis is a separate process. When you upload a Bill of Materials or trigger analysis, the [internal
+analyzer](../reference/analyzers.md#internal) matches the project's components against the mirrored data already stored
+locally. The internal analyzer makes no outbound calls during analysis, which makes findings reproducible and keeps
+analysis fast.
+
+For the full analysis workflow, see [Vulnerability analysis](architecture/design/vulnerability-analysis.md).
+
+### Turning a source off
+
+Disabling a source stops further mirroring from that source, but Dependency-Track does not delete the records it has
+already written. Vulnerabilities mirrored before the source was turned off remain in the database, and the internal
+analyzer keeps matching components against them on subsequent analysis runs. Findings already raised from that source
+are not retroactively removed either.
+
+The practical consequence is that turning a source off is reversible without re-downloading, but it does not clean up
+after itself. If you intend to drop a source entirely and have its findings disappear, expect to clear the data
+manually. Dependency-Track does not yet provide a built-in workflow for this.
+
+## How component matching works
+
+Each advisory in a mirrored source names the products it affects, using either a CPE or a PURL, and specifies the
+affected versions. The internal analyzer matches a component to an advisory by identifier scheme first and version
+second.
+
+| Source | Identifier in records | What components need |
+|:-------|:----------------------|:---------------------|
+| NVD | CPE | A valid CPE on the component |
+| GitHub advisories | PURL | A valid PURL on the component |
+| OSV | PURL | A valid PURL on the component |
+
+A component with neither a CPE nor a PURL is not evaluated.
+
+### CPE matching
+
+CPE matching follows the [NIST CPE name matching
+specification](https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7696.pdf): the analyzer compares all eleven CPE
+attributes (`part`, `vendor`, `product`, `version`, `update`, `edition`, `language`, `sw_edition`, `target_sw`,
+`target_hw`, `other`) and rejects the match if any attribute is disjoint.
+
+Two adjustments reduce false positives compared to a literal reading of the spec:
+
+- A SUBSET vendor combined with a SUPERSET product (or the reverse) is rejected, because these combinations otherwise
+  produce noisy matches across unrelated products (see [issue
+  #3178](https://github.com/DependencyTrack/dependency-track/issues/3178)).
+- Component versions of `*` (ANY) and `-` (NA) are handled per the spec, not run through range comparison.
+
+When the advisory specifies a version range, the analyzer evaluates it using the
+[vers](https://www.packageurl.org/docs/vers/introduction) range specification, with a versioning scheme inferred from
+the component's PURL when present and the `generic` scheme otherwise.
+
+### PURL matching
+
+PURL matching requires the advisory's PURL `type`, `namespace`, and `name` to match the component's exactly. For
+OS-package PURLs (`deb`, `rpm`), if both sides carry a `distro` qualifier they must agree, with semantic equivalences
+honored (for example, `debian-13` matches `trixie`).
+
+Version comparison uses [vers](https://www.packageurl.org/docs/vers/introduction) with the versioning scheme derived
+from the PURL type. If the strict scheme fails to parse a version, the analyzer falls back to the `generic` scheme to
+avoid false negatives. For `deb` and `rpm` PURLs, an `epoch` qualifier is folded into the version as `epoch:version`
+before comparison when the version does not already encode one.
+
+### Practical consequence
+
+Most modern BOM generators emit PURLs but not CPEs. As a result, components from package ecosystems (npm, Maven, PyPI,
+and so on) match GHSA and OSV well but produce few or no findings from NVD data, even when CVE records exist for those
+packages. This is the most common reason operators see fewer NVD findings than expected.
+
+If you need NVD coverage for ecosystem packages, your BOM generator must also produce CPEs for those components.
+Otherwise, rely on GHSA or OSV for open source coverage and on NVD primarily for non-package software identified by CPE.
+The OWASP SBOM Forum's [recommendations to improve the
+NVD](https://owasp.org/blog/2022/09/13/sbom-forum-recommends-improvements-to-nvd) document this CPE-vs-PURL mismatch and
+the industry effort to close it.
+
+## Aliases across sources
+
+A single vulnerability often has different identifiers in different sources. Log4Shell, for example, is `CVE-2021-44228`
+in NVD and `GHSA-jfh8-c2jp-5v3q` in GHSA.
+
+Dependency-Track records aliases when sources publish explicit links between identifiers and surfaces them in the UI.
+Aliases do not de-duplicate findings: if two enabled sources both report Log4Shell for the same component, you see two
+findings, one per source, with the alias relationship visible on each.
+
+## Mirrored sources vs. external analyzers
+
+The mirrored sources are not the only way Dependency-Track gets vulnerability data. It also ships analyzers that call
+external services at analysis time:
+
+- **OSS Index** (Sonatype)
+- **Snyk**
+- **Trivy** (against a Trivy server you operate)
+- **VulnDB** (Flashpoint, commercial)
+
+These analyzers send component identifiers to a third party on every analysis run, return findings inline, and store
+nothing locally beyond the result. They complement the mirrored sources but introduce an outbound dependency and, in
+some cases, a commercial contract. See [Vulnerability analyzers](../reference/analyzers.md) for the full list and their
+requirements.
+
+## Vulnerabilities you define yourself
+
+The [private vulnerability repository](../reference/datasources/private-vulnerability-repository.md) holds
+vulnerabilities you define yourself. Use it for vulnerabilities in internal code or for pre-disclosure tracking.
+Findings from private vulnerabilities behave the same as findings from public sources.

docs/concepts/changes-in-v5.md

@@ -168,6 +168,7 @@ remediation steps, lives in the
   use for them), and its file storage abstracts over backends like S3
   rather than assuming a local filesystem to serve from. Dependency-Track
   no longer serves this endpoint. Fetch feeds directly from NIST or run a dedicated mirror.
+  See [Running air-gapped](../guides/administration/running-air-gapped.md) for hosting an internal NVD or OSV mirror.
 
 [CEL]: https://cel.dev/
 [Protobuf]: https://protobuf.dev/

docs/guides/administration/.pages

@@ -7,6 +7,8 @@ nav:
   - configuring-secret-management.md
   - configuring-internal-ca.md
   - configuring-http-proxy.md
+  - configuring-vulnerability-sources.md
+  - running-air-gapped.md
   - configuring-observability.md
   - configuring-ldap.md
   - configuring-oidc.md

docs/guides/administration/configuring-http-proxy.md

@@ -1,9 +1,9 @@
 # Configuring an HTTP proxy
 
-The Dependency-Track API server makes outbound HTTP and HTTPS calls to mirror vulnerability data sources, fetch
-package metadata from repositories, perform OIDC discovery, deliver webhooks, and reach other integrations. In
-environments where outbound traffic must traverse a corporate proxy, configure the API server to route those calls
-through it.
+The Dependency-Track API server makes outbound HTTP and HTTPS calls to mirror
+[vulnerability data sources](configuring-vulnerability-sources.md), fetch package metadata from repositories, perform
+OIDC discovery, deliver webhooks, and reach other integrations. In environments where outbound traffic must traverse a
+corporate proxy, configure the API server to route those calls through it.
 
 Proxy configuration applies to the API server only. The frontend is a static single-page app served to the
 user's browser; any requests it appears to make actually originate from the browser itself.

docs/guides/administration/configuring-internal-ca.md

@@ -15,7 +15,8 @@ Common scenarios where this occurs:
 - **LDAPS**: connecting to an LDAP server using TLS ([Configuring LDAP](configuring-ldap.md))
 - **OIDC**: the identity provider uses a privately signed certificate ([Configuring OIDC](configuring-oidc.md))
 - **HTTP proxy**: an intercepting TLS proxy sits between Dependency-Track and external services
-- **Internal vulnerability sources**: a private vulnerability repository or API server with an internal certificate
+- **Internal vulnerability sources**: an internal mirror of NVD or OSV behind a privately signed certificate (see
+  [Configuring vulnerability sources](configuring-vulnerability-sources.md) and [Running air-gapped](running-air-gapped.md))
 
 The solution is to import your internal CA certificate into the JVM truststore and
 mount the modified truststore into the container.

docs/guides/administration/configuring-vulnerability-sources.md

@@ -0,0 +1,103 @@
+# Configuring vulnerability sources
+
+Dependency-Track can mirror three public vulnerability data sources into its internal database: the National
+Vulnerability Database (NVD), GitHub advisories, and OSV. You pick which ones to enable, configure them through the web
+UI, and trigger an initial mirror so findings appear without waiting for the next scheduled run.
+
+For background on what each source provides, when one is enough, and how the internal analyzer turns mirrored data into
+findings, see [About vulnerability data sources](../../concepts/about-vulnerability-data-sources.md).
+
+## Prerequisites
+
+For each source you plan to enable, allow outbound HTTPS access from the API server to the corresponding host:
+
+| Source | Host |
+|:-------|:-----|
+| NVD | `nvd.nist.gov` |
+| GitHub advisories | `api.github.com` |
+| OSV | `storage.googleapis.com` |
+
+If outbound traffic must go through a proxy, see [Configuring an HTTP proxy](configuring-http-proxy.md). If the proxy
+intercepts TLS, see [Configuring internal CA trust](configuring-internal-ca.md).
+
+The GitHub advisories source also requires a GitHub personal access token (PAT). The token needs no scopes, but the
+GitHub GraphQL API rejects unauthenticated requests. Create one at
+[github.com/settings/tokens](https://github.com/settings/tokens). NVD and OSV do not require credentials.
+
+## Enabling sources
+
+Open **Administration > Vulnerability Sources** in the web UI. Each source has its own configuration panel. The following steps
+cover the least configuration needed for findings.
+
+### NVD
+
+1. Open **Administration > Vulnerability Sources > NVD**.
+2. Enable the source.
+3. Select **Test** to verify the configured endpoint is reachable and that the feed file format matches what
+   Dependency-Track expects.
+4. Select **Mirror now** to download the feed immediately. The first mirror downloads the entire dataset and can take a
+   while.
+
+![NVD configuration](../../assets/images/guides/administration/configuring-vulnerability-sources/nvd.png)
+
+NVD records describe affected products as CPEs. The internal analyzer skips components that lack a valid CPE when
+evaluating NVD data. If you expect findings for open source packages identified by PURL and see none from NVD, that is
+the reason. See [How component matching
+works](../../concepts/about-vulnerability-data-sources.md#how-component-matching-works).
+
+### GitHub advisories
+
+1. Open **Administration > Vulnerability Sources > GitHub**.
+2. Enable the source.
+3. Enter the GitHub PAT from the prerequisites.
+4. Select **Mirror now** to download advisories immediately.
+
+![GitHub Advisories configuration](../../assets/images/guides/administration/configuring-vulnerability-sources/github.png)
+
+### OSV
+
+1. Open **Administration > Vulnerability Sources > OSV**.
+2. Enable the source.
+3. Select the ecosystems you want to mirror. Dependency-Track downloads only the ecosystems you select.
+4. Select **Mirror now** to download the selected ecosystems immediately.
+
+![OSV configuration](../../assets/images/guides/administration/configuring-vulnerability-sources/osv.png)
+
+!!! tip
+    For Debian, select the **Debian** ecosystem rather than individual Debian version ecosystems. The Debian package is
+    a superset of all version-specific ones.
+
+## Triggering an initial mirror
+
+After enabling a source for the first time, use **Mirror now** rather than waiting for the next scheduled run. The first
+NVD mirror in particular can take a long time, and you want it underway before users start uploading BOMs and expecting
+findings.
+
+Mirror progress and errors appear in the API server logs, so tail the logs during initial setup if you need to follow
+what each mirror is doing.
+
+## Scheduling mirror runs
+
+Each source has its own cron property. Mirrors also run on instance startup. To change the schedule, set the
+corresponding property:
+
+- NVD: [`dt.task.nist.mirror.cron`](../../reference/configuration/properties.md#dttasknistmirrorcron)
+- GitHub advisories:
+  [`dt.task.git.hub.advisory.mirror.cron`](../../reference/configuration/properties.md#dttaskgithubadvisorymirrorcron)
+- OSV: [`dt.task.osv.mirror.cron`](../../reference/configuration/properties.md#dttaskosvmirrorcron)
+
+## Verifying findings
+
+Once a mirror completes, upload a BOM for a project that contains components you know to be vulnerable, or trigger
+analysis on an existing project. Findings should appear within seconds of analysis completing. If they do not, check the
+API server logs for mirror errors and confirm the components carry the identifier the source uses for matching (CPE for
+NVD, PURL for GitHub advisories and OSV).
+
+## See also
+
+- [About vulnerability data sources](../../concepts/about-vulnerability-data-sources.md)
+- [Vulnerability datasources reference](../../reference/datasources/index.md)
+- [Vulnerability analyzers reference](../../reference/analyzers.md)
+- [Running air-gapped](running-air-gapped.md)
+- [Configuring an HTTP proxy](configuring-http-proxy.md)
+- [Configuring internal CA trust](configuring-internal-ca.md)

docs/guides/administration/migrating-from-v4.md

@@ -366,6 +366,7 @@ It drops analyzer and vulnerability-source credentials as part of the encrypted-
 After the migration, open the **Repositories**, **Analyzers**, and **Vulnerability Sources** pages in the v5 administrator panel
 and re-enter each credential through the secret manager.
 Turn the affected repositories back on.
+See [Configuring vulnerability sources](configuring-vulnerability-sources.md) for the full procedure to re-enable each mirror.
 See [Configuring secret management](configuring-secret-management.md) if you have not yet set the secret-encryption key for the v5 deployment.
 
 ### Encrypted property values