Skip to content

ci: move dependency audits off unrelated PRs, add daily sweep - #1063

Merged
SnowboardTechie merged 13 commits into
mainfrom
bryan/scoped-dep-audit
Aug 18, 2026
Merged

ci: move dependency audits off unrelated PRs, add daily sweep#1063
SnowboardTechie merged 13 commits into
mainfrom
bryan/scoped-dep-audit

Conversation

@SnowboardTechie

@SnowboardTechie SnowboardTechie commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • A newly published advisory against a dependency already on main currently reds every open PR that touches a package's source, even when the PR changes no dependencies. This moves audits to where dependencies actually change, plus a daily sweep on main.
  • Time to review: 10 minutes

Changes proposed

  • Removes the audit step from the five per-package CI workflows (ci-lib-core, ci-lib-cli, ci-lib-ts-sdk, ci-lib-changelog-emitter, ci-website-preview). These ran on any PR touching the package's source, so an advisory that could only be fixed on main blocked every in-flight PR for that package.
  • Adds deps-audit.yml: a daily workspace-wide pnpm audit on main (15:00 UTC, plus workflow_dispatch), opening a tracking issue labeled dependencies on failure or commenting on the existing one. Advisories that drop with no dependency PR in flight are caught within a day. An advisory with no upstream fix yet is acknowledged through audit.ignore, documented below, which stops both the failing job and the repeat comments until a patched version ships.
  • Documents the audit topology in DEPENDENCY_MANAGEMENT.md: where each audit runs, what severity it blocks on, why per-package CI no longer audits, and how to handle an advisory with no upstream fix.

Context for reviewers

pnpm dependency-changing PRs stay gated. Any pnpm dependency change in a workspace package updates the root pnpm-lock.yaml, which triggers the audit in ci-catalog-validation.yml. That gate is genuinely workspace-wide, not root-importer-only: pnpm audit --json at the root reports 1269 audited dependencies across the pnpm workspace, currently with zero advisories at any severity. The Python SDK resolves through its own poetry.lock and currently has no vulnerability-audit gate, as documented below.

Severity thresholds differ by surface, deliberately. The removed steps failed at low (the four lib packages ran bare pnpm audit) and at high (the website's audit:high). Dependency-changing PRs are gated at moderate by catalog validation, its existing setting. The daily sweep runs at low, so low-severity advisories still surface within a day — as a tracking issue rather than a PR blocker.

The HOLD-branch gating carve-out is subsumed, not lost. The if: github.base_ref == 'main' guard existed so advisories fixable only on main wouldn't block PRs into HOLD-* batching branches. Audits no longer run on per-package PRs at any base, which covers the same case more directly.

Out of scope, unchanged: the two Node templates keep their own audits (standalone projects outside the workspace, already tightly path-scoped). The fast-api template and the two example APIs keep poetry audit non-blocking — they sit in the manual maintenance tier, so no automated PR is queued to fix what a gating audit would flag.

Additional information

deps-audit.yml triggers on changes to itself, so the run on this PR exercises the real sweep end to end:

Run pnpm audit
No known vulnerabilities found

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

🚀 Website Preview Deployed!

Preview your changes at: https://pr-1063-common-grants.brian-derfer.workers.dev

Built from commit ce54418

New advisories against deps already on main were redding every open PR
that touched a package's source, even when the PR changed no deps.

- Per-package CI workflows no longer run the audit; it only ever ran on
  PRs there, and dep-changing PRs are already gated by the audit in
  ci-catalog-validation.yml (every dep change updates the root lockfile,
  which triggers it).
- audit-deps.js gains --recursive: plain `pnpm list` at the workspace
  root only covers the root importer (~49 pkgs), so the no-filter audit
  in ci-catalog-validation.yml was silently auditing almost nothing.
  With --recursive it covers all workspace projects (~800 pkgs).
- New deps-audit.yml runs the workspace-wide audit daily on main and
  opens/updates a tracking issue on failure, so advisories that drop
  with no dep PR in flight are still caught within a day.

Templates and examples keep their own audits: they are standalone
projects outside the workspace, tightly path-scoped already.
- postcss@<=8.5.22 -> >=8.5.23 (GHSA-fxqj-rqcc-2cmp, moderate) via
  website>@astrojs/react>vite>postcss; resolves to 8.5.25.
- body-parser@<1.20.6 -> >=1.20.6 <2 (GHSA-v422-hmwv-36x6, low) via
  lib/cli>express; bounded below 2 so express 4 keeps the 1.x line.

Verified: pnpm audit and audit-deps.js --recursive both clean; cli and
website test suites green against the new resolutions.
@SnowboardTechie
SnowboardTechie force-pushed the bryan/scoped-dep-audit branch from ececad9 to 1ce18cc Compare August 4, 2026 16:09
@github-actions github-actions Bot added the dependencies Pull requests that update a dependency file label Aug 4, 2026
@SnowboardTechie
SnowboardTechie marked this pull request as ready for review August 4, 2026 16:21
# Conflicts:
#	.github/scripts/audit-deps.js
#	.github/workflows/ci-catalog-validation.yml
#	.github/workflows/ci-lib-changelog-emitter.yml
#	.github/workflows/ci-lib-cli.yml
#	.github/workflows/ci-lib-core.yml
#	.github/workflows/ci-lib-ts-sdk.yml
#	.github/workflows/ci-website-preview.yml
#	pnpm-lock.yaml
#	pnpm-workspace.yaml
The validate job no longer runs an audit step, so the comment describing
checks/tests/audit was stale.

@karinamzalez karinamzalez left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving with questions!

  • In the case that the issue creation step fails for some reason, would someone get notified? / do we get slack pings or something when scheduled workflows fail in general?
  • re living documentation: would we want to add a few lines to DEPENDENCY_MANAGEMENT.md?
    • sidenote, PR description mentions that continue-on-error rule is in there but I'm not finding it.

Also! re: the second opinion topics from your PR description:

  • severity threshold sounds good to me! Just thinking if all the lows now go to the daily sweep.. there's no way to say "awknoledged but no upstream fix yet".. So one unpatchable low could mean a failing job and a new "Still failing" comment every day. Maybe we could run the sweep at moderate, or add a way to acknowledge a finding?
  • HOLD carve-out-- agreed!

An advisory with no upstream fix would otherwise draw a "still failing"
comment every day. One open issue carries the signal; the body links the
workflow run list so it stays current instead of pinning a stale run.
DEPENDENCY_MANAGEMENT.md had no audit content at all, so the topology
lived only in workflow comments. Adds a section covering each audit, its
threshold, and why per-package CI no longer audits.

The non-blocking poetry audits cited a rule that section did not contain
("Tier 2 manually-maintained", "CVE tracking handled outside CI"). Cites
now point at the Maintenance tiers section, which does back the claim.
# Conflicts:
#	.github/workflows/ci-website-preview.yml
Suppressing a GHSA defers the problem, so an ignore entry now needs an
upstream issue to watch and a tracking issue here that carries the bump
and the removal once a fix ships.
@SnowboardTechie

SnowboardTechie commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

On the issue-creation failure: no, a failed filing is indistinguishable from a successful one. The step only fires when the audit step itself fails, so the job is already red from that, and a non-zero gh issue create doesn't change the run conclusion. Prerequisite failures like checkout or install skip the filing entirely now and surface only through that same email. The only signal left is the scheduled-run failure email, which GitHub sends solely to whoever last modified the schedule: cron block, so it reaches me and no one else. The advisory itself survives, since the next day's run re-attempts the filing, but the filing failure is invisible.

No Slack pings connected today. Nothing routes Actions failures into Slack, so a failed scheduled run stays in the Actions tab until someone opens it. deps-catalog-check failed on schedule on 2026-08-10 and is still untouched.

Good catch on the DEPENDENCY_MANAGEMENT.md reference. That rule wasn't in the file, or anywhere else. The poetry audit steps were citing the doc for a claim it never made. There's now a ## Dependency audits section covering where each audit runs, what severity it blocks on, and why the per-package workflows no longer audit, and those comments now point at the Maintenance tiers section that does back them.

The acknowledgment path already exists, it just wasn't written down. pnpm reads the ignore list out of pnpm-workspace.yaml under audit.ignore, which replaced auditConfig.ignoreGhsas in 11.16.0, and that older spelling is what #710 cleared the last two stale entries from. I added it to the doc section in this PR, with a requirement attached: an ignore entry needs an issue in the upstream repo tracking the fix, and an issue here to watch it and do the bump when it lands. Once a GHSA is on the list the job goes green and the daily comments stop. The sweep never closes its own issue though, so that one stays open until the fix ships and the ignore comes out.

`gh issue list` resolves both --search and --label through GitHub issue
search, which is index-backed, so two runs close together can each miss a
just-filed issue and open a duplicate. Reading /issues by label goes
straight through. The endpoint returns PRs too, hence the filter.
failure() is true after any earlier step, so a checkout, setup or install
failure filed an issue asserting advisories were found when the audit
never ran. Gate on the step outcome instead; prerequisite failures stay
visible through the scheduled-run notification. A concurrency group
serializes the list-then-file so overlapping runs cannot double-file.

Docs: audit.ignore replaces auditConfig.ignoreGhsas as of pnpm 11.16.0,
and the topology now says the Python SDK has no audit gate rather than
implying every dependency change reaches the root lockfile.
@SnowboardTechie

Copy link
Copy Markdown
Collaborator Author

@karinamzalez your dedup note on the sibling PR applied here too, so the sweep looks its issue up by label over REST now, with a concurrency group on top. Same reasoning as my reply there.

The issue step was also gated on failure(), so a checkout or install failure would have filed an issue claiming advisories were found. It's on the audit step's own outcome now.

@SnowboardTechie
SnowboardTechie merged commit f616085 into main Aug 18, 2026
13 checks passed
@SnowboardTechie
SnowboardTechie deleted the bryan/scoped-dep-audit branch August 18, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants