ci: move dependency audits off unrelated PRs, add daily sweep - #1063
Conversation
|
🚀 Website Preview Deployed! Preview your changes at: https://pr-1063-common-grants.brian-derfer.workers.dev Built from commit ce54418 |
New advisories against deps already on main were redding every open PR that touched a package's source, even when the PR changed no deps. - Per-package CI workflows no longer run the audit; it only ever ran on PRs there, and dep-changing PRs are already gated by the audit in ci-catalog-validation.yml (every dep change updates the root lockfile, which triggers it). - audit-deps.js gains --recursive: plain `pnpm list` at the workspace root only covers the root importer (~49 pkgs), so the no-filter audit in ci-catalog-validation.yml was silently auditing almost nothing. With --recursive it covers all workspace projects (~800 pkgs). - New deps-audit.yml runs the workspace-wide audit daily on main and opens/updates a tracking issue on failure, so advisories that drop with no dep PR in flight are still caught within a day. Templates and examples keep their own audits: they are standalone projects outside the workspace, tightly path-scoped already.
- postcss@<=8.5.22 -> >=8.5.23 (GHSA-fxqj-rqcc-2cmp, moderate) via website>@astrojs/react>vite>postcss; resolves to 8.5.25. - body-parser@<1.20.6 -> >=1.20.6 <2 (GHSA-v422-hmwv-36x6, low) via lib/cli>express; bounded below 2 so express 4 keeps the 1.x line. Verified: pnpm audit and audit-deps.js --recursive both clean; cli and website test suites green against the new resolutions.
ececad9 to
1ce18cc
Compare
# Conflicts: # pnpm-lock.yaml
# Conflicts: # .github/scripts/audit-deps.js # .github/workflows/ci-catalog-validation.yml # .github/workflows/ci-lib-changelog-emitter.yml # .github/workflows/ci-lib-cli.yml # .github/workflows/ci-lib-core.yml # .github/workflows/ci-lib-ts-sdk.yml # .github/workflows/ci-website-preview.yml # pnpm-lock.yaml # pnpm-workspace.yaml
The validate job no longer runs an audit step, so the comment describing checks/tests/audit was stale.
karinamzalez
left a comment
There was a problem hiding this comment.
Approving with questions!
- In the case that the issue creation step fails for some reason, would someone get notified? / do we get slack pings or something when scheduled workflows fail in general?
- re living documentation: would we want to add a few lines to DEPENDENCY_MANAGEMENT.md?
- sidenote, PR description mentions that
continue-on-errorrule is in there but I'm not finding it.
- sidenote, PR description mentions that
Also! re: the second opinion topics from your PR description:
- severity threshold sounds good to me! Just thinking if all the lows now go to the daily sweep.. there's no way to say "awknoledged but no upstream fix yet".. So one unpatchable low could mean a failing job and a new "Still failing" comment every day. Maybe we could run the sweep at moderate, or add a way to acknowledge a finding?
- HOLD carve-out-- agreed!
An advisory with no upstream fix would otherwise draw a "still failing" comment every day. One open issue carries the signal; the body links the workflow run list so it stays current instead of pinning a stale run.
DEPENDENCY_MANAGEMENT.md had no audit content at all, so the topology
lived only in workflow comments. Adds a section covering each audit, its
threshold, and why per-package CI no longer audits.
The non-blocking poetry audits cited a rule that section did not contain
("Tier 2 manually-maintained", "CVE tracking handled outside CI"). Cites
now point at the Maintenance tiers section, which does back the claim.
# Conflicts: # .github/workflows/ci-website-preview.yml
… daily" This reverts commit cf142db.
Suppressing a GHSA defers the problem, so an ignore entry now needs an upstream issue to watch and a tracking issue here that carries the bump and the removal once a fix ships.
|
On the issue-creation failure: no, a failed filing is indistinguishable from a successful one. The step only fires when the audit step itself fails, so the job is already red from that, and a non-zero No Slack pings connected today. Nothing routes Actions failures into Slack, so a failed scheduled run stays in the Actions tab until someone opens it. Good catch on the The acknowledgment path already exists, it just wasn't written down. pnpm reads the ignore list out of |
`gh issue list` resolves both --search and --label through GitHub issue search, which is index-backed, so two runs close together can each miss a just-filed issue and open a duplicate. Reading /issues by label goes straight through. The endpoint returns PRs too, hence the filter.
failure() is true after any earlier step, so a checkout, setup or install failure filed an issue asserting advisories were found when the audit never ran. Gate on the step outcome instead; prerequisite failures stay visible through the scheduled-run notification. A concurrency group serializes the list-then-file so overlapping runs cannot double-file. Docs: audit.ignore replaces auditConfig.ignoreGhsas as of pnpm 11.16.0, and the topology now says the Python SDK has no audit gate rather than implying every dependency change reaches the root lockfile.
|
@karinamzalez your dedup note on the sibling PR applied here too, so the sweep looks its issue up by label over REST now, with a concurrency group on top. Same reasoning as my reply there. The issue step was also gated on |
Summary
maincurrently reds every open PR that touches a package's source, even when the PR changes no dependencies. This moves audits to where dependencies actually change, plus a daily sweep onmain.Changes proposed
ci-lib-core,ci-lib-cli,ci-lib-ts-sdk,ci-lib-changelog-emitter,ci-website-preview). These ran on any PR touching the package's source, so an advisory that could only be fixed onmainblocked every in-flight PR for that package.deps-audit.yml: a daily workspace-widepnpm auditonmain(15:00 UTC, plusworkflow_dispatch), opening a tracking issue labeleddependencieson failure or commenting on the existing one. Advisories that drop with no dependency PR in flight are caught within a day. An advisory with no upstream fix yet is acknowledged throughaudit.ignore, documented below, which stops both the failing job and the repeat comments until a patched version ships.DEPENDENCY_MANAGEMENT.md: where each audit runs, what severity it blocks on, why per-package CI no longer audits, and how to handle an advisory with no upstream fix.Context for reviewers
pnpm dependency-changing PRs stay gated. Any pnpm dependency change in a workspace package updates the root
pnpm-lock.yaml, which triggers the audit inci-catalog-validation.yml. That gate is genuinely workspace-wide, not root-importer-only:pnpm audit --jsonat the root reports 1269 audited dependencies across the pnpm workspace, currently with zero advisories at any severity. The Python SDK resolves through its ownpoetry.lockand currently has no vulnerability-audit gate, as documented below.Severity thresholds differ by surface, deliberately. The removed steps failed at
low(the fourlibpackages ran barepnpm audit) and athigh(the website'saudit:high). Dependency-changing PRs are gated atmoderateby catalog validation, its existing setting. The daily sweep runs atlow, so low-severity advisories still surface within a day — as a tracking issue rather than a PR blocker.The HOLD-branch gating carve-out is subsumed, not lost. The
if: github.base_ref == 'main'guard existed so advisories fixable only onmainwouldn't block PRs intoHOLD-*batching branches. Audits no longer run on per-package PRs at any base, which covers the same case more directly.Out of scope, unchanged: the two Node templates keep their own audits (standalone projects outside the workspace, already tightly path-scoped). The fast-api template and the two example APIs keep
poetry auditnon-blocking — they sit in the manual maintenance tier, so no automated PR is queued to fix what a gating audit would flag.Additional information
deps-audit.ymltriggers on changes to itself, so the run on this PR exercises the real sweep end to end: