fix(ci): unblock the JS tooling audit gate without lowering it

[rob-archastro]

Review on ArchCode

Closes #45

Problem and author intent

npm audit --audit-level=moderate runs as step four of ci.yml, before Python is even installed, and again as a gate inside release.yml and regenerate-sdk.yml. Advisories published since this repo's last green run turned it red, so every PR fails before a single Python test executes, and the release workflow cannot reach its bump-and-publish steps.

The intent is to get the gate green and keep it meaningful — not to lower it.

The failure as a system story

Actors: the committed package-lock.json, GitHub's live advisory database, and the CI job.

PR opened (any diff, any branch)
  -> ci.yml step 2: npm ci --ignore-scripts
     -> ci.yml step 4: npm audit --audit-level=moderate
        -> reads committed package-lock.json
        -> queries the LIVE advisory database
           -> 9 vulnerabilities -> exit 1
  -> Setup Python        NEVER RUNS
  -> Ruff / Unit tests   NEVER RUNS

The audit never reads the diff. Its result is a function of today's date against a lockfile that has not changed, so the gate went red on its own: main last ran green on 2026-08-19 and has had no push since, leaving a stale green badge on a repo where the next PR from anyone fails identically. Verified by diffing the lockfile against main (identical) and reproducing the nine findings locally against it.

Because release.yml runs the same step, this also blocks shipping #44 and, downstream, ArchAstro/firstlanding#13724.

What changed

Two overrides that fix cleanly.

• fast-uri → 4.1.4. Note the shape of this one: package.json already pinned fast-uri to 4.1.2 as an earlier remediation, and the new advisory range is 4.0.0 - 4.1.2. The previous fix had become the vulnerability.
• qs → 6.16.0.

That takes 9 findings to 6, and moderate to zero.

The remaining six are one chain with no upgrade path.

@stoplight/prism-cli 5.16.0
└─ @stoplight/prism-http 5.16.0
   └─ @stoplight/http-spec 7.1.0
      └─ postman-collection 4.5.0
         └─ @faker-js/faker 5.5.3   ← GHSA-qxc2-j82w-r537, high

Everything that looks like a fix was tried and does not work:

• postman-collection@5.3.1 (latest) still pins @faker-js/faker: "5.5.3" exactly. Upgrading it changes nothing.
• @stoplight/prism-cli@5.16.0 is already the newest published version. There is nothing to upgrade into.
• Overriding @faker-js/faker to 10.5.0 does reach found 0 vulnerabilities — and breaks postman-collection/lib/superstring, so pytest tests/contract dies during collection and all 2616 contract tests become uncollectable. That trade is strictly worse than the red gate.

So the waiver is scoped, not global. npm audit has no way to waive a single advisory — the only knob is --audit-level, and raising it to high to clear this one would also hide the next real moderate. scripts/audit-js-tooling.mjs reads npm audit --json and fails on anything at moderate or above that is not explicitly listed, so the bar stays where it was for everything else. All three workflows that audit — ci.yml, release.yml, regenerate-sdk.yml — call it.

A report that failed to build is refused rather than parsed: a registry failure also exits non-zero, with an {"error": ...} body and no vulnerabilities map, which would otherwise read as a clean tree whose waiver had gone stale.

The waiver for GHSA-qxc2-j82w-r537 records why it is acceptable, and the reasoning is about reachability, not convenience:

• The advisory is faker.helpers.fake executing a caller-supplied template. We never call it.
• tests/contract/conftest.py starts prism mock <openapi.json> against a spec we generate ourselves, with no --dynamic flag — the file's own comment already says static responses are deliberate. No Postman collection is ever parsed, so postman-collection never runs, and faker generates nothing.
• Prism is a devDependency of a private: true tooling package. It is not in the published wheel.

A waiver cannot quietly outlive its justification. Each entry carries a reason and an expires date, and the script fails when an entry expires and when an entry stops matching any advisory — so a stale waiver is a build failure telling you to delete it, not a silent mute.

Intentionally unchanged: the moderate threshold, every other override, and all Python tooling.

Testing

node scripts/audit-js-tooling.mjs — passes:

JS tooling audit clean at moderate and above (waived: GHSA-qxc2-j82w-r537).

A gate that only passes proves nothing, so all three failure paths were exercised against the real report:

1. Unwaived advisory (waiver key renamed) → exit 1, HIGH GHSA-qxc2-j82w-r537 (@faker-js/faker) — Faker: helpers.fake exploitable into arbritary code execution
2. Expired waiver (expires set to 2026-01-01) → exit 1, waiver expired 2026-01-01; re-review it
3. Stale waiver (entry matching nothing) → exit 1, waiver no longer matches any advisory; delete it from ALLOWED
4. Unreachable registry (stubbed {"error": {"code": "ENETUNREACH"}}) → exit 1, npm audit could not produce a report: ..., and crucially not a stale-waiver message

The point of the change is that Prism keeps working, so that is verified directly rather than assumed:

• npm ci --ignore-scripts then uv run pytest tests/contract — 2616 passed
• uv run pytest tests/test_http_client.py tests/harness — 48 passed
• uv run ruff check — clean; uv run ruff format --check — 157 files already formatted

Those are the same gates release.yml runs before bumping, so the release path is verified end to end on this branch.

There is no new automated test file. The gate script is the check, it runs on every PR and every release through the two workflow steps this PR rewires, and its own failure modes were exercised by hand as listed above. Calling that a regression test would overstate it.

Scope and risk

CI and dev tooling only. Nothing in src/, nothing in the published wheel. Risk: low — the worst case is the gate being wrong about a waiver, and it fails closed in every direction tested above.

User impact

None. No SDK behaviour changes.

Follow-ups

• Merging this unblocks fix(http): encode list query params as key[]=item so the server keeps every value #44 (list query param encoding), which must be released before ArchAstro/firstlanding#13724 can bump its pins.
• The faker waiver expires 2026-12-31. The durable fix is a mock server whose dependency tree we control; if this chain re-breaks before then, that is the signal to do it rather than extend the date.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MMtJ4fwgPjiG8Zcv6bEkBi

[rob-archastro]

Orchestrator review (read-only, verified on b6b6243): GO once one line is fixed. The gate script, the two overrides, the lockfile delta, and the waiver reasoning all check out; the failure paths listed in the description are the right ones and the script fails closed on each. The reachability argument holds against tests/contract/conftest.py: Prism is started as prism mock <spec> --port --host with no --dynamic, so no Postman collection parsing and no faker template execution.

Defect (must fix in this PR): the description says the audit runs in two workflows, but there are three. .github/workflows/regenerate-sdk.yml:50 still runs npm audit --audit-level=moderate after npm install --package-lock-only, so the next manual regenerate dispatch fails on the same faker advisory this PR waives. Swap it to node scripts/audit-js-tooling.mjs like the other two.

Non-blocking:

• If npm audit --json cannot reach the registry it exits non-zero with an {"error": …} body; the script then parses an empty vulnerabilities map and reports the faker waiver as stale ("delete it from ALLOWED"), which is fail-closed but misleading. A one-line if (report.error) throw before the loop makes the message truthful.
• package.json description: the em dash was re-escaped to \u2014 with no semantic change; npm will likely rewrite it back on the next install. Worth reverting to keep the diff to the two overrides.
• No automated test for the script is an honest call at this size; the by-hand failure-path exercise in the description is adequate.

[rob-archastro]

All three addressed in d9a99ff. Verified each claim against source before changing anything.

Defect — fixed. .github/workflows/regenerate-sdk.yml:50 was indeed still on the raw npm audit --audit-level=moderate, and you're right about the consequence: the next manual regenerate dispatch would have failed on the exact advisory this branch waives, after npm install --package-lock-only had already rewritten the lockfile. Now node scripts/audit-js-tooling.mjs, same as the other two. All three workflows:

.github/workflows/ci.yml:36:            run: node scripts/audit-js-tooling.mjs
.github/workflows/release.yml:46:       run: node scripts/audit-js-tooling.mjs
.github/workflows/regenerate-sdk.yml:50: run: node scripts/audit-js-tooling.mjs

I've corrected "two workflows" to three in the PR description as well — that was wrong in the same way the code was.

Registry failure — fixed. Confirmed the misleading path and added an explicit refusal before the loop. Exercised it with a stubbed {"error": {"code": "ENETUNREACH", ...}} body:

Error: npm audit could not produce a report: request to https://registry.npmjs.org/-/npm/v1/security/advisories/bulk failed
exit=1

Still fail-closed, and it no longer tells you to delete a live waiver — grep -ci ALLOWED on the output is 0.

Em dash — reverted. You were right about the cause: rewriting package.json through json.dump defaults to ensure_ascii=True, which escaped it. git diff origin/main -- package.json is now exactly the two override lines and nothing else.

Re-verified after the changes: gate passes (clean at moderate and above (waived: GHSA-qxc2-j82w-r537)), unwaived-advisory and expired-waiver paths both still exit 1, tests/contract 2616 passed, unit+harness 48 passed, ruff check and format clean.

Agreed on leaving the script without an automated test at this size.