feat(core): scans run under a wall-clock budget (#56) - #58
Merged
Conversation
`SCAN_TIMEOUT_MS` was declared and never read; the only bound on a scan was the 10 s per request, so a slow origin that answered every request stretched a scan without end (twilio.com: 1054 s in the first corpus pass). `runScan` now creates a budget from `timeoutMs` (default `SCAN_TIMEOUT_MS`, raised from 60 s to 180 s, which clears the corpus p95 of 109 s). Every request answers to the caller's signal and to the budget; only the caller's signal ends the scan. When the budget runs out, requests in flight abort, no further request is sent, and every audit not yet started, or still running, reports `na` tagged `skipped:scan-budget`. A running audit is withheld because a refused request reads to it as a broken link or a missing artifact: a claim about the clock, not the site. A 429 retry wait ends with the budget and keeps the observed 429. `conditions.budget` records limit, elapsed, exhausted and skipped count. The budget's cut counts with the gate's against the 0.35 unscored threshold. The CLI gains `--timeout <seconds>` (config key `timeout`), refuses bare, negative and non-numeric values, and prints one line when the budget went. Markdown and HTML reports show the budget beside the other conditions. Closes #56. Claude-Session: https://claude.ai/code/session_01RT21vyEPL6uSMWwUTytw6p
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Closes #56. A scan now runs under a wall-clock budget.
runScanreadstimeoutMs(defaultSCAN_TIMEOUT_MS, raised from 60 s to 180 s: it clears the corpus p95 of 109 s, where 60 s would have cut one site in five).0disables it; negative or NaN throwsRangeError.nataggedskipped:scan-budget. A running audit is withheld rather than believed: a refused request reads to it as a broken link or a missing artifact.conditions.budgetrecordslimitMs,elapsedMs,exhausted,skippedCount. The budget's cut counts with the gate's against the 0.35 unscored threshold; the reason names the budget.budget.formatBudget,budgetReason,skippedMassShareexported from core.--timeout <seconds>(config keytimeout, onAgentLighthouseConfig). Bare, negative and non-numeric values from the flag or the file are refused. One terminal line when the budget went. Markdown and HTML reports show the budget beside the other conditions.docs/cli.md,docs/config.md,docs/evidence/corpus.md, nightly workflow comment.Measured
twilio.com, the 1054 s outlier, scanned in 117 s with per-audit tracing on 2026-09-03; the slowest audit was
machine-discovery/llms-txt-links-validat 54 s. With--timeout 30the same scan stopped at 30.1 s, withheld 145 audits (2 of them mid-run) and reported no score with the budget named.Review
Two reviewer rounds. Blockers found and fixed: a budget expiring during a 429 backoff threw instead of returning a report; audits still running at the abort turned unsent requests into
fail(no-broken-links); the 429 path reported a throttling site as unreachable; the retry's abort race was dead on an already-aborted signal. Tests drive the cut from mocked slow audit-phase fetches with an abort-aware mock; the call-count comparison bypasses the origin cache.Changeset
minorfor core, cli, report (4.1.0), per the release instruction. Note: a scan of a slow origin that used to return a score after many minutes now returnsoverallScore: nullat 180 s unless--timeoutis raised; the changeset says so.Gate: build, test (5350), typecheck, lint, check:dossiers, check:requires, check:audit-map, format:check all green.
https://claude.ai/code/session_01RT21vyEPL6uSMWwUTytw6p