ci: a test-only change under studio or player needs no captures - #4956
Conversation
Tests are not behaviour a user sees, like the markdown the check already skips. A bench-harness PR (#4953) was asked for Before and After screenshots.
Edit accuracy: accurate 1216 (base branch 1216), smooth 1108 of thoseThe gate passes. Quarantined, measured but not gated (1)
|
terencecho
left a comment
There was a problem hiding this comment.
Approved at 67a14da3aaee189e3b663f9ef1c03a689106f68a on code merit. The captures gate now excludes nested test/spec files and Markdown from the watched Studio/player diff before applying the visual-change and media rules; a mixed test-plus-production diff still enforces those rules. The added tests cover nested bench/test paths, Markdown, and the mixed production case, while existing visual rejection remains tested. I checked the exact-head script and workflow wiring; all 11 required checks passed, including captures and test reachability. I reviewed source via the GitHub API and did not run local tests.
— Review by tai (pr-review)
What
The Studio and player captures check (
scripts/check-pr-captures.mjs) asked for Before and After screenshots on PRs that only change tests underpackages/studioorpackages/player. It already skipped markdown there as "docs, not behaviour"; tests are the same kind of file. Paths under atests/or__tests__/directory and files named*.test.*or*.spec.*are no longer watched, so a test-only diff passes with nothing to show, and a test file no longer spends the "No visible change" line budget.Why
#4953 changes only the edit accuracy bench harness (
packages/studio/tests/e2e/edit-accuracy/). The check counted its 44 lines as a Studio change and failed it, although nothing a user sees changes.Related work
Refs #4953.
How
One test-path pattern next to
DOC_FILE, applied inisWatched, the single filter every diffed path goes through (parseNumstat). The oldTEST_FILEpattern only exempted*.test|spec.[jt]sx?from the visual-file rule (it missed.mjsand thetests/tree); it goes, since a test path never reaches that rule now. The existing case that passed a.test.tsxstraight toevaluatenow goes throughparseNumstat, as real runs do.Test plan
.mjsundertests/, a.spec.tsxand a.test.tsare filtered out and an empty body passes. Fails on main (all four files kept), passes here.node --test scripts/check-pr-captures.test.mjs: 75 pass, 0 fail, 3 runs in a row.No visible change
CI script only.
Size
One pattern and one test case.