Repository navigation
ci(astro-kit): kit visual compares with --ci and reports on every pull request (PR B of 2) - #586
Merged
Merged
Conversation
…e, with --ci for a missing baseline The verify job drew the kit on ubuntu-latest against visual/linux, which was never committed: every baseline was written and the step could not fail on a pixel. The kit visual now runs in its own workflow inside mcr.microsoft.com/playwright:v1.63.0-noble (the lockfile's version, pinned by a test), only when the kit, the starter, the SDK, the lockfile or the workflow change, reading the repository only. visual.mjs --ci fails on a missing baseline (keeping the shot) instead of writing it; a hand-run with `update` uploads every linux baseline to commit.
…kit-visual run in Playwright's image 831 PNGs from run 38898912113 (mcr.microsoft.com/playwright:v1.63.0-noble), every fixture of the 35 components at 390/768/1280; a sample checked by eye (each component at 390 and 1280, the ja/ar/upper wordmarks, the serif heading hooks). From here the kit-visual run compares against them.
…l request (PR B of 2) The kit-visual job now runs visual.mjs --ci, so a drawing that moves, a size that changes or a fixture with no linux baseline fails. It is a required check: the workflow runs on every pull request, a light `changes` job reads the files the pull request changes, and kit-visual is skipped (which passes) unless the kit, the starter, the SDK, the lockfile or the workflow changed, so other pull requests do not pull the image. Only failing shots are uploaded, or every baseline on a hand-run with `update`.
…force # Conflicts: # .github/workflows/kit-visual.yml # packages/astro-kit/tests/kit-visual-ci.test.ts
✅ Deploy Preview for contentrain-ai ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
This was referenced Oct 16, 2026
Member
Author
|
Read @67919bf0 (t2): one-flag HOLD, otherwise OK.
Must-fix: add The 0.2% tolerance question is fine to leave as a separate decision. |
Member
Author
|
OK @4201dac5 (t2): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The kit visual CI, PR B of 2: CI now compares and fails, and
kit-visualcan be a required check. Reader: t2. GENERIC.#585 (PR A) put the kit visual in its own job inside Playwright's image and committed the first linux baselines (831 PNGs, compare run green with 0 written). This PR makes the job enforce them.
The change (
.github/workflows/kit-visual.yml)--cion every run except a hand-run withupdate. A drawing that moves (more than MAX_DIFF_RATIO, 0.2%), a size that changes, or a fixture with no linux baseline fails the job.paths:filter.changesjob (plain ubuntu, about 10 s) checks out withfetch-depth: 0and grepsgit diff --name-only <base> HEADfor the watched paths:packages/astro-kit/,templates/astro-starter/,packages/types/,packages/query/,scripts/local-sdk.mjs,pnpm-lock.yamland this workflow. A hand-run (no pull request) always draws.kit-visualhasneeds: changesandif: needs.changes.outputs.kit == 'true'. A job skipped by itsifcounts as passing for a required check, so a pull request outside those paths gets a passingkit-visualwithout pulling the image.*.actual.png,*.diff.png) go to the artifactastro-kit-visual-linux-failed. Withupdate, every baseline goes toastro-kit-visual-linux. A passing compare uploads nothing; before, it was 50 MB per run.tests/kit-visual-ci.test.tsnow also pins nopaths:filter, theifgate, every watched path in the grep, and--cion the run line (4/4).Branch protection
The check to require is
kit-visual(the job id, in workflow "Kit visual").changesreports too, but it does not need to be required. Changing the protection setting is ORK's, after this merges.Proof
kit-visualdraws and compares with--ci. It passed in 4m49s, with nothing written: run 38901105701.kit-visualmust fail and upload the failed shots.stats/three-plain@390 changed size (390×428 vs 392), and @768 and @1280 had 3183 px differ. The artifactastro-kit-visual-linux-failed(138 KB) held only those shots.kit-visualmust be skipped and pass.changespassed in 8 s, andkit-visualwas skipping (0 s), which counts as passing: run 38901174244.Changesets
None: CI only.