48 screenshot comparisons across three viewports and two themes, run inside a pinned container so the pixels are identical on every machine — with a published baseline / actual / diff gallery.
Functional tests assert that a button exists, is labelled, and can be clicked. None of that notices when the button turns the wrong shade of grey, loses its corner radius, or shifts 40px left on tablet. This is the suite that does.
Three plausible CSS changes were injected — a lighter tile label, a card radius drifting from the design system's 14px, a stat value losing a little size and its tabular figures. Every element still renders, still carries its test id, still passes every functional assertion.
Nine screenshot comparisons failed.
Reproduce it yourself — this is a real code path, not a mock-up:
npm run demo:regressionThat serves the broken stylesheet, runs the desktop project, and builds the gallery below into gallery/index.html.
Every failed comparison becomes a card: baseline, actual, and the pixel diff, plus a slider to wipe between before and after. It is generated from Playwright's own artefacts by a zero-dependency script and published to GitHub Pages on every run.
Two pages are published on every run:
- The latest run — on a healthy build this is, correctly, empty
- A worked example — the injected regression above, with all nine diffs and the wipe slider
This is the decision the whole repository rests on.
Font rasterisation differs between operating systems — the same page on macOS and on Linux differs by thousands of pixels, and that difference is a property of the renderer, not a regression. A baseline is therefore only meaningful against the renderer that produced it.
So the suite runs inside mcr.microsoft.com/playwright:v1.62.1-noble, and CI runs the job in that same image. A developer on macOS, a developer on Windows and the pipeline all compare identical pixels.
npm test # runs in the container — the supported path
npm run test:update # re-record the baselines after an intended change{platform} is kept in the snapshot path template as a safety net: a run on the host writes …-darwin.png and can never silently overwrite the committed …-linux.png baselines that CI compares against.
| Capability | Where to look |
|---|---|
| Containerised, reproducible baselines | scripts/docker-test.sh, ci.yml |
| Full-page snapshots in both themes | tests/full-page.spec.ts |
| Component-level snapshots for failure isolation | tests/components.spec.ts |
| Masking genuinely dynamic content | tests/components.spec.ts |
| Hover / focus state coverage | tests/states.spec.ts |
| Determinism fixture (animations, fonts, scroll, caret) | src/fixtures.ts |
| Tolerance tuning, and why | playwright.config.ts |
| Diff gallery generator | scripts/build-gallery.mjs |
| Design-system assertions that are not screenshots | tests/accessibility.spec.ts |
| Spec | Comparisons | Focus |
|---|---|---|
full-page |
2 × 3 viewports | Whole-page layout, light and dark |
components |
7 × 3 viewports | Tiles, three charts, table, top bar, plus a masked variant |
states |
3 × 3 viewports | Focus ring, selected filter, post-toggle repaint |
accessibility |
4 assertions | Chart alt text, legend presence, status-plus-label, no redundant legend |
48 screenshot comparisons + 12 assertions, in about 12 seconds.
Requires Docker and Node 20+.
npm ci
npm testnpm run test:update # re-record baselines after an intended change
npm test -- --project=mobile # one viewport
npm run demo:regression # break it on purpose, then build the gallery
npm run gallery # build the gallery from the last run
npm run serve # serve the app at :4173 to look at it
npm run typecheckNo retries. A visual test that passes on retry is not flaky — it is comparing against a baseline that no longer describes the page. Hiding that behind a retry is how a suite stops meaning anything.
Tolerance is a budget, not a dial. maxDiffPixels: 120 absorbs sub-pixel antialiasing without letting a real change through. A two-pixel shift is noise; a moved button is not. Raising the threshold until the suite goes green is the failure mode this guards against.
The app signals when it is done. body[data-render-complete="true"] is set as the last statement of the render. Waiting on an explicit signal beats networkidle on a page that makes no network requests after load.
Determinism is designed in, not patched on. The app under test has no Date.now(), no Math.random(), no network call and no entry animation. The fixture then disables transitions, forces instant scrolling, hides the caret and awaits document.fonts.ready — each closing a specific, known source of run-to-run drift.
Masking is for content that legitimately changes — a run timestamp, a build number. The layout around it is still verified. Reaching for a mask to silence a real difference is the misuse, so the one masked test says in the spec exactly why it is masked.
Both page-level and component-level snapshots. The page-level check catches layout shifts between components; the component-level ones localise a failure once it happens. A full-page snapshot alone tells you something changed; it does not tell you what.
Some rules are assertions, not screenshots. "Every chart has a text alternative" and "status is never colour alone" are structural facts. A pixel comparison would flag them for the wrong reasons, so they are asserted directly.
A QA metrics dashboard, built for this repository: stat tiles, a pass-rate trend line, a stacked outcome chart, a duration ranking and a flaky-test table, in light and dark themes, responsive down to 390px.
It follows a documented visualisation method rather than personal taste: a single hue for magnitude, a reserved status palette for pass/flaky/fail that is never reused as a series colour, an icon-and-word pairing wherever a status appears so colour never carries meaning alone, a legend for the one multi-series chart and none for the single-series one, and selective direct labels rather than a number on every point. Those rules are what accessibility.spec.ts holds in place.
- The app under test is part of this repository, so the baselines cannot be invalidated by someone else's deployment — a deliberate contrast with the Playwright and Selenium suites, which drive a public demo site.
- This repository contains no code, data, or credentials from any employer. Every line was written for this portfolio.
- playwright-ts-ui-framework — UI automation in Playwright + TypeScript
- rest-api-automation-suite — API testing with contract schemas
- selenium-java-testng-framework — Selenium + TestNG in Java
MIT