chore(e2e): run Playwright visual tests in Docker (#DS-5311) by lskramarov · Pull Request #1845 · koobiq/angular-components

lskramarov · 2026-08-05T10:12:11Z

No description provided.

github-actions · 2026-08-05T10:16:57Z

🚨 E2E tests failed

Review the report for details.

💡 Comment /approve-snapshots to approve snapshot changes.

github-actions · 2026-08-05T10:16:58Z

Visit the preview URL for this PR (updated for commit b49c949):

https://koobiq-next--prs-1845-6woeds21.web.app

_{(expires Sat, 08 Aug 2026 15:47:20 GMT)}

_{🔥 via Firebase Hosting GitHub Action 🌎}

_{Sign: c9e37e518febda70d0317d07e8ceb35ac43c534c}

artembelik · 2026-08-05T10:26:35Z

+yarn run e2e:docker:update-snapshots    # Run E2E tests in Docker and update the baselines
 ```

+The committed baselines under `__screenshots__` are compared with `threshold: 0` and have no


это не нужно в AGENTS.md, дубль из packages/e2e/README.md

Одно для человека другое для агента, так же ? и тут скорее из ридми нужно выбрасывать и для агента больше писать..

github-actions · 2026-08-05T11:03:02Z

🚨 E2E tests failed

Review the report for details.

💡 Comment /approve-snapshots to approve snapshot changes.

lskramarov · 2026-08-05T12:11:42Z

/approve-snapshots

github-actions · 2026-08-05T12:12:02Z

🔄 Updating snapshots.

github-actions · 2026-08-05T12:18:35Z

⚠️ No snapshot changes detected.

NikGurev · 2026-08-05T13:19:34Z

Two things worth resolving before merge:

1. assert-browsers.js doesn't validate webkit

tools/e2e/assert-browsers.js only checks for chromium and chromium-headless-shell against the image's /ms-playwright contents. But e2e:setup in package.json installs both chromium and webkit — if any visual spec runs against webkit, a stale image digest would silently ship a mismatched webkit build with no build-time failure, unlike chromium which is caught. Could you either add webkit to the required array, or confirm webkit isn't exercised by this suite and drop a comment explaining why it's excluded?

2. No layer caching for the Docker build in CI

tools/e2e/run.js always runs docker compose run --rm --build, and I don't see actions/cache, a BuildKit registry cache (--cache-from/--cache-to), or a prebuilt/pushed image anywhere in the workflows. GitHub-hosted runners don't carry Docker layer cache between jobs, so every CI run seems to pay the full cost of the Node install, yarn install --immutable, and the font layer from scratch. Given the goal here was partly to cut CI time (removing the ~174MB Playwright browser download), it'd be good to confirm actual before/after wall-clock numbers, since a full image rebuild every run could offset that savings.

Copilot

Pull request overview

This PR makes Playwright visual regression testing reproducible by running the component E2E suite inside a Docker image that matches CI’s OS/font stack and the exact @playwright/test version. It also updates CI to run the same Docker-based workflow and documents the supported local/CI snapshot update path.

Changes:

Add a Docker image + docker compose runner (e2e:docker) to run Playwright component tests in a CI-matching environment and safely update committed baselines.
Update GitHub Actions E2E workflows to execute the suite (and snapshot approval flow) via the Docker runner instead of installing browsers on the runner.
Add worker-count override validation in playwright.config.ts, plus documentation and repo attributes/ignore rules to support reliable Docker builds and screenshot handling.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tools/e2e/run.js	Wrapper that validates prerequisites, derives Playwright version from `package.json`, and runs the suite via Compose (with an update-snapshots overlay when needed).
tools/e2e/Dockerfile	Digest-pinned Playwright base image with deterministic fonts, Yarn 4 shim, Node version aligned to `.nvmrc`, and a browser-revision assertion.
tools/e2e/docker-compose.yml	Compose service for running the suite with sensible defaults (platform pin, worker cap, bind-mounted outputs).
tools/e2e/docker-compose.update.yml	Overlay that mounts `packages/components` only when updating snapshots, to write baselines back to the working tree.
tools/e2e/assert-browsers.js	Build-time check ensuring the image’s baked browsers match what `playwright-core` expects for the installed version.
playwright.config.ts	Adds validated `PLAYWRIGHT_WORKERS` override handling to prevent silent “0 tests ran” success cases.
packages/e2e/README.md	Documents Docker as the supported path for visual tests and snapshot regeneration.
package.json	Adds `e2e:docker` and `e2e:docker:update-snapshots` scripts.
docs/guides/06-testing.md	Expands testing docs with Docker-based visual testing guidance and worker-count rationale.
AGENTS.md	Adds the Docker E2E commands and guidance to avoid non-Linux baseline regeneration.
.github/workflows/e2e.yml	Runs component E2E tests via `npm run e2e:docker` (container builds/installs internally).
.github/workflows/e2e-approve-snapshots.yml	Regenerates baselines via `npm run e2e:docker:update-snapshots` using the same container as the comparator job.
.gitattributes	Forces LF for `tools/e2e/**` and marks PNGs as binary to protect baseline bytes.
.dockerignore	Limits Docker build context size and excludes transient/local outputs while keeping committed baselines included.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

artembelik · 2026-08-06T13:00:46Z

не совсем понятен смысл этого скрипта, какую проблему он решает? ведь mcr.microsoft.com/playwright образ уже содержит все необходимые браузеры

artembelik · 2026-08-06T13:37:47Z

+# library on the Angular build's hot path, which costs about 5 seconds per run across the host
+# filesystem boundary (measured: 27s against 32s for the same spec). Updating snapshots is rare and
+# can afford both; a plain run should pay for neither.
+services:


обновление скриншотов и сами тесты запускается на разных platform (смотри) , могут быть расхождения, когда падает определенный скриншот тест, а при обновлении обновятся N других

в дата-грид используется единый docker-compose для обоих сценариев, мб и здесь поступим также?

artembelik · 2026-08-06T13:49:53Z

+    # by mount target in some versions and replaced them wholesale in others; spelling all three
+    # out is correct either way, and a repeated identical target is a no-op when they are merged.
+    volumes:
+      - ../../packages/components:/app/packages/components


в доке есть е2е тесты, но пока нет скриншотов, я к тому, что артефакты из документации тоже можно возвращать (это минор, можно будет при необходимости добавить)

artembelik · 2026-08-06T14:01:27Z

удобный подход, когда ты игнорируешь все по умолчанию, и кладешь в образ только то, что действительно необходимо, пример:

# Exclude everything by default, then explicitly allow only what is needed. ** # Yarn Berry binary and configuration !.yarn/releases/ !.yarn/releases/** !.yarnrc.yml !package.json !yarn.lock # Application and test source code !packages/components/

я к тому что, в данной реализации в докер образ попадают например packages/components-dev, packages/docs-examples и тд

сам файл можно положить рядом с Dockerfile в tools/e2e/

artembelik · 2026-08-06T14:03:25Z

@@ -17,7 +60,7 @@ export default defineConfig({
    fullyParallel: true,
    forbidOnly: isCI,
    retries: isCI ? 2 : 0,


в идеале конечно отключить retries в 0, чтобы исключить нестабильные тесты на этапе разработки, но лучше это отдельно сделать

artembelik · 2026-08-06T14:08:06Z

+# The two fc-match calls are the assertion, not a demonstration: they are what stops this layer
+# from being "cleaned up" later on the reasonable-sounding grounds that the fonts are bundled. A
+# regression here is otherwise silent until six screenshots disagree for no visible reason.
+RUN apt-get update \


в случае дата-грид помогла установка runs-on: ubuntu-24.04-arm в воркфлоу и platform: linux/arm64 для компоуза, но раз работает давай оставим

lskramarov requested a review from artembelik August 5, 2026 10:12

lskramarov self-assigned this Aug 5, 2026

lskramarov requested a review from NikGurev as a code owner August 5, 2026 10:12

lskramarov added enhancement New feature or request github_actions Pull requests that update GitHub Actions code labels Aug 5, 2026

lskramarov marked this pull request as draft August 5, 2026 10:21

artembelik reviewed Aug 5, 2026

View reviewed changes

lskramarov marked this pull request as ready for review August 5, 2026 12:16

lskramarov force-pushed the chore/DS-5311 branch from c9f88ee to 63f1c91 Compare August 5, 2026 12:33

NikGurev approved these changes Aug 5, 2026

View reviewed changes

lskramarov added 2 commits August 5, 2026 18:32

chore(e2e): run Playwright visual tests in Docker (#DS-5311)

53ccb2b

fix: after review

b49c949

lskramarov force-pushed the chore/DS-5311 branch from faa3740 to b49c949 Compare August 5, 2026 15:42

artembelik requested a lite review from Copilot August 5, 2026 19:06

Copilot started reviewing on behalf of artembelik August 5, 2026 19:06 View session

Copilot AI reviewed Aug 5, 2026

View reviewed changes

artembelik reviewed Aug 6, 2026

View reviewed changes

artembelik approved these changes Aug 6, 2026

View reviewed changes

Uh oh!

Conversation

lskramarov commented Aug 5, 2026

Uh oh!

github-actions Bot commented Aug 5, 2026

🚨 E2E tests failed

Uh oh!

github-actions Bot commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 5, 2026

🚨 E2E tests failed

Uh oh!

lskramarov commented Aug 5, 2026

Uh oh!

github-actions Bot commented Aug 5, 2026

Uh oh!

github-actions Bot commented Aug 5, 2026

Uh oh!

NikGurev commented Aug 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions Bot commented Aug 5, 2026 •

edited

Loading