Skip to content

feat: add auto-fixture for Istanbul E2E coverage collection#95

Open
gustavolira wants to merge 1 commit intoredhat-developer:mainfrom
gustavolira:feat/coverage-auto-fixture
Open

feat: add auto-fixture for Istanbul E2E coverage collection#95
gustavolira wants to merge 1 commit intoredhat-developer:mainfrom
gustavolira:feat/coverage-auto-fixture

Conversation

@gustavolira
Copy link
Copy Markdown
Member

Summary

  • Adds a _coverageCollector auto-fixture to the shared test object that automatically collects window.__coverage__ from the browser after each test
  • Guarded by E2E_COLLECT_COVERAGE=1 — zero overhead when disabled (no page.evaluate, no fs operations)
  • Writes Istanbul coverage JSON files to coverage/istanbul/ (configurable via COVERAGE_OUTPUT_DIR)

Why

The rhdh-plugin-export-overlays repo builds Istanbul-instrumented dynamic plugins and runs E2E tests against them. The instrumented plugins populate window.__coverage__ in the browser, but nothing collects it today.

Since all 18 workspace test files import test from @red-hat-developer-hub/e2e-test-utils/test, adding an auto-fixture here gives every test automatic coverage collection without modifying any test file.

How it works

1. Instrumented plugin loaded in browser → window.__coverage__ populated
2. Test runs and exercises plugin UI
3. _coverageCollector auto-fixture teardown fires (after test, before page closes)
4. page.evaluate() reads __coverage__ from browser
5. JSON written to coverage/istanbul/<testId>-<timestamp>.json
6. Overlay repo's coverage-reporter merges all JSONs → lcov.info
7. upload-coverage.sh sends lcov to Codecov with --slug --sha --flag

Design decisions

  • auto: true — runs for every test without test file changes
  • scope: "test" — collects per-test, not per-worker
  • await use() first — teardown runs AFTER the test but BEFORE page closes (Playwright tears down in reverse dependency order)
  • try/catch with empty catch — best-effort; if the page crashed or plugin wasn't instrumented, silently skip
  • _ prefix — Playwright convention for internal auto-fixtures
  • No new dependencies — uses only node:fs and existing path import

Related PRs

Test plan

  • yarn build succeeds
  • yarn typecheck passes
  • With E2E_COLLECT_COVERAGE=1 and an instrumented plugin: JSON files appear in coverage/istanbul/
  • Without E2E_COLLECT_COVERAGE: no JSON files, no performance impact
  • When page crashes mid-test: no error thrown, test result unaffected

🤖 Generated with Claude Code

Add a `_coverageCollector` auto-fixture that collects
`window.__coverage__` from the browser after each test and writes
it to a JSON file. This enables E2E coverage collection for
instrumented dynamic plugins without modifying any test files.

Guarded by E2E_COLLECT_COVERAGE=1 — zero overhead when disabled.
Coverage files are written to coverage/istanbul/ (or
COVERAGE_OUTPUT_DIR) and consumed by the overlay repo's coverage
reporter for merging into lcov and uploading to Codecov.

Part of the E2E coverage pipeline: RHIDP-13411

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant