-
Notifications
You must be signed in to change notification settings - Fork 0
feat: add Accessibility infra setup skill #38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
lukasmatta
wants to merge
6
commits into
master
Choose a base branch
from
35-add-skill---accessibility-checks-infrastructure-setup
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
e17cc14
Add Accessibility infra setup skill
lukasmatta 36751f9
Adjust docs
lukasmatta 9bab397
Add eval fixtures
lukasmatta 9f8f28b
Update ts
lukasmatta 33cbf17
Add missing pieces for angular app
lukasmatta 6a96310
Add docs
lukasmatta File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| # Accessibility Infra Setup Skill | ||
|
|
||
| The `accessibility-infra-setup` skill bootstraps automated accessibility (a11y) check infrastructure in an existing Angular application using [Playwright](https://playwright.dev/) + [axe-core](https://github.com/dequelabs/axe-core) (`@axe-core/playwright`). It scaffolds the plumbing to catch automatically detectable **WCAG 2.2 AA** violations and proves it works with one passing example scan. | ||
|
|
||
| It activates when you ask to set up accessibility checks, add a11y testing infrastructure, or wire axe-core into an Angular app. | ||
|
|
||
| --- | ||
|
|
||
| ## What it does | ||
|
|
||
| The skill extends your existing test setup rather than replacing it, then validates the result end to end: | ||
|
|
||
| | Step | Action | | ||
| |------|--------| | ||
| | Detect | Inspects `package.json`, the dev-server script, package manager, and any existing Playwright / Cypress setup | | ||
| | Install | Adds `@playwright/test` + `@axe-core/playwright` and installs the Chromium browser | | ||
| | Scaffold | Creates `playwright.config.ts`, a shared axe fixture, and one dummy example scan | | ||
| | Wire scripts | Adds `test:a11y*` npm scripts to `package.json` | | ||
| | Validate | Runs the dummy scan until it passes green — the definition of done | | ||
| | Document | Adds `docs/accessibility.md` (or `playwright/README.md`) with how to run and where reports land | | ||
|
|
||
| **Scope:** infrastructure plus exactly **one** dummy example scan. Authoring real accessibility tests, fixing violations, and CI wiring are all out of scope. | ||
|
|
||
| --- | ||
|
|
||
| ## Expected outcome | ||
|
|
||
| After the skill runs, the repository contains: | ||
|
|
||
| ``` | ||
| playwright/ | ||
| ├── fixtures/ | ||
| │ └── axe-helpers.ts # Shared axe fixture (WCAG 2.2 AA tags) + assertion helpers | ||
| └── a11y/ | ||
| └── example.accessibility.spec.ts # ONE dummy scan of the "/" route — passes | ||
| playwright.config.ts # `accessibility` project (Desktop Chrome, testMatch /accessibility/) | ||
| docs/accessibility.md # How to run, where reports land | ||
| ``` | ||
|
|
||
| Running `npm run test:a11y` starts the dev server, runs the dummy scan on Desktop Chrome, and passes green. | ||
|
|
||
| --- | ||
|
|
||
| ## How to trigger it | ||
|
|
||
| Ask naturally — the skill fires on intent, not exact wording: | ||
|
|
||
| ``` | ||
| set up accessibility checks | ||
| add a11y testing infrastructure | ||
| add axe-core to this Angular app | ||
| set up WCAG testing | ||
| add accessibility scans with Playwright | ||
| bootstrap a11y infra | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Requirements | ||
|
|
||
| - An existing Angular application with a runnable dev server (`npm run start` on port 4200) | ||
| - Node.js 22+ | ||
| - Network access on first run (Playwright downloads the Chromium browser) | ||
|
|
||
| --- | ||
|
|
||
| ## Coexistence rules | ||
|
|
||
| - **Never touches Cypress.** If the repo uses Cypress, its config, specs, and dependencies are left untouched — Playwright is added alongside it. | ||
| - **Extends, doesn't recreate Playwright.** When a `playwright.config.*` already exists, the skill merges an `accessibility` project into it and reuses any existing axe fixture instead of adding a duplicate. | ||
|
|
||
| --- | ||
|
|
||
| ## Installation | ||
|
|
||
| The skill is installed along with the rest of the toolkit: | ||
|
|
||
| ```bash | ||
| npx skills add https://github.com/AbsaOSS/agentic-toolkit -g | ||
| ``` | ||
|
|
||
| To install only this skill: | ||
|
|
||
| ```bash | ||
| npx skills add https://github.com/AbsaOSS/agentic-toolkit -g --skill accessibility-infra-setup | ||
| ``` | ||
|
|
||
| See [Getting Started](../getting-started.md) for the full install guide. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,174 @@ | ||
| --- | ||
| name: accessibility-infra-setup | ||
| description: > | ||
| Sets up automated accessibility (a11y) check infrastructure in an existing Angular application | ||
| using Playwright + axe-core (@axe-core/playwright) to catch automatically detectable WCAG 2.2 AA violations. | ||
| Detects the existing test setup, installs and wires Playwright with a dedicated `accessibility` project, | ||
| scaffolds a shared axe fixture, adds one dummy example scan, wires npm scripts, documents how to | ||
| run the checks, and validates that the sample passes. Activates on requests like: "set up | ||
| accessibility checks", "add a11y testing infrastructure", "add axe-core to this Angular app", | ||
| "set up WCAG testing", "add accessibility scans with Playwright", "bootstrap a11y infra". | ||
| Scope is infrastructure only plus ONE dummy example test — authoring real accessibility tests is | ||
| a separate concern and out of scope. | ||
| license: Apache-2.0 | ||
| compatibility: > | ||
| Requires an existing Angular application with a runnable dev server (`npm run start` on port 4200) | ||
| and Node.js 22+. Installs Playwright browsers, which needs network access on first run. | ||
| --- | ||
|
|
||
| # accessibility-infra-setup | ||
|
|
||
| Bootstraps Playwright + axe-core accessibility infrastructure in an existing Angular app. Follows the | ||
| [AbsaOSS/cps-shared-ui](https://github.com/AbsaOSS/cps-shared-ui) pattern, generalized for a plain | ||
| application (not a component library). | ||
|
|
||
| **Scope:** infrastructure + exactly one dummy example scan. Do **not** author real accessibility | ||
| tests here — that is a separate follow-up task. | ||
|
|
||
| ## Expected outcome | ||
|
|
||
| After this skill runs, the repository contains: | ||
|
|
||
| ``` | ||
| playwright/ | ||
| ├── fixtures/ | ||
| │ └── axe-helpers.ts # Shared axe fixture (WCAG 2.2 AA tags) + assertion helpers | ||
| └── a11y/ | ||
| └── example.accessibility.spec.ts # ONE dummy scan of the "/" route — passes | ||
| playwright.config.ts # `accessibility` project (Desktop Chrome, testMatch /accessibility/) | ||
| docs/accessibility.md # How to run, where reports land (or playwright/README.md) | ||
| ``` | ||
|
|
||
| - `package.json` gains `@playwright/test` + `@axe-core/playwright` (devDependencies) and | ||
| `test:a11y*` scripts. | ||
| - `npm run test:a11y` starts the dev server, runs the dummy scan on Desktop Chrome, and **passes | ||
| green**. That green run is the definition of done. | ||
|
|
||
| ## Workflow | ||
|
|
||
| Copy this checklist and track progress: | ||
|
|
||
| ``` | ||
| - [ ] Step 1: Detect existing test infrastructure | ||
| - [ ] Step 2: Install dependencies | ||
| - [ ] Step 3: Scaffold config + fixture + example spec | ||
| - [ ] Step 4: Wire npm scripts | ||
| - [ ] Step 5: Validate (run the dummy scan → green) | ||
| - [ ] Step 6: Document | ||
| ``` | ||
|
|
||
| ### Step 1 · Detect existing test infrastructure | ||
|
|
||
| Inspect the repo before writing anything — the goal is to **extend, not clobber**. | ||
|
|
||
| 1. Read `package.json`: note the dev-server script (usually `start` → `ng serve`, port 4200), the | ||
| package manager (from `package-lock.json` / `yarn.lock` / `pnpm-lock.yaml`), and any existing | ||
| `@playwright/test`, `@axe-core/playwright`, Cypress, or Karma entries. | ||
| 2. Check for an existing `playwright.config.*` and for Cypress (`cypress.config.*`, a `cypress/` | ||
| folder, or a `cypress` devDependency). | ||
|
|
||
| Pick exactly one path based on what Playwright infra exists — Cypress never changes the decision: | ||
|
|
||
| - **Playwright already configured** (a `playwright.config.*` exists) → **adjust it in place, do not | ||
| recreate it.** Merge in an `accessibility` project | ||
| `{ name: 'accessibility', testMatch: /accessibility/, use: { ...devices['Desktop Chrome'] } }`, | ||
| add `testIgnore: /accessibility/` to existing functional projects, and ensure `webServer` starts | ||
| the Angular dev server. Keep the user's existing `testDir` and place a11y specs accordingly. Reuse | ||
| any existing axe fixture instead of adding a second one. | ||
| - **No Playwright yet** → fresh setup. Copy `assets/playwright.config.ts` to the repo root and | ||
| scaffold the full structure in Step 3. | ||
|
|
||
| **Cypress is off-limits either way.** If Cypress is present, treat it as read-only: never edit, | ||
| migrate, or delete Cypress config, specs, or dependencies, and never fold a11y scans into Cypress. | ||
| The two runners coexist — Playwright owns accessibility, Cypress keeps whatever it already covers. | ||
| Cypress existing does **not** make this a "fresh" or "adjust" decision; only the presence/absence of | ||
| a `playwright.config.*` does. | ||
|
|
||
| Confirm the dev-server command and port with the user only if they differ from `npm run start` / | ||
| `4200`; otherwise proceed. | ||
|
|
||
| ### Step 2 · Install dependencies | ||
|
|
||
| Use the repo's package manager. For npm: | ||
|
|
||
| ```bash | ||
| npm install -D @playwright/test @axe-core/playwright | ||
| npx playwright install chromium | ||
| ``` | ||
|
|
||
| `npx playwright install chromium` downloads the browser and needs network access. If Playwright is | ||
| already installed, install `@axe-core/playwright` when it is missing, skip reinstalling | ||
| `@playwright/test`, and still ensure the Chromium browser is present. | ||
|
|
||
| ### Step 3 · Scaffold config, fixture, and example spec | ||
|
|
||
| Create these files (templates live in this skill's `assets/`): | ||
|
|
||
| 1. `playwright.config.ts` (root) — from `assets/playwright.config.ts` (fresh setup only; otherwise | ||
| merge as in Step 1). Adjust `baseURL`, `webServer.command`, and port if the app differs. | ||
| 2. `playwright/fixtures/axe-helpers.ts` — copy verbatim from `assets/axe-helpers.ts`. This is the | ||
| single source of the WCAG 2.2 AA tag set; every scan must build from `makeAxeBuilder`. | ||
| 3. `playwright/a11y/example.accessibility.spec.ts` — from `assets/example.accessibility.spec.ts`. | ||
| Verify the import path resolves to the fixture (`../fixtures/axe-helpers`) given where you place | ||
| the spec, and point `page.goto('/')` at a route that renders without auth. If the app's landing | ||
| route requires login, use a known public route instead and note it. | ||
|
|
||
| The `accessibility` substring in the spec filename is what routes it to the accessibility project — | ||
| keep it. | ||
|
|
||
| ### Step 4 · Wire npm scripts | ||
|
|
||
| Add to `package.json` `scripts` (do not clobber existing entries): | ||
|
|
||
| ```jsonc | ||
| "test:a11y": "playwright test --project=accessibility", | ||
| "test:a11y:headed": "playwright test --project=accessibility --headed", | ||
| "test:a11y:report": "playwright show-report" | ||
| ``` | ||
|
|
||
| ### Step 5 · Validate — run the dummy scan | ||
|
|
||
| Run the feedback loop until green: | ||
|
|
||
| ```bash | ||
| npm run test:a11y | ||
| ``` | ||
|
|
||
| 1. If it **passes**, the infrastructure is proven. Done. | ||
| 2. If it **fails on a real WCAG violation** on the chosen route, switch the example to a simpler | ||
| public route (the dummy test must pass to prove the plumbing — fixing app violations is out of | ||
| scope). Note the finding for the user. | ||
| 3. If it fails on **setup** (missing browser, wrong port, dev server timeout, import error), fix the | ||
| config/paths and re-run. Common causes: dev server not on 4200, `webServer.command` wrong, | ||
| Chromium not installed, fixture import path incorrect. | ||
|
|
||
| Do not finish until `npm run test:a11y` exits green. | ||
|
|
||
| ### Step 6 · Document | ||
|
|
||
| Add `assets/accessibility-README.md` to the repo as `docs/accessibility.md` (or | ||
| `playwright/README.md`). Adjust file paths/scripts to match what you created. Add a short | ||
| "Accessibility" note with the run command to the main `README.md` if one exists. | ||
|
|
||
| ## Gotchas | ||
|
|
||
| - **Separate project, Chrome only.** axe evaluates rendered DOM/ARIA, not browser rendering quirks — | ||
| scan once on Desktop Chrome. Do not fan a11y scans across webkit/firefox. | ||
| - **Wait for animations.** Scanning mid-transition produces false-positive color-contrast | ||
| violations. `waitForAnimationsToFinish` (in the fixture) prevents this — call it before every scan. | ||
| - **Never touch Cypress.** If the repo uses Cypress, leave its config, specs, and dependencies | ||
| untouched — add Playwright alongside it rather than migrating or editing anything Cypress owns. | ||
| - **Extend, don't recreate Playwright.** When a `playwright.config.*` already exists, merge the | ||
| `accessibility` project into it and reuse any existing axe fixture — do not generate a second | ||
| config or a duplicate fixture. | ||
| - **Filename routing.** A spec only lands in the accessibility project if `accessibility` is in its | ||
| filename. This is `testMatch: /accessibility/`, not a folder. | ||
| - **git-ignore artifacts.** Ensure `test-results/` and `playwright-report/` are git-ignored. | ||
| - **WCAG tag set lives in one place.** Never inline `withTags(...)` in a spec — always go through | ||
| `makeAxeBuilder`, so the standard stays consistent as scans are added later. | ||
|
|
||
| ## Out of scope | ||
|
|
||
| - Writing real accessibility tests / per-page coverage (separate skill/task). | ||
| - Fixing accessibility violations found in the app. | ||
| - pa11y-ci, Lighthouse, or CI-pipeline wiring — Playwright + axe-core only, run locally. | ||
55 changes: 55 additions & 0 deletions
55
skills/accessibility-infra-setup/assets/accessibility-README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # Accessibility Tests | ||
|
|
||
| Automated accessibility checks for this app, powered by | ||
| [Playwright](https://playwright.dev/) + [axe-core](https://github.com/dequelabs/axe-core) | ||
| (via [`@axe-core/playwright`](https://github.com/dequelabs/axe-core-npm/tree/develop/packages/playwright)). | ||
| Scans assert compliance with **WCAG 2.2 AA** (plus axe best-practice rules); manual testing is still required for full compliance. | ||
|
|
||
| ## Layout | ||
|
|
||
| ``` | ||
| playwright/ | ||
| ├── fixtures/ | ||
| │ └── axe-helpers.ts # Shared axe fixture (WCAG 2.2 AA tag set) + assertion helpers | ||
| └── a11y/ | ||
| └── example.accessibility.spec.ts # Dummy example scan of the home route | ||
| playwright.config.ts # `accessibility` project routes any *accessibility* spec here | ||
| ``` | ||
|
|
||
| Any spec file with `accessibility` in its name runs under the dedicated | ||
| `accessibility` Playwright project (Desktop Chrome only). | ||
|
|
||
| ## Running | ||
|
|
||
| ```bash | ||
| npm run test:a11y # run all accessibility scans (auto-starts the dev server) | ||
| npm run test:a11y:headed # same, with the browser visible | ||
| npm run test:a11y:report # open the last HTML report | ||
| ``` | ||
|
|
||
| Playwright auto-starts `npm run start` (the Angular dev server) and waits for | ||
| `http://localhost:4200`. If a dev server is already running locally, it is reused. | ||
|
|
||
| ## Reports and artifacts | ||
|
|
||
| - HTML report → `playwright-report/` (open with `npm run test:a11y:report`) | ||
| - Full axe results JSON attached to each test → visible in the HTML report | ||
| - Screenshots / videos / traces on failure → `test-results/` | ||
|
|
||
| Both directories are git-ignored. | ||
|
|
||
| ## Adding a scan | ||
|
|
||
| 1. Create a spec whose filename contains `accessibility`, e.g. | ||
| `playwright/a11y/checkout.accessibility.spec.ts`. | ||
| 2. Import the shared fixture: | ||
|
|
||
| ```ts | ||
| import { test, expectNoViolations, waitForAnimationsToFinish } from '../fixtures/axe-helpers'; | ||
| ``` | ||
|
|
||
| 3. Navigate, wait for animations, scan with `makeAxeBuilder()`, assert with | ||
| `expectNoViolations`. Use the example spec as a template. | ||
|
|
||
| Writing real accessibility tests is intentionally out of scope of the setup — | ||
| this directory only ships the infrastructure and one dummy example. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| import { test as base, expect, type Page } from '@playwright/test'; | ||
| import AxeBuilder from '@axe-core/playwright'; | ||
|
|
||
| type Violations = Awaited<ReturnType<AxeBuilder['analyze']>>['violations']; | ||
|
|
||
| type AxeFixture = { | ||
| makeAxeBuilder: () => AxeBuilder; | ||
| }; | ||
|
|
||
| /** | ||
| * Shared axe-core fixture. Every accessibility scan builds its AxeBuilder from | ||
| * here so the WCAG 2.2 AA tag set stays identical across the whole suite. | ||
| */ | ||
| export const test = base.extend<AxeFixture>({ | ||
| makeAxeBuilder: async ({ page }, use) => { | ||
| const makeAxeBuilder = () => | ||
| new AxeBuilder({ page }).withTags([ | ||
| 'wcag2a', | ||
| 'wcag2aa', | ||
| 'wcag21a', | ||
| 'wcag21aa', | ||
| 'wcag22a', | ||
| 'wcag22aa', | ||
| 'best-practice' | ||
| ]); | ||
| await use(makeAxeBuilder); | ||
| } | ||
| }); | ||
|
|
||
| export function formatViolations(violations: Violations): string { | ||
| if (violations.length === 0) return ''; | ||
| return violations | ||
| .map((v) => { | ||
| const nodes = v.nodes | ||
| .map((n) => ` - ${n.html}\n ${n.failureSummary}`) | ||
| .join('\n'); | ||
| return `\n[${v.impact}] ${v.id}: ${v.description}\n Help: ${v.helpUrl}\n${nodes}`; | ||
| }) | ||
| .join('\n'); | ||
| } | ||
|
|
||
| export function expectNoViolations(violations: Violations) { | ||
| expect(violations, formatViolations(violations)).toHaveLength(0); | ||
| } | ||
|
|
||
| /** | ||
| * Wait for all animations/transitions to complete before scanning. Without this, | ||
| * axe may capture intermediate states (e.g. mid-transition background color) and | ||
| * report false-positive color-contrast violations. | ||
| */ | ||
| export async function waitForAnimationsToFinish(page: Page) { | ||
| await page.evaluate(() => | ||
| Promise.all( | ||
| document | ||
| .getAnimations() | ||
| .filter((a) => a.effect?.getTiming().iterations !== Infinity) | ||
| .map((a) => a.finished.catch(() => {})) | ||
| ) | ||
| ); | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.