feat: add Accessibility infra setup skill - #38
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Missing dependency handling, package-manager validation, and inadequate evaluation fixtures must be addressed.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a skill for configuring Playwright and axe-core accessibility testing in Angular applications.
Changes:
- Adds setup workflows, reusable templates, documentation, and a sample scan.
- Adds behavioral and trigger evaluations.
- Adds the skill to the README catalog.
File summaries
| File | Review |
|---|---|
skills/accessibility-infra-setup/SKILL.md |
Must install missing axe dependencies when Playwright exists, use the detected package manager, and avoid claiming automated WCAG compliance. |
skills/accessibility-infra-setup/evals/trigger-eval.json |
Adds trigger-boundary evaluations. |
skills/accessibility-infra-setup/evals/evals.json |
Evaluation cases need representative Angular fixtures to verify behavior. |
skills/accessibility-infra-setup/assets/playwright.config.ts |
Provides the Playwright configuration template. |
skills/accessibility-infra-setup/assets/example.accessibility.spec.ts |
Provides an example accessibility scan. |
skills/accessibility-infra-setup/assets/axe-helpers.ts |
Provides shared axe utilities. |
skills/accessibility-infra-setup/assets/accessibility-README.md |
Should clarify trace availability and that axe scans do not establish WCAG compliance. |
README.md |
Adds the skill to the catalog. |
Review details
Suppressed comments (4)
skills/accessibility-infra-setup/assets/accessibility-README.md:37
- What: Traces are documented as failure artifacts, but the template uses
trace: 'on-first-retry'and disables retries locally. Why: A developer debugging a normal local failure will not find the promised trace intest-results/. How to fix: Document that traces are recorded on the first retry (CI by default), or change the trace mode to retain them on every failure.
- Screenshots / videos / traces on failure → `test-results/`
skills/accessibility-infra-setup/evals/evals.json:28
- What: This existing-Playwright eval supplies no
playwright.config.*fixture. Why: The core merge behavior—including preserving current projects andtestDir—is therefore not exercised, so a destructive overwrite could still satisfy this prose-only eval. How to fix: Add and reference a representative existing Playwright config plus package fixture.
"files": [],
skills/accessibility-infra-setup/evals/evals.json:43
- What: The Cypress regression eval has no Cypress project fixture. Why: Nothing can verify that Cypress config, specs, and dependencies remain byte-for-byte untouched while Playwright files are added. How to fix: Add and reference a minimal Angular+Cypress fixture whose before/after diff can be evaluated.
"files": [],
skills/accessibility-infra-setup/evals/evals.json:69
- What: The do-not-clobber regression has no package or Playwright config fixture containing the scripts/settings it claims to preserve. Why: The eval cannot detect loss of existing scripts or config fields, which is the behavior this case is meant to guard. How to fix: Add and reference fixtures with sentinel scripts and non-default Playwright settings, then assert they survive.
"files": [],
- Files reviewed: 8/8 changed files
- Comments generated: 5
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
I think it would be great if this skill could also set up |
Makes sense. Do we have a concrete example where Pa11y catches something Playwright + axe-core misses? If so, it’d be useful to check whether that’s just down to configuration/page state, and whether we could get the same coverage in Playwright without adding another tool. |
Adds a skill that sets up accessibility (a11y) testing infra in an existing Angular app using Playwright + axe-core, checking for WCAG 2.2 AA compliance. It detects the current test setup, scaffolds a Playwright
accessibilityproject and a shared axe fixture, drops in one dummy example scan, wires up npm scripts, and runs the scan to confirm everything works.The approach follows the a11y testing infra pattern from AbsaOSS/cps-shared-ui, adapted for a plain app rather than a component library.
Scope: just the infrastructure plus one example scan. Writing real per-page tests and fixing violations are left as follow-up work.
Notable behaviors:
npm run test:a11ypasses greenIncludes: the
SKILL.md, asset templates (config, fixture, example spec, docs), eval + trigger-eval cases, and a README catalog entry.Validation: evaluated by running the skill against an internal project, where it set up the a11y infra end to end and the sample scan passed green.
Closes #35