Skip to content

feat: add Accessibility infra setup skill - #38

Open
lukasmatta wants to merge 5 commits into
masterfrom
35-add-skill---accessibility-checks-infrastructure-setup
Open

feat: add Accessibility infra setup skill#38
lukasmatta wants to merge 5 commits into
masterfrom
35-add-skill---accessibility-checks-infrastructure-setup

Conversation

@lukasmatta

Copy link
Copy Markdown
Collaborator

Adds a skill that sets up accessibility (a11y) testing infra in an existing Angular app using Playwright + axe-core, checking for WCAG 2.2 AA compliance. It detects the current test setup, scaffolds a Playwright accessibility project and a shared axe fixture, drops in one dummy example scan, wires up npm scripts, and runs the scan to confirm everything works.

The approach follows the a11y testing infra pattern from AbsaOSS/cps-shared-ui, adapted for a plain app rather than a component library.

Scope: just the infrastructure plus one example scan. Writing real per-page tests and fixing violations are left as follow-up work.

Notable behaviors:

  • Extends an existing Playwright config instead of overwriting it
  • Leaves Cypress completely untouched if it's present
  • Done when npm run test:a11y passes green

Includes: the SKILL.md, asset templates (config, fixture, example spec, docs), eval + trigger-eval cases, and a README catalog entry.

Validation: evaluated by running the skill against an internal project, where it set up the a11y infra end to end and the sample scan passed green.

Closes #35

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Missing dependency handling, package-manager validation, and inadequate evaluation fixtures must be addressed.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a skill for configuring Playwright and axe-core accessibility testing in Angular applications.

Changes:

  • Adds setup workflows, reusable templates, documentation, and a sample scan.
  • Adds behavioral and trigger evaluations.
  • Adds the skill to the README catalog.
File summaries
File Review
skills/accessibility-infra-setup/SKILL.md Must install missing axe dependencies when Playwright exists, use the detected package manager, and avoid claiming automated WCAG compliance.
skills/accessibility-infra-setup/evals/trigger-eval.json Adds trigger-boundary evaluations.
skills/accessibility-infra-setup/evals/evals.json Evaluation cases need representative Angular fixtures to verify behavior.
skills/accessibility-infra-setup/assets/playwright.config.ts Provides the Playwright configuration template.
skills/accessibility-infra-setup/assets/example.accessibility.spec.ts Provides an example accessibility scan.
skills/accessibility-infra-setup/assets/axe-helpers.ts Provides shared axe utilities.
skills/accessibility-infra-setup/assets/accessibility-README.md Should clarify trace availability and that axe scans do not establish WCAG compliance.
README.md Adds the skill to the catalog.
Review details

Suppressed comments (4)

skills/accessibility-infra-setup/assets/accessibility-README.md:37

  • What: Traces are documented as failure artifacts, but the template uses trace: 'on-first-retry' and disables retries locally. Why: A developer debugging a normal local failure will not find the promised trace in test-results/. How to fix: Document that traces are recorded on the first retry (CI by default), or change the trace mode to retain them on every failure.
- Screenshots / videos / traces on failure → `test-results/`

skills/accessibility-infra-setup/evals/evals.json:28

  • What: This existing-Playwright eval supplies no playwright.config.* fixture. Why: The core merge behavior—including preserving current projects and testDir—is therefore not exercised, so a destructive overwrite could still satisfy this prose-only eval. How to fix: Add and reference a representative existing Playwright config plus package fixture.
      "files": [],

skills/accessibility-infra-setup/evals/evals.json:43

  • What: The Cypress regression eval has no Cypress project fixture. Why: Nothing can verify that Cypress config, specs, and dependencies remain byte-for-byte untouched while Playwright files are added. How to fix: Add and reference a minimal Angular+Cypress fixture whose before/after diff can be evaluated.
      "files": [],

skills/accessibility-infra-setup/evals/evals.json:69

  • What: The do-not-clobber regression has no package or Playwright config fixture containing the scripts/settings it claims to preserve. Why: The eval cannot detect loss of existing scripts or config fields, which is the behavior this case is meant to guard. How to fix: Add and reference fixtures with sentinel scripts and non-default Playwright settings, then assert they survive.
      "files": [],
  • Files reviewed: 8/8 changed files
  • Comments generated: 5
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread skills/accessibility-infra-setup/SKILL.md Outdated
Comment thread skills/accessibility-infra-setup/SKILL.md
Comment thread skills/accessibility-infra-setup/evals/evals.json Outdated
Comment thread skills/accessibility-infra-setup/SKILL.md Outdated
Comment thread skills/accessibility-infra-setup/assets/accessibility-README.md Outdated
@lukasmatta
lukasmatta marked this pull request as ready for review September 3, 2026 14:31
@fateeand

fateeand commented Sep 3, 2026

Copy link
Copy Markdown

I think it would be great if this skill could also set up pa11y-ci, as it sometimes identifies accessibility issues that Playwright misses, and vice versa. Even though both tools use axe-core under the hood, they can surface different findings and therefore complement each other well. We already use this setup in the components library.

@lukasmatta

Copy link
Copy Markdown
Collaborator Author

I think it would be great if this skill could also set up pa11y-ci, as it sometimes identifies accessibility issues that Playwright misses, and vice versa. Even though both tools use axe-core under the hood, they can surface different findings and therefore complement each other well. We already use this setup in the components library.

Makes sense. Do we have a concrete example where Pa11y catches something Playwright + axe-core misses? If so, it’d be useful to check whether that’s just down to configuration/page state, and whether we could get the same coverage in Playwright without adding another tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add skill - Accessibility checks infrastructure setup

3 participants