Ship adapter-neutral browser, accessibility, visual, and performance testkits

[forkwright]

Finding

Typikon has Zola-specific browser checks but no shared assurance packages for its supported rendering contract and direct consumers. Existing checks can inspect the wrong subject and do not produce visual, manual-accessibility, or repeatable performance evidence.

Evidence

Browser tooling lives under ci/ and is not exposed as testkit/{browser,accessibility,visual,performance}. It is pinned, not installed through mutable package requests: ci/tool-lock.toml fixes pa11y-ci at 4.1.1 and @playwright/test at 1.61.1 with integrity values, every gate surface renders its install line from that lock, and ci/check-tool-lock.py fails closed on any install line reintroducing an unbounded version (#194).

No testkit/ directory exists; there is no axe wiring anywhere in ci/ or ci/smoke/ (ci/check-interactive-contrast.py + ci/contrast.py are a narrower programmatic contrast checker, not accessibility scanning); no Lighthouse or other performance tooling exists; there is no NVDA/VoiceOver/TalkBack receipt scaffolding; and ci/smoke/ holds three Playwright specs (two contrast-focused, one general semantic smoke) plus a shared contrast-math helper module, no visual-diff harness. #33 and #49-#52 cover immediate gate defects; #64 covers one contrast defect; none supplies the reusable testkit contract.

The current direct consumers are Ardent Leatherworks a38adcc4c7e5a4fafe40da7938eeb137cd0f8e8c and Ardent Tools 889dd8165eb1c3a86c763590f8e154c374a2948d, both using the Zola adapter. A2Z is not yet a Typikon consumer and joins this assurance matrix only after its documented migration or another D-065-approved adoption path is implemented.

Why this matters

A green adapter build is not evidence that real routes are complete, semantically correct, accessible, visually intentional, or performant. Capture-only screenshots and one-off Lighthouse runs can certify the instrument rather than the candidate.

Desired correction

Publish framework-neutral browser, accessibility, visual, and performance testkits. Derive a non-empty route corpus and build identity; require semantic assertions; combine axe automation with typed manual NVDA/Firefox, VoiceOver/macOS+iOS Safari, TalkBack/Android, and keyboard receipts; make visual diffs assertion-bearing with explicit adjudication; and aggregate five cold Lighthouse runs.

Done when every supported adapter and every direct consumer runs the same contract; a prospective consumer joins only after it has an actual Typikon dependency; median Lighthouse performance is at least 95 with no run below 90, LCP <=2.5s, CLS <=0.05, and TBT <=150ms; typed CSS/JS budgets are enforced; missing tools/routes/evidence fail closed; and receipts bind the exact subject revision and contract digest.

Program: ardent-web-triad. Refs #33 #49 #50 #51 #52 #64 #66.

[forkwright]

Scoping concern: the numeric budgets are being fixed before the content they will measure exists

Raised while sequencing the v2 program. The testkits should be built; the specific thresholds should not be treated as settled yet.

Ground state, verified today

Every sub-claim in the finding holds — this is genuinely greenfield, not hardening:

• No testkit/ directory.
• No axe anywhere: grep -rl axe across ci/ and ci/smoke/ returns nothing. What exists is ci/check-interactive-contrast.py + ci/contrast.py, a narrow programmatic contrast checker — a materially smaller and different thing than automated accessibility scanning.
• No performance tooling at all: grep -rl -i lighthouse returns nothing.
• No NVDA/VoiceOver/TalkBack receipt scaffolding.
• ci/smoke/ holds four Playwright specs, all contrast-focused; no visual-diff harness.

The concern

The acceptance clause fixes hard numbers: 5-run median Lighthouse ≥95 with no run below 90, LCP ≤ 2.5s, CLS ≤ 0.05, TBT ≤ 150ms.

Those will be measured against pages whose largest contentful element does not exist yet. Ardent Site's own backlog (B-069) is still carrying shot-list photography placeholders, and a craft-leatherwork site's real hero and product imagery is precisely the dominant LCP and CLS risk on the page.

So fixing the thresholds now leads to one of two outcomes, both bad: the budget is quietly loosened once real photography lands, which defeats the purpose of a hard gate; or the gate blocks the photography work B-069 exists to do.

What to do instead

Build the mechanism now — the derived route corpus, build identity, axe wiring, visual-diff harness, and the Lighthouse runner with its 5-run median logic. Have it report the numbers from the first day and record them.

Validate the specific thresholds against real post-photography pages before making them non-negotiable. A budget calibrated on placeholder content is a measurement of the placeholder.

Dependencies

SOFT on #66 — the "every supported adapter" framing implies an adapter concept, but these testkits can be built today against the single Zola build path and re-pointed later. SOFT on #67 — nicer once build emits typed build identity, not blocked without it. This is the least dependency-constrained item in the chain after #55, which is worth knowing when looking for parallel work.

[forkwright]

One of this issue's two Evidence claims is now false, and it matters because these five v2 issues
drive B-068's sequencing — planning built on stale evidence orders work wrongly.

"browser tooling lives under ci/, is installed through mutable package requests, and is not
exposed as testkit/{browser,accessibility,visual,performance}"

The middle clause was true at faaa36fe. It is not true at main:

# ci/tool-lock.toml
name = "pa11y-ci"          version = "4.1.1"    integrity = "registry-version-pin"
name = "@playwright/test"  version = "1.61.1"   integrity = "registry-version-pin"

Every gate surface renders its install line from that lock (#194), typikon's own gate no longer runs
@latest — it did until tonight — and ci/check-tool-lock.py fails on any install line that
reintroduces an unbounded version. So "mutable package requests" is closed, by a different issue,
and this one should not still be counted as owning it.

What remains true and is the actual scope: the tooling is not exposed as a reusable
testkit/{browser,accessibility,visual,performance} contract, and there is no visual or
manual-accessibility evidence. That is the whole of what is left here, and it is a smaller issue than
the Evidence implies.

Worth re-reading the other four the same way before sequencing them. Two of #68's five named absences
have also landed — see my note there.

[forkwright]

Body edited to match main at d1d9b5e, per the 2026-08-21 comment above. Re-verified independently before editing: ci/tool-lock.toml pins pa11y-ci 4.1.1 / @playwright/test 1.61.1; testkit/, axe, Lighthouse, and NVDA/VoiceOver/TalkBack scaffolding are all still absent (fresh greps against main); ci/smoke/ holds interactive-state-contrast.spec.ts + interactive-state-contrast-selftest.spec.ts (contrast) + shared.spec.ts (general semantic smoke) plus the interactive-contrast.ts helper module — three specs, not four, one of them not contrast-focused. Evidence now states the pin as fact rather than the absence it used to name; Desired correction and Done-when are unchanged.