Skip to content

feat: aggregate per-platform acceptance reports into a verification matrix - #289

Merged
turinglambdaai merged 1 commit into
mainfrom
feature/acceptance-matrix
Oct 19, 2026
Merged

turinglambdaai merged 1 commit into
mainfrom
feature/acceptance-matrix

Conversation

@turinglambdaai

Copy link
Copy Markdown
Owner

Summary

Takes the first implementation slice of #287: the acceptance orchestrator
already emits a machine-readable report per platform — what was missing is
where those reports become one verdict.

raco rivet matrix <report.json>... [--json]

  • A required kind/platform cell is proven only by a passing suite of that
    kind in a report from that platform; ran-but-failed evidence fails the
    cell; an unreported platform is missing, never silently green.
  • All reports must describe the same project, so one product's green leg
    cannot vouch for another.
  • Diagnostic --suite runs are accepted and flagged as partial; merge gates
    feed complete reports.
  • Exit code is green only when every required cell is proven. Works without a
    project checkout — the aggregation host is anywhere.

Contracts and classification

Deferred (still open on #287)

Remote job triggering/orchestration and rich per-suite artifacts
(accessibility snapshots, screenshots) — the matrix consumes whatever the
legs archive today, so those can land incrementally behind the same command.

Verification

  • raco test rivet/tests/ — 10065 passed (21 new in
    acceptance-matrix.rkt: cell pass/fail/missing, partial runs, malformed
    and foreign-project rejection, exit codes, CLI smoke).
  • End-to-end raco rivet matrix on synthetic reports; usage line via
    raco rivet help.

Closes part of #287; the issue stays open for the orchestration roadmap.

…atrix

Issue #287 asks that cross-platform parity stop being implicit. The
acceptance orchestrator already emits a machine-readable report per
platform; what was missing is the place where those reports become one
verdict an agent can act on from any editing OS.

raco rivet matrix folds the reports for one commit into that verdict: a
required kind/platform cell is proven only by a passing suite of that kind
on that platform's report, failed evidence fails the cell, and an
unreported platform is missing - never silently green. All reports must
describe one project, so a green leg cannot vouch for another product.

docs/product-acceptance.md adds the guaranteed / conditional /
platform-specific capability classification and the matrix contract;
docs/agent-native.md and AGENTS.md state that the local operating system
bounds only where checks can run, never the required verification scope.
@turinglambdaai
turinglambdaai force-pushed the feature/acceptance-matrix branch from 76c41c2 to ef4e26b Compare October 19, 2026 04:16
@turinglambdaai
turinglambdaai merged commit ddaf976 into main Oct 19, 2026
23 checks passed
@turinglambdaai
turinglambdaai deleted the feature/acceptance-matrix branch October 19, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant