Repository navigation
feat: remote verification orchestration and classified acceptance evidence - #292
Merged
Merged
Conversation
Finishes the two items #287 left open after the matrix landed. raco rivet verify-remote dispatches the per-platform acceptance legs of one commit on GitHub Actions, waits for the dispatched runs, downloads the archived acceptance --json reports, and folds them into the shared verification matrix. It drives the gh CLI and nothing else: workflow names and artifact names stay the product's own conventions, with verify.yml and acceptance-report-<platform> as the defaults the product CI guide now documents. Runs are identified against a pre-dispatch baseline so a concurrent session's runs cannot be mistaken for this verification, and a red run still hands over its archived evidence because the watch exit code only sets the conclusion, never skips the download. The command exits green only when the matrix passes and every requested platform delivered a report. Acceptance evidence entries are now classified — accessibility-snapshot, screenshot, interaction-trace, log, or generic — with plain string paths staying legal as generic. The report pins each declared artifact's kind, presence, byte size, and SHA-256, so an archived report audits the artifacts it vouches for even when the files are fetched from another platform's artifact store. Closes #287.
raco test runs with the test file's directory as the working directory, so the default output directory sprouted rivet/tests/.rivet during CI and the source-package smoke rightly rejected it. Every scenario now pins an isolated output directory.
The output directory was created before dispatch, so a failed dispatch or a discovery timeout left an empty .rivet/verify-remote behind in the caller's working directory — which the source-package smoke rightly rejects. Create it after the dispatched runs are confirmed.
turinglambdaai
force-pushed
the
feature/verify-remote-orchestration
branch
from
October 19, 2026 17:56
7f1b9db to
545c224
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #287 (the two items the matrix slice left open).
1.
raco rivet verify-remote— orchestrate the remote legs from one editing hostThe command dispatches each platform's acceptance leg on GitHub Actions, waits for the dispatched runs, downloads the archived
acceptance --jsonreports, and folds them into the sharedmatrixaggregation — so an agent editing on one OS receives the other platforms' evidence instead of assuming correctness from local results alone.Design points:
ghCLI and nothing else. Defaults (verify.yml,acceptance-report-<platform>) encode the convention the product CI guide documents;--workflow/--artifact-prefix/--platform/--timeout-seconds/--output-directoryadapt it.--jsonadds per-run conclusions and report paths to the matrix document.The product CI guide (
docs/ci.md,docs/ci.zh-CN.md) now archives every leg's report asacceptance-report-<platform>withif: always()— the exact convention the command consumes.2. Classified evidence as first-class report fields
Manifest evidence entries accept a classified form alongside legacy strings:
Kinds:
accessibility-snapshot,screenshot,interaction-trace,log,generic(bare paths stay legal asgeneric). Accessibility snapshots are documented as the primary cross-platform record; screenshots are supplemental because native presentations are expected to differ.The
--jsonreport now pins every declared entry's kind, presence, byte size, and SHA-256 — an archived report audits the artifacts it vouches for even when the files live in another platform's artifact store.missing-evidencestays for backward-compatible tooling.Validation
raco test rivet/tests/: 10110 tests pass locally (verify-remote +16 acceptance cases + all prior suites).raco rivet verify-remote --helpdispatches through the lazy command loader; usage lists the command.