feat: remote verification orchestration and classified acceptance evidence

AGENTS.md

@@ -80,7 +80,9 @@ required acceptance suites pass on Windows, macOS, and Linux. The local
 operating system determines only which checks can run locally; it does not
 reduce the required verification scope. Archive every platform leg's
 `acceptance --json` report and prove the union with
-`raco rivet matrix <reports...> --json` (issue #287). Classify desktop
+`raco rivet matrix <reports...> --json`; `raco rivet verify-remote`
+dispatches those legs and pulls their archived reports into the matrix from
+this host when the platforms are not local (issue #287). Classify desktop
 capabilities as guaranteed, conditional, or platform-specific as documented
 in `docs/product-acceptance.md`.
 

CHANGELOG.md

@@ -2,6 +2,19 @@
 
 ## Unreleased
 
+- Make verification evidence and orchestration first-class for distributed
+  agents (issue #287). `raco rivet verify-remote` dispatches the
+  per-platform acceptance legs of one commit on GitHub Actions, waits for the
+  runs, downloads the archived `acceptance --json` reports, and folds them
+  into the verification matrix, so an agent edits on one OS and is judged by
+  the others. Acceptance evidence entries are now classified —
+  `accessibility-snapshot`, `screenshot`, `interaction-trace`, `log`, or
+  `generic` — and the report pins each declared artifact's kind, presence,
+  byte size, and SHA-256, while plain string evidence paths stay legal as
+  `generic`. The product CI guide archives every leg's report as
+  `acceptance-report-<platform>` with `if: always()`, which is exactly the
+  convention the new command consumes.
+
 - Add the crash-safe process-external Linux AppImage handoff (issue #290).
   `AppImageHandoff::LaunchReplacement` in the linked `rivet::platform` target
   writes a generated first-party shell script — the counterpart of the

docs/ci.md

@@ -60,7 +60,13 @@ jobs:
       - run: raco pkg install --auto --no-docs 'https://github.com/turinglambdaai/rivet.git#v0.12.0'
       - run: raco rivet doctor
       - run: raco rivet schema check rivet-schema.json
-      - run: raco rivet acceptance --json
+      - run: raco rivet acceptance --json > acceptance-report.json
+        shell: bash
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: acceptance-report-windows
+          path: acceptance-report.json
       - run: raco rivet build
 
   macos-host:
@@ -80,7 +86,13 @@ jobs:
       - run: raco pkg install --auto --no-docs 'https://github.com/turinglambdaai/rivet.git#v0.12.0'
       - run: raco rivet doctor
       - run: raco rivet schema check rivet-schema.json
-      - run: raco rivet acceptance --json
+      - run: raco rivet acceptance --json > acceptance-report.json
+        shell: bash
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: acceptance-report-macos
+          path: acceptance-report.json
       - run: raco rivet build
 
   linux-host:
@@ -97,10 +109,23 @@ jobs:
         run: raco pkg install --auto --no-docs 'https://github.com/turinglambdaai/rivet.git#v0.12.0'
       - run: raco rivet doctor
       - run: raco rivet schema check rivet-schema.json
-      - run: raco rivet acceptance --json
+      - run: raco rivet acceptance --json > acceptance-report.json
+        shell: bash
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: acceptance-report-linux
+          path: acceptance-report.json
       - run: raco rivet build
 ```
 
+Each job writes its `acceptance --json` report to a file and archives it as
+`acceptance-report-<platform>` with `if: always()`, so a red leg still hands
+over its evidence. Those names are exactly what
+[`raco rivet verify-remote`](product-acceptance.md#orchestrate-remote-verification-from-one-host)
+expects: from any editing host it dispatches the legs, waits for the runs,
+downloads the archived reports, and folds them into the verification matrix.
+
 The Linux order is load-bearing: install the Rivet package **after**
 `setup-embed-racket`. The action's minimal Racket build intentionally ships
 without packages; installing Rivet afterward resolves `cext-lib`, `crypto-lib`,

docs/ci.zh-CN.md

@@ -55,7 +55,13 @@ jobs:
       - run: raco pkg install --auto --no-docs 'https://github.com/turinglambdaai/rivet.git#v0.12.0'
       - run: raco rivet doctor
       - run: raco rivet schema check rivet-schema.json
-      - run: raco rivet acceptance --json
+      - run: raco rivet acceptance --json > acceptance-report.json
+        shell: bash
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: acceptance-report-windows
+          path: acceptance-report.json
       - run: raco rivet build
 
   macos-host:
@@ -75,7 +81,13 @@ jobs:
       - run: raco pkg install --auto --no-docs 'https://github.com/turinglambdaai/rivet.git#v0.12.0'
       - run: raco rivet doctor
       - run: raco rivet schema check rivet-schema.json
-      - run: raco rivet acceptance --json
+      - run: raco rivet acceptance --json > acceptance-report.json
+        shell: bash
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: acceptance-report-macos
+          path: acceptance-report.json
       - run: raco rivet build
 
   linux-host:
@@ -92,10 +104,23 @@ jobs:
         run: raco pkg install --auto --no-docs 'https://github.com/turinglambdaai/rivet.git#v0.12.0'
       - run: raco rivet doctor
       - run: raco rivet schema check rivet-schema.json
-      - run: raco rivet acceptance --json
+      - run: raco rivet acceptance --json > acceptance-report.json
+        shell: bash
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: acceptance-report-linux
+          path: acceptance-report.json
       - run: raco rivet build
 ```
 
+每个 job 都把 `acceptance --json` 报告写入文件，并以
+`acceptance-report-<platform>` 为名、`if: always()` 归档——失败的腿也要交出
+证据。这些名字正是
+[`raco rivet verify-remote`](product-acceptance.md#orchestrate-remote-verification-from-one-host)
+的默认约定：在任一编辑主机上触发各平台腿、等待运行、下载归档报告并折叠成
+验证矩阵。
+
 Linux 中的顺序是必要条件：必须在 `setup-embed-racket` **之后**安装 Rivet。
 该 action 准备的是刻意不附带任何包的 minimal Racket；此时再安装 Rivet，
 `cext-lib`、`crypto-lib`、`keel` 等完整依赖才会进入嵌入式 runtime 实际读取的

docs/product-acceptance.md

@@ -33,8 +33,12 @@ Commit the manifest at the project root:
              (platforms . (windows))
              (command . ("pwsh" "-File" "scripts/e2e/windows.ps1"))
              (timeout-seconds . 900)
-             (evidence . (".rivet/acceptance/windows/trace.json"
-                          ".rivet/acceptance/windows/accessibility.json"))))))
+             (evidence
+              . (#hasheq((path . ".rivet/acceptance/windows/accessibility.json")
+                         (kind . accessibility-snapshot))
+                 #hasheq((path . ".rivet/acceptance/windows/screen.png")
+                         (kind . screenshot))
+                 ".rivet/acceptance/windows/trace.json"))))))
 ```
 
 The supported suite kinds are:
@@ -56,6 +60,16 @@ suite starts and then requires the suite to recreate it, so a stale artifact
 from an earlier green run cannot satisfy the current run. `native-ui`,
 `package`, and `release` suites must declare at least one evidence file.
 
+Evidence entries are classified so remote consumers know what a green suite
+actually produced. A bare path string means the `generic` kind; a hash declares
+`path` and one `kind` of `accessibility-snapshot`, `screenshot`,
+`interaction-trace`, `log`, or `generic`. Accessibility snapshots are the
+primary cross-platform record — semantic UI state that is comparable across
+platforms — while screenshots are supplemental because native presentations
+are expected to differ. The `--json` report pins every declared entry's kind,
+presence, byte size, and SHA-256, so an archived report audits the artifacts
+it vouches for.
+
 `requirements` is the reviewable policy. For every kind/platform pair there
 must be at least one matching suite or the manifest is rejected before a child
 process starts. Product owners should protect changes to requirements, CI,
@@ -148,6 +162,35 @@ only when every required kind/platform cell is proven.
 Feed the reports for the commit under verification; the matrix aggregates
 what it is given and does not heuristically prefer fresher runs.
 
+## Orchestrate remote verification from one host
+
+`raco rivet verify-remote` closes the loop for an agent editing away from the
+platform legs: it dispatches each platform's acceptance leg on GitHub Actions,
+waits for the runs, downloads the archived `acceptance --json` reports, and
+folds them into the matrix:
+
+```bash
+raco rivet verify-remote --repo OWNER/REPO --ref <branch-or-sha>
+raco rivet verify-remote --repo OWNER/REPO --ref <sha> --platform linux
+raco rivet verify-remote --repo OWNER/REPO --ref <sha> --json
+```
+
+The command drives the `gh` CLI and nothing else. The defaults encode one
+convention — workflow `verify.yml`, artifacts named
+`acceptance-report-<platform>` — and `--workflow`/`--artifact-prefix` adapt it
+to a product's own naming. The workflow contract is: a `workflow_dispatch`
+trigger with a `platform` input (`linux`, `windows`, or `macos`), one
+`raco rivet acceptance --json` per leg, and the report uploaded with
+`if: always()` so a red leg still hands over its evidence. Re-running only the
+affected platform is just `--platform <name>`; the final verdict still needs
+every required platform's report, because missing platforms are never silently
+green.
+
+The command exits green only when the matrix passes and every requested
+platform delivered a report. Run collections land under
+`.rivet/verify-remote/` (adjust with `--output-directory`), and `--json` adds
+the per-run conclusions and report paths to the matrix document.
+
 ## What counts as complete
 
 RackUnit is the normal owner of shared Racket behavior. A product that claims

rivet-cli/acceptance.rkt

@@ -13,19 +13,29 @@
          racket/port
          racket/string
          racket/system
+         "../rivet/distribution/crypto.rkt"
          "project.rkt")
 
 (provide default-acceptance-manifest-name
          acceptance-kinds
+         evidence-kinds
          supported-platforms
          (struct-out acceptance-suite)
          load-acceptance-manifest
          acceptance-report
+         evidence-entries
          run-acceptance-command)
 
 (define default-acceptance-manifest-name "rivet-acceptance.rktd")
 (define acceptance-kinds
   '(unit contract bridge native-build native-ui package release))
+;; Normalized evidence classes (issue #287): accessibility snapshots are the
+;; primary cross-platform acceptance evidence, screenshots are supplemental
+;; because native UIs are expected to look different, and traces plus logs
+;; carry the interaction and failure narrative. Plain string paths in a
+;; manifest stay legal and mean 'generic.
+(define evidence-kinds
+  '(accessibility-snapshot screenshot interaction-trace log generic))
 (define supported-platforms '(windows macos linux))
 (define maximum-captured-output-bytes (* 64 1024))
 
@@ -95,10 +105,64 @@
               (string-ci=? (path->string (first parts)) ".rivet")
               (string-ci=? (path->string (second parts)) "acceptance")))))
 
-(define (valid-evidence? value)
-  (and (list? value)
-       (andmap acceptance-evidence-path-string? value)
-       (= (length value) (length (remove-duplicates value string=?)))))
+;; Manifest evidence entries are either a bare path string (legacy shape,
+;; normalized to the 'generic kind) or a hash with exactly `path` and `kind`.
+;; Internally each entry is a (path . kind) pair.
+(define (parse-evidence-entry path suite-id raw)
+  (cond
+    [(string? raw)
+     (unless (acceptance-evidence-path-string? raw)
+       (manifest-error path
+                       "invalid suite evidence path"
+                       "suite" suite-id
+                       "expected"
+                       "a unique file path below .rivet/acceptance/"
+                       "value" raw))
+     (cons raw 'generic)]
+    [(hash? raw)
+     (define path-value
+       (hash-ref raw 'path
+                 (lambda ()
+                   (manifest-error path
+                                   "invalid suite evidence entry"
+                                   "suite" suite-id
+                                   "expected" "a hash with path and kind"
+                                   "value" raw))))
+     (unless (acceptance-evidence-path-string? path-value)
+       (manifest-error path
+                       "invalid suite evidence path"
+                       "suite" suite-id
+                       "expected"
+                       "a unique file path below .rivet/acceptance/"
+                       "value" path-value))
+     (define kind-value
+       (hash-ref raw 'kind
+                 (lambda ()
+                   (manifest-error path
+                                   "invalid suite evidence entry"
+                                   "suite" suite-id
+                                   "expected" "a hash with path and kind"
+                                   "value" raw))))
+     (unless (memq kind-value evidence-kinds)
+       (manifest-error path
+                       "invalid suite evidence kind"
+                       "suite" suite-id
+                       "expected" evidence-kinds
+                       "value" kind-value))
+     (unless (= (length (hash-keys raw)) 2)
+       (manifest-error path
+                       "invalid suite evidence entry"
+                       "suite" suite-id
+                       "expected" "a hash with exactly path and kind"
+                       "value" raw))
+     (cons path-value kind-value)]
+    [else
+     (manifest-error path
+                     "invalid suite evidence entry"
+                     "suite" suite-id
+                     "expected"
+                     "a path string or a hash with path and kind"
+                     "value" raw)]))
 
 (define (parse-suite path raw)
   (unless (hash? raw)
@@ -134,14 +198,17 @@
                     "suite" id
                     "expected" "integer from 1 through 7200"
                     "value" timeout-seconds))
-  (define evidence (hash-ref raw 'evidence '()))
-  (unless (valid-evidence? evidence)
+  (define evidence-raw (hash-ref raw 'evidence '()))
+  (define evidence
+    (map (lambda (entry) (parse-evidence-entry path id entry)) evidence-raw))
+  (unless (= (length evidence)
+             (length (remove-duplicates evidence equal?)))
     (manifest-error path
                     "invalid suite evidence"
                     "suite" id
                     "expected"
                     "unique file paths below .rivet/acceptance/"
-                    "value" evidence))
+                    "value" evidence-raw))
   (when (and (memq kind '(native-ui package release))
              (null? evidence))
     (manifest-error path
@@ -280,7 +347,8 @@
 (define (prepare-evidence! project suite)
   ;; Evidence belongs under Rivet's generated root, so removing stale files is
   ;; both safe and necessary: an earlier green run must not satisfy this run.
-  (for ([relative (in-list (acceptance-suite-evidence suite))])
+  (for ([entry (in-list (acceptance-suite-evidence suite))])
+    (define relative (car entry))
     (define path (project-path project relative))
     (for/fold ([parent (rivet-project-root project)])
               ([part (in-list
@@ -302,9 +370,23 @@
       (delete-file path))))
 
 (define (evidence-missing project suite)
-  (for/list ([relative (in-list (acceptance-suite-evidence suite))]
-             #:unless (file-exists? (project-path project relative)))
-    relative))
+  (for/list ([entry (in-list (acceptance-suite-evidence suite))]
+             #:unless (file-exists? (project-path project (car entry))))
+    (car entry)))
+
+;; First-class evidence records (issue #287): the report pins each declared
+;; artifact's kind, presence, size, and digest so remote consumers can audit
+;; what a green suite actually produced without trusting prose.
+(define (evidence-entries project suite)
+  (for/list ([entry (in-list (acceptance-suite-evidence suite))])
+    (define relative (car entry))
+    (define path (project-path project relative))
+    (define present (file-exists? path))
+    (hash 'path relative
+          'kind (symbol->string (cdr entry))
+          'present present
+          'bytes (if present (file-size path) 'null)
+          'sha256 (if present (string-downcase (sha256-file/hex path)) 'null))))
 
 (define (execute-suite project suite)
   (define command (acceptance-suite-command suite))
@@ -334,7 +416,7 @@
           (hash 'id (symbol->string (acceptance-suite-id suite))
                 'kind (symbol->string (acceptance-suite-kind suite))
                 'command (acceptance-suite-command suite)
-                'evidence (acceptance-suite-evidence suite)
+                'evidence (evidence-entries project suite)
                 'status "fail"
                 'duration-ms
                 (inexact->exact
@@ -371,7 +453,7 @@
     (hash 'id (symbol->string (acceptance-suite-id suite))
           'kind (symbol->string (acceptance-suite-kind suite))
           'command (acceptance-suite-command suite)
-          'evidence (acceptance-suite-evidence suite)
+          'evidence (evidence-entries project suite)
           'status (if passed? "pass" "fail")
           'duration-ms
           (inexact->exact

rivet-cli/inspect.rkt

@@ -294,6 +294,9 @@
         (list (action "inspect" "raco rivet inspect --json" #f)
               (action "diagnose" "raco rivet doctor --json" #f)
               (action "acceptance" "raco rivet acceptance --json" #f)
+              (action "verify-remote"
+                      "raco rivet verify-remote --repo OWNER/REPO --ref HEAD"
+                      #f)
               (action "inspect-schema" "raco rivet schema --json" #f)
               (action "save-schema-baseline"
                       "raco rivet schema --output rivet-schema.json"