From 8d75c51eec0d43701c48a847d6bec74696f1dbc7 Mon Sep 17 00:00:00 2001 From: Hero-Gamer Date: Tue, 22 Sep 2026 13:49:24 -0400 Subject: [PATCH 1/3] =?UTF-8?q?docs:=20Mutation-Gated=20Testing=20?= =?UTF-8?q?=E2=80=94=20demonstrable=20killer=20per=20#676?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agents/skills/ship-pr/SKILL.md | 12 ++++++++++++ AGENTS.md | 7 +++++++ TESTING.md | 24 ++++++++++++++++++++++++ 3 files changed, 43 insertions(+) diff --git a/.agents/skills/ship-pr/SKILL.md b/.agents/skills/ship-pr/SKILL.md index f92c00620..73237873a 100644 --- a/.agents/skills/ship-pr/SKILL.md +++ b/.agents/skills/ship-pr/SKILL.md @@ -155,3 +155,15 @@ If CI `coverage` fails, add a pin and push. Floor: [`TESTING.md`](../../../TESTI Operator musl binaries are [`docs/releases.md`](../../../docs/releases.md). Do not `nix build .#rbitcoin-musl` on a feature branch. + +### Behavior-Killer Gate + +Before first production commit: +1. Classify changed behavior (Tier0–5) +2. Identify killer: test + exact observable assertion +3. Prove killer fails without the behavior +4. Run `cargo mutants --in-diff` +5. Classify all survivors +6. CI passed alone is not justification + +Docs-only change: exempt from mutants gate. diff --git a/AGENTS.md b/AGENTS.md index c08ca012c..29fe6672b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -124,3 +124,10 @@ Playbooks: Owner playbook: [`docs/releases.md`](docs/releases.md). - Core functional harness: [`.agents/skills/core-functional/SKILL.md`](.agents/skills/core-functional/SKILL.md). + +# Hard Rules — Mutation-Gated Testing +- Every new production behavior must have a demonstrable killer: test/journey/check + exact observable assertion that fails when behavior is removed/inverted. Name alone is not proof. +- Assert observable state, not implementation text. +- Mutation survivors must be classified: 0 / irrelevant with reason / killed by journey with demonstration / Tier4 fixture. "CI passed" alone is insufficient. +- Specialized / property tests cover behavior classes `cargo-mutants` cannot model well (concurrency, ordering, cancellation, time). +- Full policy lives in TESTING.md. diff --git a/TESTING.md b/TESTING.md index 2efdcf5b1..6dde1d4d7 100644 --- a/TESTING.md +++ b/TESTING.md @@ -522,3 +522,27 @@ python3 scripts/core-functional/check_inventory.py ./scripts/overlay-functional/run.sh.test.sh ./scripts/overlay-functional/run.sh --list ``` + +## Mutation and Behavior-Killer Policy + +Every new production behavior must have a demonstrable killer appropriate to its behavior class. + +**What counts as behavior:** new rejection/acceptance decision, error code, RPC field, new file/cleanup path/cap/format/crash recovery, new ranking/selection/timeout/prefill/persistence, new concurrency/cancellation/ordering guarantee. Refactor/move with no behavior change needs no new killer but must show 0 applicable survivors. + +**Tier0 — Primitives / unsafe / serialization / SIMD / arithmetic:** independent reference/oracle, not copy of optimized impl. Boundary + invalid encoding. Unsafe = narrowest invariant + MIRI where applicable. + +**Tier1 — Observable decisions (consensus, RPC, mempool, CLI/config):** assert exact variant/code, not `is_err()` or string matching. Exercise real accept/reject path. + +**Tier2 — Stateful P2P / IBD / Time / Concurrency:** assert intermediate state — not just final tip height. Injected mock clock is default. Concurrency/async: `cargo-mutants` advisory only; needs property test for ordering/cancellation. + +**Tier3 — Store / File / Crash / Schema:** lifecycle = precondition + operation + postcondition. Crash: interrupt → reopen → assert documented recovery. Schema: test every compatibility direction. + +**Tier4 — Tooling / CI / Docs:** no mutants gate. + +**Tier5 — Docs-only:** exempt if diff only `*.md` and no `crates/` change. + +**First-commit gate:** +```bash +git diff origin/master.. --unified=0 > /tmp/pr.diff +cargo test -p --lib +cargo mutants -p --in-diff /tmp/pr.diff -q From cddabd5ed2ae7c1e1d110e32b56d75972bf2bf07 Mon Sep 17 00:00:00 2001 From: Hero-Gamer Date: Tue, 22 Sep 2026 13:54:33 -0400 Subject: [PATCH 2/3] docs: align style and gate commands per #676 --- AGENTS.md | 4 ++-- TESTING.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 29fe6672b..57da893fd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -125,8 +125,8 @@ Playbooks: - Core functional harness: [`.agents/skills/core-functional/SKILL.md`](.agents/skills/core-functional/SKILL.md). -# Hard Rules — Mutation-Gated Testing -- Every new production behavior must have a demonstrable killer: test/journey/check + exact observable assertion that fails when behavior is removed/inverted. Name alone is not proof. + +- **Mutation-gated testing:** Every new production behavior must have a demonstrable killer: test/journey/check + exact observable assertion that fails when behavior is removed/inverted. Name alone is not proof. - Assert observable state, not implementation text. - Mutation survivors must be classified: 0 / irrelevant with reason / killed by journey with demonstration / Tier4 fixture. "CI passed" alone is insufficient. - Specialized / property tests cover behavior classes `cargo-mutants` cannot model well (concurrency, ordering, cancellation, time). diff --git a/TESTING.md b/TESTING.md index 6dde1d4d7..f520f5107 100644 --- a/TESTING.md +++ b/TESTING.md @@ -544,5 +544,5 @@ Every new production behavior must have a demonstrable killer appropriate to its **First-commit gate:** ```bash git diff origin/master.. --unified=0 > /tmp/pr.diff -cargo test -p --lib -cargo mutants -p --in-diff /tmp/pr.diff -q +cargo test -p --lib -- --quiet +cargo mutants -p --in-diff /tmp/pr.diff -j2 -- --skip core -q From dd70da6cb439750c5ca9ca577e862f988ec7e54f Mon Sep 17 00:00:00 2001 From: "rearden-grok[bot]" <317016512+rearden-grok[bot]@users.noreply.github.com> Date: Wed, 23 Sep 2026 11:46:29 -0700 Subject: [PATCH 3/3] ci: require the PR mutants shards A shard that finishes with a non-zero cargo mutants exit fails the check. A shard killed at 30 minutes exits 0, so a timeout is a pass. Docs-only diffs still report the check. The policy text matches that gate instead of a second local command. --- .agents/skills/ship-pr/SKILL.md | 16 ++----- .github/workflows/ci.yml | 75 +++++++++++++++++++++++++-------- .github/workflows/mutants.yml | 2 +- AGENTS.md | 10 ++--- CONTRIBUTING.md | 2 +- TESTING.md | 26 ++---------- docs/releases.md | 2 +- scripts/pr-checks-watch.sh | 4 ++ scripts/pr-checks-watch.test.sh | 10 ++++- 9 files changed, 83 insertions(+), 64 deletions(-) diff --git a/.agents/skills/ship-pr/SKILL.md b/.agents/skills/ship-pr/SKILL.md index 73237873a..dbd8247e2 100644 --- a/.agents/skills/ship-pr/SKILL.md +++ b/.agents/skills/ship-pr/SKILL.md @@ -75,7 +75,9 @@ Coverage and native `windows` / `macos` stay GitHub Actions; ## Push and poll Required jobs: `fmt`, `deny`, `clippy`, `ast-grep`, `test`, `windows`, -`macos`, `coverage`, `nixos-module-eval`. Structural scan is +`macos`, `coverage`, `nixos-module-eval`, and `mutants (0/4)` through +`mutants (3/4)`. A mutants shard killed at 30 minutes is a pass; a +finished non-zero `cargo mutants` exit is a fail. Structural scan is `./scripts/ast-grep.sh`. `windows` / `macos` are native store plus `--smoke`, not operator zips. @@ -155,15 +157,3 @@ If CI `coverage` fails, add a pin and push. Floor: [`TESTING.md`](../../../TESTI Operator musl binaries are [`docs/releases.md`](../../../docs/releases.md). Do not `nix build .#rbitcoin-musl` on a feature branch. - -### Behavior-Killer Gate - -Before first production commit: -1. Classify changed behavior (Tier0–5) -2. Identify killer: test + exact observable assertion -3. Prove killer fails without the behavior -4. Run `cargo mutants --in-diff` -5. Classify all survivors -6. CI passed alone is not justification - -Docs-only change: exempt from mutants gate. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 02b7e94c2..48c560f2d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -31,8 +31,9 @@ env: # check (a skip would hide required deny / ast-grep). # clippy, test, windows, and macos start immediately. clippy stays parallel: # coverage waits on it, and a cold rust-cache is a full workspace compile. -# coverage and the advisory mutants shards wait for fmt / clippy / test. -# Weekly full mutants sweep stays mutants.yml. +# coverage and the mutants shards wait for fmt / clippy / test. +# A shard killed at 30m passes; a finished shard with a non-zero +# cargo-mutants exit fails. Weekly sweep stays mutants.yml. # nixos-module-runtime is a labeled job (nixos-module-runtime.yml), not this file. # Operator binaries are GitHub Releases (release.yml), not this workflow. # @@ -205,8 +206,8 @@ jobs: - name: store + node smoke run: ./scripts/ci-os-smoke.sh - # Crates, Cargo manifest/lock, or these workflow files. Docs-only PRs - # skip the four shard runners. Starts immediately; shards still wait for test. + # Crates, Cargo manifest/lock, or these workflow files. The shard jobs + # still run on a docs-only PR and exit 0 so the required check reports. mutants-diff: if: github.event_name == 'pull_request' runs-on: ubuntu-latest @@ -233,21 +234,21 @@ jobs: echo "rust=false" >> "$GITHUB_OUTPUT" fi - # PR in-diff, four shards, same window as coverage. Advisory: a missed - # mutant does not fail the job. One ~300-mutant diff at -j 2 is ~50m and - # was canceled at 30m. Weekly sweep stays mutants.yml. Owner: TESTING.md. + # PR in-diff, four shards. Required. A shard that finishes with a + # non-zero cargo-mutants exit fails the check. A shard killed at 30m + # (GNU timeout 124) passes, even if the partial log shows MISSED. + # Docs-only diffs exit 0 so the check still reports. Weekly sweep + # stays mutants.yml. Owner: TESTING.md. mutants: name: mutants (${{ matrix.shard }}/4) - needs: [fmt, clippy, test, mutants-diff] + needs: [fmt, clippy, test] if: >- github.event_name == 'pull_request' && needs.fmt.result == 'success' && needs.clippy.result == 'success' && needs.test.result == 'success' - && needs.mutants-diff.result == 'success' - && needs.mutants-diff.outputs.rust == 'true' runs-on: ubuntu-latest - timeout-minutes: 30 + timeout-minutes: 40 strategy: fail-fast: false matrix: @@ -264,35 +265,73 @@ jobs: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: fetch-depth: 0 + - name: rust or workflow diff + id: diff + env: + BASE_REF: ${{ github.base_ref }} + run: | + set -euo pipefail + git diff "origin/${BASE_REF}".. --unified=0 -- \ + crates Cargo.toml Cargo.lock \ + .github/workflows/ci.yml .github/workflows/mutants.yml \ + > git.diff + if [[ -s git.diff ]]; then + echo "rust=true" >> "$GITHUB_OUTPUT" + else + echo "rust=false" >> "$GITHUB_OUTPUT" + echo "no rust or workflow diff; shard passes" + fi - uses: dtolnay/rust-toolchain@a5f673d0ba8626c3977bb416a1612774bc82181b # rustc 1.95.0 + if: steps.diff.outputs.rust == 'true' - uses: taiki-e/install-action@9114bf4d891761788c546334fd37538eae1bf8b3 # v2.87.16 + if: steps.diff.outputs.rust == 'true' with: tool: cargo-mutants@27.1.0 - name: in-diff shard id: run - continue-on-error: true - env: - BASE_REF: ${{ github.base_ref }} + if: steps.diff.outputs.rust == 'true' run: | set -euo pipefail - git diff "origin/${BASE_REF}".. --unified=0 > git.diff set +e - cargo mutants --workspace --in-diff git.diff \ - --shard "${{ matrix.shard }}/4" --sharding slice -j 2 \ + timeout --signal=TERM --kill-after=60s 30m \ + cargo mutants --workspace --in-diff git.diff \ + --shard "${{ matrix.shard }}/4" --sharding slice -j 2 \ > mutants-shard.log 2>&1 ec=$? set -e missed=$(grep -c '^MISSED' mutants-shard.log || true) echo "missed=${missed}" >> "$GITHUB_OUTPUT" cat mutants-shard.log + if [[ "$ec" -eq 124 ]]; then + echo "result=timeout" >> "$GITHUB_OUTPUT" + echo "shard hit the 30m limit; counting as success" + exit 0 + fi + echo "result=finished" >> "$GITHUB_OUTPUT" exit "$ec" - name: summary if: always() run: | { echo "### mutants ${{ matrix.shard }}/4" - echo "Advisory (not a merge gate). Outcome: ${{ steps.run.outcome }}. Missed: ${{ steps.run.outputs.missed }}." + echo "Required. Outcome: ${{ steps.run.outcome }}. Result: ${{ steps.run.outputs.result }}. Missed: ${{ steps.run.outputs.missed }}." + echo "A 30m timeout counts as success. A finished non-zero exit fails." } >> "$GITHUB_STEP_SUMMARY" + - name: fail unless success or timeout + if: always() && steps.diff.outputs.rust == 'true' + run: | + set -euo pipefail + result="${{ steps.run.outputs.result }}" + outcome="${{ steps.run.outcome }}" + if [[ "$result" == "timeout" || "$outcome" == "cancelled" ]]; then + echo "timeout counts as success" + exit 0 + fi + if [[ "$outcome" == "success" && "$result" == "finished" ]]; then + exit 0 + fi + echo "mutants shard failed (outcome=${outcome} result=${result})" + exit 1 - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: always() with: diff --git a/.github/workflows/mutants.yml b/.github/workflows/mutants.yml index 27b5d2741..fa63d7c9c 100644 --- a/.github/workflows/mutants.yml +++ b/.github/workflows/mutants.yml @@ -1,5 +1,5 @@ # Weekly cargo-mutants. Not a required check. -# PR in-diff shards are ci.yml job `mutants` (parallel with coverage). +# PR in-diff shards are ci.yml job `mutants` (required; a 30m timeout passes). # Must pass --workspace: default-members is rbitcoin-node only. # Owner: TESTING.md (Mutation testing). Snapshot lists: docs/mutants/. diff --git a/AGENTS.md b/AGENTS.md index 57da893fd..9fb76978a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -124,10 +124,6 @@ Playbooks: Owner playbook: [`docs/releases.md`](docs/releases.md). - Core functional harness: [`.agents/skills/core-functional/SKILL.md`](.agents/skills/core-functional/SKILL.md). - - -- **Mutation-gated testing:** Every new production behavior must have a demonstrable killer: test/journey/check + exact observable assertion that fails when behavior is removed/inverted. Name alone is not proof. -- Assert observable state, not implementation text. -- Mutation survivors must be classified: 0 / irrelevant with reason / killed by journey with demonstration / Tier4 fixture. "CI passed" alone is insufficient. -- Specialized / property tests cover behavior classes `cargo-mutants` cannot model well (concurrency, ordering, cancellation, time). -- Full policy lives in TESTING.md. +- PR `mutants (0/4)` through `mutants (3/4)` are required. A shard that hits + the 30 minute limit passes. A finished shard with a non-zero + `cargo mutants` exit fails. Owner: [`TESTING.md`](TESTING.md). diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2913a6543..7f32294b8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -241,7 +241,7 @@ to it. Darwin operator binaries come from the `macos-14` release job, not Nix. Required **jobs** match [`.github/workflows/ci.yml`](./.github/workflows/ci.yml) (`fmt`, `deny`, `clippy`, `ast-grep`, `test`, `windows`, `macos`, -`coverage`, `nixos-module-eval`) so a red run shows which gate failed without +`coverage`, `nixos-module-eval`, `mutants (0/4)` … `mutants (3/4)`) so a red run shows which gate failed without digging into a monolithic job log. Label **`nixos-module-runtime`** to run the NixOS module qemu test (not eval). That job is not required. diff --git a/TESTING.md b/TESTING.md index f520f5107..758d7d91f 100644 --- a/TESTING.md +++ b/TESTING.md @@ -254,7 +254,7 @@ matches scalar in default tests). Owner: [`docs/quality.md`](./docs/quality.md). | **cargo-crap** | After LCOV, `./scripts/coverage.sh` calls `./scripts/coverage-crap.sh` (skip if `cargo-crap` missing). `--fail-above --threshold 30`; `.cargo-crap.toml` allowlists today's production CRAP>30 functions (remove a name when it scores ≤30). Dry-run: `CRAP_DRY_RUN=1 ./scripts/coverage-crap.sh`. Self-test: `./scripts/coverage-crap.test.sh` | Rides required `coverage`. No `--fail-regression` (llvm-cov coverage % jitters per function) | | **coverage ignore / badge** | `./scripts/coverage.test.sh` (filename ignore, Tier A IBD not skipped, 92% floor, Shields JSON). Publish dry-run: `BADGE_DRY_RUN=1 ./scripts/publish-coverage-badge.sh` | `test` job self-test; `coverage` job writes `coverage/badge.json` and, on green `master`, pushes `badges/coverage.json` | | **Miri** | `./scripts/miri.sh` → `cargo +nightly miri test -p rbitcoin-primitives`. Dry-run: `MIRI_DRY_RUN=1 ./scripts/miri.sh`. Self-test: `./scripts/miri.test.sh` | Nightly `miri.yml` (not required). Never `--workspace` | -| **cargo-mutants** | Must pass `--workspace` (`Cargo.toml` `default-members` is `rbitcoin-node` only; without `--workspace` the list is ~7 mutants). PR: `git diff origin/.. --unified=0 > git.diff` then `cargo mutants --workspace --in-diff git.diff --shard k/4 --sharding slice -j 2` (same flags on every shard). Weekly: `cargo mutants --workspace --shard N/8 -j 2`. Snapshot of missed/timeouts: [`docs/mutants/`](docs/mutants/). | PR: `ci.yml` `mutants`, 4 shards, needs fmt/clippy/test (same window as coverage). `continue-on-error` (not a merge gate). Skipped when the diff does not touch `crates/`, `Cargo.toml`, `Cargo.lock`, `ci.yml`, or `mutants.yml`. Weekly `mutants.yml` Sunday 03:00 UTC, 8 shards, 360min cap (not required). `workflow_dispatch`. First Sunday wall-clock is still a measurement. | +| **cargo-mutants** | Must pass `--workspace` (`Cargo.toml` `default-members` is `rbitcoin-node` only; without `--workspace` the list is ~7 mutants). PR: `git diff origin/.. --unified=0 -- crates Cargo.toml Cargo.lock .github/workflows/ci.yml .github/workflows/mutants.yml > git.diff` then `cargo mutants --workspace --in-diff git.diff --shard k/4 --sharding slice -j 2` (same flags on every shard). Weekly: `cargo mutants --workspace --shard N/8 -j 2`. Snapshot of missed/timeouts: [`docs/mutants/`](docs/mutants/). | PR: `ci.yml` `mutants (0/4)`–`mutants (3/4)`, required, needs fmt/clippy/test. A shard that finishes with a non-zero exit fails. A shard killed at 30 minutes counts as a pass, even if the partial log shows `MISSED`. No rust or workflow diff: the shard exits 0. Weekly `mutants.yml` Sunday 03:00 UTC, 8 shards, 360min cap (not required). `workflow_dispatch`. | Artifact silos above are unchanged: ast-grep / Miri dry-run / crap dry-run do not write `target/`. `mutants.out/` is gitignored. @@ -523,26 +523,8 @@ python3 scripts/core-functional/check_inventory.py ./scripts/overlay-functional/run.sh --list ``` -## Mutation and Behavior-Killer Policy +## What a mutant kill looks like -Every new production behavior must have a demonstrable killer appropriate to its behavior class. +The PR gate is the four `mutants` shards in [`ci.yml`](.github/workflows/ci.yml), described in the table above. Do not run a different `cargo mutants` command and treat that as the gate. -**What counts as behavior:** new rejection/acceptance decision, error code, RPC field, new file/cleanup path/cap/format/crash recovery, new ranking/selection/timeout/prefill/persistence, new concurrency/cancellation/ordering guarantee. Refactor/move with no behavior change needs no new killer but must show 0 applicable survivors. - -**Tier0 — Primitives / unsafe / serialization / SIMD / arithmetic:** independent reference/oracle, not copy of optimized impl. Boundary + invalid encoding. Unsafe = narrowest invariant + MIRI where applicable. - -**Tier1 — Observable decisions (consensus, RPC, mempool, CLI/config):** assert exact variant/code, not `is_err()` or string matching. Exercise real accept/reject path. - -**Tier2 — Stateful P2P / IBD / Time / Concurrency:** assert intermediate state — not just final tip height. Injected mock clock is default. Concurrency/async: `cargo-mutants` advisory only; needs property test for ordering/cancellation. - -**Tier3 — Store / File / Crash / Schema:** lifecycle = precondition + operation + postcondition. Crash: interrupt → reopen → assert documented recovery. Schema: test every compatibility direction. - -**Tier4 — Tooling / CI / Docs:** no mutants gate. - -**Tier5 — Docs-only:** exempt if diff only `*.md` and no `crates/` change. - -**First-commit gate:** -```bash -git diff origin/master.. --unified=0 > /tmp/pr.diff -cargo test -p --lib -- --quiet -cargo mutants -p --in-diff /tmp/pr.diff -j2 -- --skip core -q +A new production behavior needs a test that fails when that behavior is removed or inverted. Assert the observable result (the error variant, the bytes, the height), not the test name. Concurrency, ordering, and cancellation are not what `cargo-mutants` models; those need a direct test. A diff that does not touch `crates/`, the Cargo manifests, or the mutants workflows does not run mutants. diff --git a/docs/releases.md b/docs/releases.md index ea48e8605..63bba5f7e 100644 --- a/docs/releases.md +++ b/docs/releases.md @@ -103,7 +103,7 @@ Cargo.toml). Those PRs run Core functional even without a label. | Check | Who | |-------|-----| -| `fmt` `deny` `clippy` `ast-grep` `test` `windows` `macos` `coverage` `nixos-module-eval` | Every PR (`ci.yml`) | +| `fmt` `deny` `clippy` `ast-grep` `test` `windows` `macos` `coverage` `nixos-module-eval` `mutants (0/4)`–`(3/4)` | Every PR (`ci.yml`). A mutants shard killed at 30 minutes passes; a finished non-zero exit fails. | | `core-functional` | Nightly, `workflow_dispatch`, label **`core-functional`**, label **`release`**, **or** ship version | | `overlay-functional` | Nightly (`42 6`), `workflow_dispatch`, label **`overlay-functional`**, label **`release`**, **or** ship version. Not required. Not in `release-extra` yet. | | `warnet-example` | Label **`warnet`** or `workflow_dispatch`. Two-tank Docker lab. Not required. Not a ship gate. | diff --git a/scripts/pr-checks-watch.sh b/scripts/pr-checks-watch.sh index 05ff6b775..93649fac1 100755 --- a/scripts/pr-checks-watch.sh +++ b/scripts/pr-checks-watch.sh @@ -21,6 +21,10 @@ REQUIRED=( macos coverage nixos-module-eval + "mutants (0/4)" + "mutants (1/4)" + "mutants (2/4)" + "mutants (3/4)" ) usage() { diff --git a/scripts/pr-checks-watch.test.sh b/scripts/pr-checks-watch.test.sh index c88bb2bf5..d90c599cf 100755 --- a/scripts/pr-checks-watch.test.sh +++ b/scripts/pr-checks-watch.test.sh @@ -45,7 +45,11 @@ all_green="$( windows pass 1m \ macos pass 50s \ coverage pass 2m \ - nixos-module-eval pass 32s + nixos-module-eval pass 32s \ + 'mutants (0/4)' pass 2m \ + 'mutants (1/4)' pass 2m \ + 'mutants (2/4)' pass 2m \ + 'mutants (3/4)' pass 2m )" rc=0 out="$(CI_PR_CHECKS_TEXT="$all_green" "$RUN" --once 2>&1)" || rc=$? @@ -63,6 +67,10 @@ analyze_pending_windows_green="$( macos pass 50s \ coverage pass 2m \ nixos-module-eval pass 32s \ + 'mutants (0/4)' pass 2m \ + 'mutants (1/4)' pass 2m \ + 'mutants (2/4)' pass 2m \ + 'mutants (3/4)' pass 2m \ 'Analyze (rust)' pending 0 )" rc=0