Skip to content

fix(workbench): give the workbench a ref so reviewer clones succeed - #565

Merged
piekstra merged 3 commits into
mainfrom
fix/workbench-refs
Aug 13, 2026
Merged

fix(workbench): give the workbench a ref so reviewer clones succeed#565
piekstra merged 3 commits into
mainfrom
fix/workbench-refs

Conversation

@piekstra

Copy link
Copy Markdown
Contributor

The symptom

On a real PR, four of five reviewers reported 0 findings and the rollup rendered it as a clean review. They had not found nothing — they had never run:

security:code-auditor       incomplete_failed
rust:implementation-tests   incomplete_failed
architecture:solid          incomplete_failed
policies:conventions        incomplete_failed
documentation:docs          complete_broad     ← the only agent needing no checkout

A reviewer that cannot start contributes zero findings, so a total failure of the reviewer fleet is indistinguishable from an all-clear. That is the part worth fixing regardless of the cause.

The cause

Prepare builds the workbench with initfetch --no-tags <remote> <sha>checkout --detach. Fetching by SHA writes only FETCH_HEAD, and a detached checkout writes no ref, so the resulting .git has no refs/:

workbench/.git:  FETCH_HEAD HEAD config description hooks index info logs objects
healthy clone:   HEAD config description hooks index info logs objects packed-refs refs

Git does not consider a directory without refs/ a repository, so the per-reviewer git clone --no-hardlinks <workbench> <workspace> fails with repository does not exist. Confirmed by hand: copying that workbench and writing a single ref makes the identical clone succeed.

The fix

git update-ref refs/heads/cr-review-head <head SHA> after the checkout. HEAD stays detached, so the pinned-checkout semantics are unchanged; the repo simply becomes clonable.

Test

TestPrepareLeavesWorkbenchClonable asserts the head is reachable through a real ref and then performs the actual git clone --no-hardlinks a reviewer workspace uses. Verified it fails without the fix and passes with it. Full suite green.

Prepare builds the workbench with init + fetch-by-SHA + checkout --detach, so
the directory ends up with FETCH_HEAD and no refs/ at all. Git does not treat
such a directory as a repository, so the per-reviewer

    git clone --no-hardlinks <workbench> <reviewer-workspace>

fails with "repository does not exist" for every agent that needs a workspace.

The failure is silent and actively misleading. A reviewer that cannot start is
recorded as incomplete_failed and contributes **zero findings**, which the
rollup then renders as a clean review. On a real PR this produced "0 findings"
across four of five reviewers -- only the one agent that reads the diff without
a checkout actually ran -- and the result read as an all-clear.

Writing refs/heads/cr-review-head at the head SHA after checkout makes the
workbench a valid, clonable repository. HEAD stays detached, so nothing else
about the pinned checkout changes.

The regression test asserts the property that matters -- that the workbench can
actually be cloned the way a reviewer workspace is created -- and fails without
the fix.

@piekstra-dev piekstra-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR Review

Reviewed commit: 992b1f3e4931
Profile: reviewer - Posting as: piekstra-dev

Summary

Reviewer Findings
go:implementation-tests 0
harness-engineering:repo-health 0
security:code-auditor 0
architecture:solid 2
architecture:solid (2 findings)

Major - internal/workbench/workbench.go:154

U-L1 (behavioral contract): Prepare now establishes a new postcondition — the workbench is clonable because it has refs/heads/cr-review-head — but the reuse fast path does not check it. reusable (workbench.go:189-226) validates metadata, commitPresent for base/head, and verifyClean; none of those touch refs/, and metadataSchemaVersion is unchanged at 2 with schema 1 still accepted (workbench.go:204). A workbench built by a pre-fix binary therefore passes the gate verbatim, Prepare returns early, and every git clone --no-hardlinks in prepareReviewerWorkspace (workbench.go:416) still fails with "repository does not exist". That is precisely the silent zero-findings outcome this PR exists to fix, and it hits the most likely user: someone resuming the run whose reviewer fleet just died (NewestCompatibleIncompleteRun reuses the existing run's artifact dir, including workbench/).

Both paths out of Prepare should guarantee the same postcondition. Concrete fix: either add the ref check to the reuse gate — e.g. a deps.gitCommand(ctx, repoDir, "rev-parse", "--verify", workbenchHeadRef+"^{commit}") alongside the existing commitPresent calls, or fold it into verifyClean so build and reuse share one clonability assertion — or bump metadataSchemaVersion to 3 and stop accepting 1/2 in reusable, forcing a rebuild of workbenches that predate the ref. The check-based option is cheaper and keeps durable task fingerprints stable, which the schema-1 reuse note in docs/checkout-native-review-contract.md:106-109 explicitly cares about. Whichever you pick, the existing TestPrepareLeavesWorkbenchClonable should get a sibling that prepares a workbench, deletes refs/heads/cr-review-head, calls Prepare again, and asserts the result is clonable.

Minor - internal/workbench/workbench.go:147

U-G1 (new durable surface gets a documented contract): the workbench's ref set is now load-bearing shared state — workbench/repo/ is a durable run artifact whose shape docs/checkout-native-review-contract.md:44-80 and 90-109 specify, and the reviewer-clone consumer depends on it. The ref is described only in this inline comment. Worth an amendment to that doc's workbench-layout notes ('workbench/repo/ is a clean pinned checkout at the PR head SHA, carrying refs/heads/cr-review-head so the per-reviewer clone succeeds'), so the next change to Prepare does not re-lose it.

The same amendment should settle a second question this line decides implicitly: only the head gets a ref, and git clone transfers only ref-reachable objects, so the base commit that ensureCommit deliberately fetches (workbench.go:134) and that reusable asserts is present (workbench.go:215) does not survive into reviewer workspaces whenever base is not an ancestor of head — the normal case for a branch cut from an older main. Reviewers cannot then run git log base..HEAD or git diff <base> locally. This is defensible either way: reviewers are handed the provider-generated diff.patch and nothing in the pipeline reads base from the workspace, so it may simply be out of the workbench's job. If you want the base reachable, the fix is one more update-ref refs/heads/cr-review-base <base SHA> next to this call; if you do not, say so in the doc so the omission reads as a decision rather than an oversight. Evidence that would change this to a real defect: any reviewer prompt or workspace tool that resolves the base SHA inside workspaceRepo.

Reviewer Coverage

  • go:implementation-tests — complete (broad); skipped: none; constraints: none
  • harness-engineering:repo-health — complete (broad); skipped: none; constraints: none
  • security:code-auditor — complete (broad); inspected 1 assigned file (2 inspected across reviewers): internal/workbench/workbench.go; skipped: none; constraints: none
  • architecture:solid — complete (broad); inspected 1 assigned file (2 inspected across reviewers): internal/workbench/workbench.go; skipped: none; constraints: Only the assigned file internal/workbench/workbench.go was reviewed as a finding surface; workbench_test.go, runartifact, pipeline, and docs/checkout-native-review-contract.md were read for context only. Sandbox denied go test and git invocations in the reviewer workspace, so the fix and the reuse-path reasoning were verified by code reading rather than by running the suite. The PR's second stated symptom (a fleet-wide reviewer failure rendering as a clean review in the rollup) lives outside the assigned file and was not reviewed.
Inspected files (2)
  • internal/workbench/workbench.go
  • internal/workbench/workbench_test.go

0 PR discussion threads considered. 0 summarized; 0 resolved.


Completed in 5m 13s | $3.20 | claude-sonnet-5, claude-opus-5 | cr dev
Field Value
Model claude-sonnet-5, claude-opus-5
Reviewers go:implementation-tests, harness-engineering:repo-health, security:code-auditor, architecture:solid
Engine claude_cli · claude-sonnet-5, claude-opus-5
Reviewed by cr · piekstra-dev
Duration 5m 13s wall · 7m 54s compute
Cost $3.20
Tokens 116 in / 31.9k out

Per-workstream usage

  • orchestrator-selection — claude-sonnet-5
    • In: 6
    • Out: 2.6k
    • Cache read: 66.7k
    • Cache create: 17.7k
    • Cost: $0.17
    • Duration: 30s
  • go:implementation-tests — claude-sonnet-5
    • In: 14
    • Out: 4.5k
    • Cache read: 246.6k
    • Cache create: 46.2k
    • Cost: $0.42
    • Duration: 1m 00s
  • harness-engineering:repo-health — claude-sonnet-5
    • In: 26
    • Out: 5.7k
    • Cache read: 568.1k
    • Cache create: 48.4k
    • Cost: $0.55
    • Duration: 1m 15s
  • security:code-auditor — claude-sonnet-5
    • In: 12
    • Out: 2.2k
    • Cache read: 183.7k
    • Cache create: 24.5k
    • Cost: $0.24
    • Duration: 35s
  • architecture:solid — claude-opus-5
    • In: 52
    • Out: 16.0k
    • Cache read: 1.1M
    • Cache create: 64.9k
    • Cost: $1.60
    • Duration: 3m 48s
  • orchestrator-rollup — claude-sonnet-5
    • In: 6
    • Out: 965
    • Cache read: 101.3k
    • Cache create: 31.2k
    • Cost: $0.23
    • Duration: 43s

Comment thread internal/workbench/workbench.go
Comment thread internal/workbench/workbench.go
…e ref

Review raised that Prepare's new postcondition -- the workbench is clonable
because it carries a head ref -- was established only on the build path, while
the reuse fast path returned early without checking it.

Adding the assertion is right regardless, so it is here. But the specific
scenario in the finding does not appear reachable: a workbench missing refs/
entirely is already rejected, because commitPresent and verifyClean shell out
to git and fail in a directory git does not consider a repository. I tried
several ways to construct a reuse that reaches a reviewer clone with a broken
workbench and could not; deleting only the ref leaves refs/ in place, which git
still accepts and clones fine.

So the check stays as belt-and-braces for the narrower case (refs/ present, head
ref gone) and to stop a future change to those checks quietly dropping
clonability -- and the comment says exactly that rather than claiming to fix a
failure I could not reproduce. No test accompanies it: a test that passes with
the change reverted verifies nothing, and I would rather ship the guard honestly
labelled than a green assertion that discriminates nothing.

Also documents the ref in the checkout contract, per the same review. That
includes the consequence the reviewer surfaced: only the head is given a ref, so
the base commit does not reach reviewer workspaces when base is not an ancestor
of head. Recorded as a decision with the one-line fix if it ever needs to change.
@piekstra

Copy link
Copy Markdown
Contributor Author

Both findings addressed in d4b668f, one of them partly as a decline with reasons — flagging that explicitly so it can be overruled.

Major (reuse path): the assertion is added — both exits from Prepare should guarantee the same postcondition, and a future change to the existing checks shouldn't be able to drop clonability quietly.

But I could not reproduce the failure as described, and I want to be honest about that rather than let the commit imply otherwise. A workbench missing refs/ entirely is already rejected by the reuse gate: commitPresent and verifyClean shell out to git, and those fail in a directory git doesn't consider a repository. I tried it several ways — deleting only the head ref leaves refs/ in place, which git still accepts and clones from fine, so reusable returning true there is harmless.

So the check ships as belt-and-braces for the narrower case (refs/ present, head ref gone), and the comment says that rather than claiming a fix I couldn't demonstrate. No test accompanies it — I wrote one, confirmed it passed with the change reverted, and removed it. A test that doesn't discriminate reads as coverage while verifying nothing, which is the same class of problem as this PR's original bug.

If you can construct the reuse-reaches-clone case, I'll happily add the test and drop the hedging in the comment.

Minor (documentation): done. docs/checkout-native-review-contract.md now records the ref as load-bearing, and settles the second question you raised — only the head gets a ref, so the base commit doesn't reach reviewer workspaces when base isn't an ancestor of head. Written as a decision with the one-line change if it ever needs to be otherwise, rather than leaving it to read as an oversight.

Worth noting: this PR was reviewed by a cr binary built from this branch, because the released one hits the very bug being fixed. All four reviewers ran and none reported incomplete_failed — which is the fix demonstrating itself.

@piekstra-dev piekstra-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR Review

Reviewed commit: d4b668f4d386
Profile: reviewer - Posting as: piekstra-dev

Summary

Reviewer Findings
go:implementation-tests 1
harness-engineering:repo-health 0
security:code-auditor 0
architecture:solid 0
go:implementation-tests (1 finding)

Major - internal/workbench/workbench_test.go:724

The reuse-path fix (workbench.go:230, refPresent check against workbenchHeadRef) has no test proving it, even though it guards the exact silent-zero-findings scenario this PR exists to fix. TestPrepareReusesValidV1WorkbenchWithoutRewritingMetadata only exercises reuse of a workbench that already has the ref (built by this same fixed code in the same test run), so it can't distinguish 'refPresent correctly gates reuse' from 'refPresent is a no-op'. Nothing in the suite deletes refs/heads/cr-review-head from an otherwise-valid, metadata-matching workbench and asserts Prepare rebuilds it into a clonable state rather than reusing it verbatim -- which is precisely the pre-fix-binary/resumed-run scenario the reuse check was added for. Add a sibling to TestPrepareLeavesWorkbenchClonable that prepares a workbench, runs update-ref -d refs/heads/cr-review-head (or equivalent) against the resulting repo to simulate a workbench built before this fix, calls Prepare again with an unchanged Request, and asserts (a) Prepare did not skip rebuilding via the reuse fast path and (b) the resulting workbench is clonable via the same git clone --no-hardlinks assertion already used in TestPrepareLeavesWorkbenchClonable. Without this, a future refactor of reusable() or refPresent() that silently breaks the gate would pass the full suite.

Reviewer Coverage

  • go:implementation-tests — complete (constrained); inspected 2 assigned files (3 inspected across reviewers): internal/workbench/workbench.go, internal/workbench/workbench_test.go; skipped: none; constraints: none
  • harness-engineering:repo-health — complete (constrained); skipped: none; constraints: none
  • security:code-auditor — complete (constrained); inspected 1 assigned file (3 inspected across reviewers): internal/workbench/workbench.go; skipped: none; constraints: none
  • architecture:solid — complete (constrained); inspected 1 assigned file (3 inspected across reviewers): internal/workbench/workbench.go; skipped: none; constraints: Both findings from the prior round are resolved at d4b668f: the reuse gate now asserts the head ref (workbench.go:230), and the doc amendment records the ref plus the deliberate head-only decision (checkout-native-review-contract.md:77-85). Findings are limited to the assigned file internal/workbench/workbench.go; workbench_test.go and docs/checkout-native-review-contract.md changed in this PR and were read for context only. Sandbox denied go test and git invocations from this workspace, so the reuse-path and clonability reasoning is from code reading, not from running the suite.
Inspected files (3)
  • docs/checkout-native-review-contract.md
  • internal/workbench/workbench.go
  • internal/workbench/workbench_test.go

0 PR discussion threads considered. 0 summarized; 0 resolved.


Completed in 2m 22s | $3.43 | claude-sonnet-5, claude-opus-5 | cr dev
Field Value
Model claude-sonnet-5, claude-opus-5
Reviewers go:implementation-tests, harness-engineering:repo-health, security:code-auditor, architecture:solid
Engine claude_cli · claude-sonnet-5, claude-opus-5
Reviewed by cr · piekstra-dev
Duration 2m 22s wall · 5m 05s compute
Cost $3.43
Tokens 80 in / 20.4k out

Per-workstream usage

  • go:implementation-tests — claude-sonnet-5
    • In: 22
    • Out: 6.3k
    • Cache read: 849.3k
    • Cache create: 75.2k
    • Cost: $0.80
    • Duration: 1m 23s
  • harness-engineering:repo-health — claude-sonnet-5
    • In: 22
    • Out: 5.9k
    • Cache read: 825.2k
    • Cache create: 82.8k
    • Cost: $0.83
    • Duration: 1m 23s
  • security:code-auditor — claude-sonnet-5
    • In: 12
    • Out: 1.4k
    • Cache read: 224.8k
    • Cache create: 39.1k
    • Cost: $0.32
    • Duration: 23s
  • architecture:solid — claude-opus-5
    • In: 18
    • Out: 5.2k
    • Cache read: 644.4k
    • Cache create: 74.2k
    • Cost: $1.19
    • Duration: 1m 27s
  • orchestrator-rollup — claude-sonnet-5
    • In: 6
    • Out: 1.7k
    • Cache read: 112.9k
    • Cache create: 37.3k
    • Cost: $0.28
    • Duration: 27s

Comment thread internal/workbench/workbench_test.go
piekstra-dev
piekstra-dev previously approved these changes Aug 13, 2026

@piekstra-dev piekstra-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR Review

Reviewed commit: 0f6dfb29f1d3
Profile: reviewer - Posting as: piekstra-dev

Summary

Reviewer Findings
go:implementation-tests 0
harness-engineering:repo-health 0
security:code-auditor 0
architecture:solid 0

Reviewer Coverage

  • go:implementation-tests — complete (constrained); inspected 2 assigned files (3 inspected across reviewers): internal/workbench/workbench.go, internal/workbench/workbench_test.go; skipped: none; constraints: none
  • harness-engineering:repo-health — complete (constrained); skipped: none; constraints: none
  • security:code-auditor — complete (constrained); inspected 1 assigned file (3 inspected across reviewers): internal/workbench/workbench.go; skipped: none; constraints: none
  • architecture:solid — complete (constrained); inspected 1 assigned file (3 inspected across reviewers): internal/workbench/workbench.go; skipped: none; constraints: All three prior findings are resolved at 0f6dfb2: reuse gate asserts the head ref (workbench.go:230), the doc records the ref and the head-only decision, and TestPrepareRestoresHeadRefOnReuse covers the gate's rejection branch. Findings are scoped to the assigned file internal/workbench/workbench.go; workbench_test.go and docs/checkout-native-review-contract.md also changed and were read for context only. Sandbox denied go test and git execution from this workspace, so the diff was judged by code reading rather than by running the suite. The comment at workbench.go:225-229 claims a workbench with no refs/ is already rejected by commitPresent/verifyClean. Consistent with the PR's own findings and the author's reproduction attempt, but not independently verified here since git could not be run.
Inspected files (3)
  • docs/checkout-native-review-contract.md
  • internal/workbench/workbench.go
  • internal/workbench/workbench_test.go

0 PR discussion threads considered. 0 summarized; 0 resolved.


Completed in 1m 39s | $3.42 | claude-sonnet-5, claude-opus-5 | cr dev
Field Value
Model claude-sonnet-5, claude-opus-5
Reviewers go:implementation-tests, harness-engineering:repo-health, security:code-auditor, architecture:solid
Engine claude_cli · claude-sonnet-5, claude-opus-5
Reviewed by cr · piekstra-dev
Duration 1m 39s wall · 2m 25s compute
Cost $3.42
Tokens 52 in / 8.6k out

Per-workstream usage

  • go:implementation-tests — claude-sonnet-5
    • In: 12
    • Out: 2.0k
    • Cache read: 532.4k
    • Cache create: 87.5k
    • Cost: $0.71
    • Duration: 28s
  • harness-engineering:repo-health — claude-sonnet-5
    • In: 16
    • Out: 2.6k
    • Cache read: 688.2k
    • Cache create: 96.8k
    • Cost: $0.83
    • Duration: 38s
  • security:code-auditor — claude-sonnet-5
    • In: 6
    • Out: 743
    • Cache read: 144.9k
    • Cache create: 48.1k
    • Cost: $0.34
    • Duration: 12s
  • architecture:solid — claude-opus-5
    • In: 12
    • Out: 2.8k
    • Cache read: 502.9k
    • Cache create: 92.3k
    • Cost: $1.24
    • Duration: 54s
  • orchestrator-rollup — claude-sonnet-5
    • In: 6
    • Out: 537
    • Cache read: 125.0k
    • Cache create: 40.3k
    • Cost: $0.29
    • Duration: 11s

The previous commit added the reuse-path check without a test and said so,
because the test I wrote passed with the change reverted. Review pointed out
the reason, and it was a flaw in my assertion rather than in the scenario: I
was asserting the reused workbench is clonable, which does not discriminate,
since deleting only the ref leaves refs/ in place and git still clones from it
happily.

The discriminating assertion is that the ref comes back. Without the gate,
Prepare takes the reuse path and the ref stays deleted; with it, the workbench
is rebuilt and the ref resolves to the head SHA again. Confirmed red with the
gate removed and green with it.

Comment updated to describe what the check actually catches -- refs/ present
but the head ref absent, which every other reuse check accepts -- rather than
hedging about a scenario I had not managed to construct.
@piekstra
piekstra force-pushed the fix/workbench-refs branch from b490d78 to 4cfc86c Compare August 13, 2026 19:30

@piekstra-dev piekstra-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR Review

Reviewed commit: 4cfc86cd4b9a
Profile: reviewer - Posting as: piekstra-dev

Summary

Reviewer Findings
go:implementation-tests 0
harness-engineering:repo-health 0
security:code-auditor 0
architecture:solid 0

Reviewer Coverage

  • go:implementation-tests — complete (constrained); inspected 2 assigned files (3 inspected across reviewers): internal/workbench/workbench.go, internal/workbench/workbench_test.go; skipped: none; constraints: none
  • harness-engineering:repo-health — complete (constrained); skipped: none; constraints: none
  • security:code-auditor — complete (constrained); inspected 1 assigned file (3 inspected across reviewers): internal/workbench/workbench.go; skipped: none; constraints: none
  • architecture:solid — complete (constrained); inspected 1 assigned file (3 inspected across reviewers): internal/workbench/workbench.go; skipped: none; constraints: All three prior findings remain resolved: build path sets the ref (workbench.go:154), the reuse gate asserts it (workbench.go:230), TestPrepareRestoresHeadRefOnReuse covers the rejection branch, and the contract doc records the head-only ref decision. Findings are scoped to the assigned file internal/workbench/workbench.go; workbench_test.go and docs/checkout-native-review-contract.md also changed and were read for context only. Sandbox denied go test and git execution from this workspace, so the diff was judged by code reading rather than by running the suite. The comment at workbench.go:225-229 claims a workbench with no refs/ is already rejected by commitPresent/verifyClean. Plausible and author-verified, but not independently confirmed here since git could not be run. internal/workbench/workbench.go is unchanged at 4cfc86c relative to the previously reviewed 0f6dfb2 (still +31 -2); only the test file grew, refactoring the clone assertion into cloneWorkbench/gitCommandSucceeds helpers.
Inspected files (3)
  • docs/checkout-native-review-contract.md
  • internal/workbench/workbench.go
  • internal/workbench/workbench_test.go

0 PR discussion threads considered. 0 summarized; 0 resolved.


Completed in 1m 23s | $3.83 | claude-sonnet-5, claude-opus-5 | cr dev
Field Value
Model claude-sonnet-5, claude-opus-5
Reviewers go:implementation-tests, harness-engineering:repo-health, security:code-auditor, architecture:solid
Engine claude_cli · claude-sonnet-5, claude-opus-5
Reviewed by cr · piekstra-dev
Duration 1m 23s wall · 2m 00s compute
Cost $3.83
Tokens 48 in / 7.1k out

Per-workstream usage

  • go:implementation-tests — claude-sonnet-5
    • In: 12
    • Out: 1.9k
    • Cache read: 593.3k
    • Cache create: 100.1k
    • Cost: $0.81
    • Duration: 29s
  • harness-engineering:repo-health — claude-sonnet-5
    • In: 12
    • Out: 2.1k
    • Cache read: 645.7k
    • Cache create: 109.9k
    • Cost: $0.88
    • Duration: 30s
  • security:code-auditor — claude-sonnet-5
    • In: 6
    • Out: 668
    • Cache read: 162.6k
    • Cache create: 56.8k
    • Cost: $0.40
    • Duration: 11s
  • architecture:solid — claude-opus-5
    • In: 12
    • Out: 2.0k
    • Cache read: 593.4k
    • Cache create: 108.2k
    • Cost: $1.43
    • Duration: 38s
  • orchestrator-rollup — claude-sonnet-5
    • In: 6
    • Out: 443
    • Cache read: 129.1k
    • Cache create: 43.3k
    • Cost: $0.31
    • Duration: 10s

@piekstra
piekstra merged commit a225189 into main Aug 13, 2026
10 checks passed
@piekstra
piekstra deleted the fix/workbench-refs branch August 13, 2026 19:35
piekstra added a commit that referenced this pull request Aug 13, 2026
…566)

## The problem

Zero findings and "did not run" are the same number and opposite
meanings. One says the code was examined and is clean; the other says
nothing was examined. The summary table rendered both as `0`.

On a real PR, this is what a run looked like where **four of five
reviewers failed to start**:

```
| Reviewer                  | Findings |
| security:code-auditor     | 0        |
| rust:implementation-tests | 0        |
| architecture:solid        | 0        |
| policies:conventions      | 0        |
| documentation:docs        | 1        |
```

Only `documentation:docs` actually ran — it is the one agent that needs
no repo checkout. The failures *were* recorded, but as
`incomplete_failed` in a coverage section well below the headline table.
The summary is the part that gets read, pasted, and acted on, and it
said all-clear.

I nearly merged on that reading, twice.

## Scope

**The outcome logic is already correct** —
`hasIncompleteReviewerCoverage` coerces approval away when coverage is
incomplete, so the PR was never actually approved. This change is purely
about what a human reads.

## The fix

A reviewer whose coverage status did not produce a result renders as `⚠️
did not run` instead of a count, reusing the existing
`coverageResultProduced` helper.

Reviewers **absent** from coverage keep their number, so an unknown
status cannot turn a genuine zero into a phantom failure. That case has
its own test.

## Tests

Two tests, both confirmed to **fail with the change reverted**:

- a failed reviewer is not rendered as `0`, while a genuinely clean
reviewer keeps its honest `0` in the same table
- unknown coverage leaves the count untouched

Full suite green.

## Related

[#565](#565)
fixes the *cause* of that particular fleet failure (the workbench has no
`refs/`, so per-reviewer clones fail). This one fixes the reporting,
which matters for any reviewer failure regardless of cause — the two are
independent.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants