Skip to content

fix(code-review): stop a docs-only BHA skip from blocking (ISS-10869) - #206

Merged
peterulsteen merged 4 commits into
mainfrom
fix/iss-10869-docs-only-bha-skip
Oct 1, 2026
Merged

peterulsteen merged 4 commits into
mainfrom
fix/iss-10869-docs-only-bha-skip

Conversation

@peterulsteen

Copy link
Copy Markdown
Contributor

Summary

On a docs-only diff, arbitrate-budget sets budget.bha_partitions to 0 on purpose, because there is no code for Bug Hunter A to read. derive-spawn-spec then recorded bug_hunter_a as skipped with reason: "budget_capped". bug_hunter_a is a required reviewer, so a Required reviewer dropped: bug_hunter_a coverage gap followed. It said the spawn budget was capped and told the operator to raise --cap, and the verdict came out CHANGES_REQUESTED. Measured on symphony-alpha PR #7712, a single .md file.

  • arbitrate-budget writes budget.docs_only: true into coverage.json.final when it waives the BHA floor. This happens on the arbitrated path and on the blocked_by_verify path. Other diffs don't get the key, so golden fixtures don't change.
  • derive-spawn-spec records a zero cap that carries the marker as skipped[].reason: "docs_only". That reason is benign, like no_partitions, so no coverage gap is emitted and the verdict no longer blocks.
  • A zero cap without the marker still records budget_capped and still emits the HIGH gap. The pre-fix BLOCKING path once hard-coded 0 on code PRs, which is that case. Partitions the budget can't fit also still produce the gap.
  • docs_only is added to SPAWN_SPEC_SKIP_REASONS, and SCHEMA.md and start.md list it with the other benign skip reasons.
  • code-review goes 3.10.2 → 3.10.3, with a CHANGELOG entry.

Affected: plugins/code-review/tools/python/{code_review_helpers,code_review_schema,test_code_review_helpers}.py, plugins/code-review/{SCHEMA.md,commands/start.md,.claude-plugin/plugin.json}, CHANGELOG.md.

Test plan

  • Reproduced on main (ed49565, v3.10.2) by driving arbitrate-budget → derive-spawn-spec → finalize-result → verdict on a docs-only diff: budget_capped, one HIGH gap, CHANGES_REQUESTED. With the fix the same chain gives reason docs_only, no gap, and APPROVED.
  • New TestISS10869DocsOnlyBhaSkip runs that chain for three diffs: docs-only (no gap, APPROVED), mixed docs+code (BHA floor applies, bha_p0 spawns), and cap pressure (3 partitions against a budget of 1: two budget_capped HIGH gaps, CHANGES_REQUESTED). It also checks that a zero cap without the marker still gives the HIGH gap.
  • Counterfactuals. Each of these mutations turns its test red, and the tests pass again once restored:
    • reverting the fix
    • dropping docs_only from the benign set
    • ignoring the marker
    • treating every zero as docs-only
    • any instead of all docs files
    • treating budget_capped as benign
  • uv run pytest plugins/code-review/ at bcd0519: 1401 passed, 3 skipped. uv run ruff check . and uv run pyright are clean, and git diff --check is clean.
  • Two review passes, a review-soul critic and a Python correctness reviewer, found nothing BLOCKING or HIGH. Stale comments claiming docs-only diffs produce no_partitions are fixed in bcd0519: the partitioner keeps docs files.
  • Not changed here: _is_docs_only counts every .md and .txt file as docs, including requirements.txt and CMakeLists.txt. That rule already decided when BHA gets zero partitions. After this PR, the zero also no longer raises the required-reviewer gap. Narrowing the rule is a separate decision.

Closes ISS-10869.

🤖 Generated with Claude Code

peterulsteen and others added 2 commits September 29, 2026 05:54
- arbitrate-budget writes budget.docs_only: true when it waives the
  BHA floor, so the zero bha_partitions carries its cause
- derive-spawn-spec records that zero as skipped reason "docs_only",
  a benign skip that emits no coverage-gap finding
- A zero cap without the marker, and partitions beyond
  bha_partitions, still record budget_capped and the required gap
- docs_only added to SPAWN_SPEC_SKIP_REASONS, SCHEMA.md, start.md
- Bump code-review to 3.10.3

Testing: pytest plugins/code-review (1401 passed, 3 skipped); ruff
and pyright clean; arbitrate -> derive-spawn-spec -> finalize-result
-> verdict chain run for docs-only, mixed and cap-pressure diffs

Risks: None identified; the budget key is additive and absent
unless the diff is docs-only

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The partitioner keeps docs files, so a docs-only diff has
  partitions; no_partitions means every file was cached
- Rename the zero-cap fleet-summary test, which no longer
  describes the docs-only case

Testing: ruff clean; pytest plugins/code-review

Risks: None identified; comments and a test name only

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@mikeangstadt mikeangstadt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right fix for the right reason. The marker-on-the-budget approach beats re-deriving docs-only in derive-spawn-spec, the counterfactual set is real, and SCHEMA.md + start.md + the schema constant all moved together. One thing about operator visibility, see the comment.

Comment thread plugins/code-review/tools/python/code_review_helpers.py
peterulsteen and others added 2 commits October 1, 2026 17:05
Resolve the CHANGELOG.md conflict by keeping main's code v1.16.0-v1.16.3
entries and placing this branch's code-review v3.10.3 entry above them
(newest-first). main did not bump code-review (still 3.10.2), so this
branch's single 3.10.2 -> 3.10.3 bump stands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- _render_fleet_notes adds an informational "BHA skipped on a
  docs-only diff." note when spawn.json.spec.skipped[] holds a
  docs_only entry; before this the fleet summary said nothing
- The docs_only skipped entry carries budget_cap: 0 and
  partition_count, as the zero-cap budget_capped entry does
- SCHEMA.md and CHANGELOG.md (code-review v3.10.3) updated

Testing: TestISS10869DocsOnlyBhaSkip docs-only case asserts the new
fields and renders the fleet summary; removing the note line or the
two fields turns it red. uv run pytest plugins/: 2179 passed,
3 skipped. ruff and pyright clean.

Risks: None identified

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@peterulsteen
peterulsteen merged commit 3e89492 into main Oct 1, 2026
7 checks passed
@peterulsteen
peterulsteen deleted the fix/iss-10869-docs-only-bha-skip branch October 1, 2026 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants