Keep Greptile's pause notice out of head coverage, and its notices out of claims by ejc3 · Pull Request #926 · ejc3/fcvm

ejc3 · 2026-09-13T16:33:33Z

Installing Greptile on this repo can no longer mark an unreviewed head as reviewed, and its notices and triggers stop reading as findings.

Contract

A Greptile pause notice never covers a head, whoever posts it.
Greptile's pause review and its  notices, from the Greptile bot, need no disposition. The same text with anything added, or from another account, stays claimable.
@greptileai and @greptile-apps, alone or followed by review, are triggers. Any other comment that mentions Greptile is a comment like any other.
.greptile/config.json keeps Greptile's findings where the gate reads them.

Greptile is not added to VERDICT_BOTS, so it grants no no-findings coverage and its summary comment stays claimable. Binding a clean Greptile result to the head through its "Greptile Review" check run is a follow-up PR stacked on this one.

The hole

Out of open-source review credits, Greptile posts a COMMENTED review on the head whose whole body is:

Greptile has paused reviews on this repository — it used its 2000 free open-source review credits for this billing period. Reviews resume automatically on September 19. ...

(manaflow-ai/cmux pr10764, review 5025193699.) The gate counts any non-empty review body as coverage of its commit. Once a disposition answered that notice, the gate reported CLEAR for a head nobody reviewed. On the unfixed gate, the case a greptile pause review on the head, answered, is still an unreviewed head got rc=0.

Changes

scripts/check-review-threads.sh
- covers rejects a body that is Greptile's pause notice.
- Both review reads fetch author.__typename, so only Bot greptile-apps is exempt from answering the pause notice.
- New whole-body classifiers for the pause notice and the three observed status notices: the file limit in both wordings (pr12342, pr7670) and the error (pr6705). They use only the normalizations gate-discard-sites.sh lists.
- TRIGGER_RE adds the Greptile trigger forms.
.greptile/config.json: shouldUpdateDescription: false and updateSummaryOnly: false (the gate never reads the PR description), statusCheck: true (a check run, not a status comment the gate has no shape for), autoApprove.enabled: false (an APPROVED review covers the head), and triggerOnUpdates: false (each review spends a free open-source credit, so ask with @greptileai).
AGENTS.md names the Greptile notices, the trigger and the config.

Tests

All written against the unfixed gate, in both harnesses:

scripts/test-check-review-threads.sh, finding 47: 11 cases red before the fix, and 7 guards green before and after (appended text, a human author, the wrong bot, a question for Greptile, no coverage from a notice).
tests/test_log_scan.rs: greptile_pause_notice_covers_nothing_and_its_notices_are_not_findings and greptile_config_keeps_its_results_where_the_gate_reads_them.

shell harness, fix applied:                   passed=292 failed=0
shell harness, fix reverted:                  passed=281 failed=11
rust Greptile test, fix applied:              1 test run: 1 passed
rust Greptile test, fix reverted:             1 test run: 0 passed, 1 failed
config pin, no .greptile/config.json:         1 test run: 0 passed, 1 failed
config pin, with it:                          1 test run: 1 passed
config pin, shouldUpdateDescription flipped:  1 test run: 0 passed, 1 failed
test_log_scan, whole binary:                  30 tests run: 30 passed
gate-discard-sites.sh:                        28 discarding call(s) in VERDICT_JQ, 0 unaccounted

Fixture bodies are byte-identical to what greptile-apps[bot] posted on manaflow-ai/cmux, converted to the GraphQL shape the gate fetches.

Downstream impact: the merge gate. Every change narrows what counts as coverage, or exempts one observed notice or trigger shape from the Greptile bot account.

Summary by CodeRabbit

New Features
- Added support for Greptile review notices and trigger comments in review status handling.
- Greptile notices about review pauses, file limits, or errors no longer count as completed reviews or actionable findings.
- Configured Greptile results to appear in status checks and review threads, with reviews triggered on pull request opening or explicit requests.
Bug Fixes
- Improved handling of Greptile notices so genuine findings and questions remain actionable.

@greptile-apps

…t of claims Greptile, out of open-source review credits, posts a COMMENTED review on the head whose whole body says reviews are paused (manaflow-ai/cmux pr10764, review 5025193699). The gate counts any non-empty review body as coverage of its commit, so once a disposition answered that notice the head read as reviewed: `a greptile pause review on the head, answered` reported CLEAR (rc=0) on the unfixed gate. - covers: a body that is Greptile's pause notice covers nothing, whoever posts it. - The pause review from Bot greptile-apps needs no disposition. Reviews are now fetched with author.__typename in both reads, so a human account posting the same text stays claimable. - A top-level comment from Bot greptile-apps whose whole body is one of its observed  notices (file limit in both wordings it has used, error; pr12342, pr7670, pr6705) is a notice: no disposition, no coverage. Anything appended, or another account, stays claimable. - TRIGGER_RE accepts @greptileai and @greptile-apps, alone or followed by review. `@greptileai is this thread-safe?` stays claimable. - Greptile is not added to VERDICT_BOTS; its summary comment stays claimable until no-findings coverage is bound to its check run. - .greptile/config.json: findings stay in comments and threads the gate reads (shouldUpdateDescription false, updateSummaryOnly false), results post as a check run (statusCheck true), auto-approval off, and pushes do not trigger reviews (triggerOnUpdates false; each review spends a free open-source credit). - AGENTS.md names the Greptile notices, the trigger and the config. Tests, written against the unfixed gate: - scripts/test-check-review-threads.sh, finding 47: 11 cases red before the fix, 7 guards green before and after. - tests/test_log_scan.rs: greptile_pause_notice_covers_nothing_and_its_notices_are_not_findings, greptile_config_keeps_its_results_where_the_gate_reads_them. Shell harness, fix applied: passed=292 failed=0 Shell harness, fix reverted: passed=281 failed=11 Rust Greptile test, fix applied: 1 test run: 1 passed Rust Greptile test, fix reverted: 1 test run: 0 passed, 1 failed scripts/gate-discard-sites.sh: 28 discarding call(s) in VERDICT_JQ, 0 unaccounted Config pin, no .greptile/config.json: 1 test run: 0 passed, 1 failed Config pin, with it: 1 test run: 1 passed Config pin, shouldUpdateDescription flipped: 1 test run: 0 passed, 1 failed test_log_scan, whole binary: 30 tests run: 30 passed

ejc3 · 2026-09-13T16:33:35Z

@codex review

chatgpt-codex-connector · 2026-09-13T16:33:37Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

coderabbitai · 2026-09-13T16:33:41Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 0819bed5-8671-4fd8-a74c-50245636c4c4

📥 Commits

Reviewing files that changed from the base of the PR and between 92ced2a and 8913994.

📒 Files selected for processing (5)

.claude/CLAUDE.md
.greptile/config.json
scripts/check-review-threads.sh
scripts/test-check-review-threads.sh
tests/test_log_scan.rs

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

Walkthrough

The change adds Greptile configuration and updates the review gate to recognize Greptile bots, triggers, status notices, and pause notices. Greptile pause notices no longer provide head coverage. Tests validate classification, coverage, claims, and configuration.

Changes

Greptile review-gate integration

Layer / File(s)	Summary
Greptile configuration and notice contract `.claude/CLAUDE.md`, `.greptile/config.json`	The documentation defines Greptile notice handling and trigger commands. The configuration enables status checks, disables description and summary-only updates, disables update triggers, and keeps auto-approval disabled.
Payload classification and coverage rules `scripts/check-review-threads.sh`	The gate reads review author types, recognizes Greptile triggers and status notices, classifies matching pause notices as non-claimable, and excludes them from head coverage.
Behavior validation `scripts/test-check-review-threads.sh`, `tests/test_log_scan.rs`	Tests cover Greptile pause, status, error, trigger, claim, coverage, and configuration behavior.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant GitHubAPI
  participant CheckReviewThreads
  participant ReviewGate
  GitHubAPI->>CheckReviewThreads: return reviews and comments
  CheckReviewThreads->>CheckReviewThreads: classify Greptile notices and triggers
  CheckReviewThreads->>ReviewGate: pass claims and coverage
  ReviewGate->>ReviewGate: evaluate review status

Merge Risk: ⚪ Minimal · up to 89139

The Greptile notice and coverage behavior matches the documented, tested contract. No actionable merge risk remains.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely summarizes the main changes: Greptile pause notices no longer count as head coverage, and Greptile notices no longer count as claims.
Docstring Coverage	✅ Passed	Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 3 files. (2 skipped: 2 u…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

Create stacked PR
Commit on current branch

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/greptile-review-gate

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

chatgpt-codex-connector · 2026-09-13T16:33:43Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

ejc3 · 2026-09-13T16:42:41Z

@greptileai

ejc3

NOT-A-DEFECT: the @greptileai comment on this PR is a trigger command asking Greptile to review this head, not a finding. This PR is the change that teaches the gate to exempt it.

ejc3 · 2026-09-13T21:17:14Z

@coderabbitai full review

coderabbitai · 2026-09-13T21:17:19Z

✅ Action performed

Full review finished.

ejc3

NOT-A-DEFECT: CodeRabbit's walkthrough is the summary of its review of 89139943, which generated no actionable comments, so it carries no finding to answer. Its pre-merge check notes, such as docstring coverage, are documentation thresholds, not defect claims.

ejc3 commented Sep 13, 2026

View reviewed changes

ejc3 mentioned this pull request Sep 13, 2026

Count a clean Greptile review as head coverage, bound through its check run #927

Merged

ejc3 closed this Sep 13, 2026

ejc3 reopened this Sep 13, 2026

ejc3 commented Sep 13, 2026

View reviewed changes

ejc3 merged commit 034fadd into main Sep 13, 2026
38 of 49 checks passed

ejc3 deleted the feat/greptile-review-gate branch September 13, 2026 23:31

Navigation Menu

Keep Greptile's pause notice out of head coverage, and its notices out of claims - #926

Keep Greptile's pause notice out of head coverage, and its notices out of claims#926
ejc3 merged 1 commit into
mainfrom
feat/greptile-review-gate

ejc3 commented Sep 13, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

ejc3 commented Sep 13, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 13, 2026

Uh oh!

coderabbitai Bot commented Sep 13, 2026 •

edited

Loading

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

chatgpt-codex-connector Bot commented Sep 13, 2026

Uh oh!

ejc3 commented Sep 13, 2026

Uh oh!

ejc3 left a comment

Uh oh!

ejc3 commented Sep 13, 2026

Uh oh!

coderabbitai Bot commented Sep 13, 2026 •

edited

Loading

Uh oh!

ejc3 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ejc3 commented Sep 13, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contract

The hole

Changes

Tests

Summary by CodeRabbit

Uh oh!

ejc3 commented Sep 13, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 13, 2026

Uh oh!

coderabbitai Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

chatgpt-codex-connector Bot commented Sep 13, 2026

Uh oh!

ejc3 commented Sep 13, 2026

Uh oh!

ejc3 left a comment

Choose a reason for hiding this comment

Uh oh!

ejc3 commented Sep 13, 2026

Uh oh!

coderabbitai Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ejc3 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ejc3 commented Sep 13, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Sep 13, 2026 •

edited

Loading

coderabbitai Bot commented Sep 13, 2026 •

edited

Loading