Repository navigation
ci(grok): place findings inline and tighten the review - #420
Merged
Merged
Conversation
A Grok run cancelled by a new push, or stopped by the timeout, left its earlier change request standing, so the PR showed as blocked until a later run finished and dismissed it. Every review is now a comment. Each inline finding still opens a review thread, and the ruleset requires threads to be resolved before merge, so findings still gate the merge. The step that dismissed earlier change requests has nothing left to dismiss and is removed. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Grok flagged std::fs::File::lock as unstable five times on one PR, although the CI toolchain ships it as stable. Its recall of Rust lags the toolchain. The workflow installs the stable toolchain CI tests with and names its version in the prompt, with the instruction to treat every API it ships as stable. The prompt also asks for one finding per root cause, so one mistake no longer opens a thread at each site. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
A P0 finding breaks every input, so it blocks the merge as a change request. Lower priorities stay comments gated by their review threads. The step that dismisses the bot's earlier change requests returns, so a review of a later commit replaces a P0 block. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ba5ecff4b4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
⚡ Grok Review
Reviewed commit: 366976bc2f
❌ Patch is incorrect: 1 finding.
The payload still requests changes for a priority-0 finding, so a cancelled or timed-out run can leave a stale change request that blocks the merge.
Stop requesting changes for priority-0 findings (
.github/grok-review-payload.jq#L38)
AbysmalBiscuit
changed the title
GitHub rejects a whole review when one comment falls outside the diff. Grok cited lines past a hunk edge or outside any hunk on 10 of 40 recent runs, so every finding on those runs fell back to the review body, with no thread and no suggestion. The log kept only "HTTP 422", not the reason. Each finding now carries a quote of its cited line. The post step moves the finding to the nearest line holding that quote, trims its range to the last hunk it overlaps, and otherwise anchors it at the file's nearest diff line with a link to its real lines. Only a finding whose quote is in no tracked file, or whose file is not in the diff, stays in the body. A failed post prints GitHub's reason. The review object is read from .structuredOutput first. Earlier change requests are dismissed only after the new review posts, so a failed post leaves the old block in place. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Grok runs with XAI_API_KEY in its environment and can read files. A prompt injection could have it quote the key in a finding, and Actions masks secrets only in logs, not in a review posted through the API. The post step now fails instead of posting a payload that contains the key. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Authors fixed about one in six Grok findings, against about two in three for the Codex connector. Grok also paged pr.diff through read_file 1000 lines at a time, which cost large PRs many slow turns. The static instructions move to .github/grok-review-prompt.md. They take the eight criteria of Codex's review rubric, a false-positive list after Anthropic's code-review plugin, a default of no findings, and the quote field the post step places comments by. The workflow appends the PR, the toolchain, and the diff itself, which falls back to pr.diff only past 1 MB. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Each run reviewed the whole PR from scratch, so a push brought back findings the author had already fixed or answered, often in new words. A step before the review reads the PR's review threads and review bodies over GraphQL and lists this reviewer's earlier findings: each thread with its state and replies, then titles found only in a review body. The prompt tells Grok to report none of them again unless the new commits made the issue worse. Inline comments carry a hidden grok-review marker, so the list still finds them once the review posts through the app identity. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Grok reads .agents/ as a config root, and the PR controls it, yet the step deleted only .grok, .claude, .cursor and .mcp.json before running with --trust. The comment above the step read as "drop --trust" while the command kept the flag, and it said headless dontAsk cancels the turn, which the Grok source does not bear out. It now says why the deletion comes first. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
AbysmalBiscuit
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Agent summary
Grok review lost most of its value on the way to the PR. On 10 of 40 recent runs GitHub rejected the inline review, because one comment cited a line outside the diff, so every finding fell back to the review body with no thread. Authors fixed about one in six of its findings, against about two in three for the Codex connector. It repeated findings on every push, took many slow turns paging
pr.diff, and left stale change requests behind when a run was cancelled..github/grok-review-prompt.md: the eight criteria of Codex's review rubric, a false-positive list after Anthropic's code-review plugin, a default of no findings, and the toolchain CI builds with. The diff goes in the prompt, and earlier findings, with each thread's state and replies, come with an instruction not to repeat them..agents/with the other harness config before--trustloads it, and the comment on that step now says why.Caveats
0 departures
0 loose ends
0 assumptions
Conceptual Changes
None
Proof
.github/grok-review-payload.jqgivesREQUEST_CHANGESfor a P0 finding andCOMMENTfor a P1..github/grok-review-earlier.jqon PR fix: report a missing home instead of panicking #416's threads lists both resolved threads with their replies and the five findings that went only into review bodies.pull_requestruns the workflow from the PR's head.Merge Danger
Door:↔️ two-way
Blast Radius: ⚙️ service
🤖 Generated with Claude Opus 5.5 via Claude Code