feat: add agent reviewer pack for pr-review-loop - #223
Conversation
Root AGENT-REVIEWERS.md indexing 11 reviewers with spawn conditions, severity conventions (P1/P2/P3 mapped to beads), output format, and deferral rules. Specs in .reviewers/: 7 adapted from 521-studios/wyrd (complexity, error-handling, test-coverage, resource-leak, dead-code, logging, importlib-resources) and 4 project-specific (migration- discipline, serverless-architecture, frontend-conventions, credentials-hygiene). Adds *.har to .gitignore since upcoming Thingiverse work uses HAR captures that carry live credentials. Closes openforge_catalog-1v5 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
The catalog frontend is a full SPA, so hook correctness (deps arrays, effect cleanup, fetch races, list rendering at catalog scale) and promise correctness (floating promises, resp.ok, swallowed rejections) get dedicated reviewers rather than riding along in frontend-conventions. Part of openforge_catalog-1v5 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
|
/gemini review |
devonjones
left a comment
There was a problem hiding this comment.
Code Review (Claude)
I've reviewed the changes in this PR. Here's my analysis:
I read every file in full (AGENT-REVIEWERS.md, all 13 .reviewers/*.md, the .gitignore diff, root CLAUDE.md, openforge/CLAUDE.md, src/CLAUDE.md), and cross-checked the pack's factual claims against the actual repo: openforge/db/fixtures/__init__.py, openforge/db/schema/version_12.py and the migration runner, the Postgres connection pattern in openforge/app/, .github/workflows/test.yaml, package.json, next.config.js, and the pr-review-loop skill's discover-agents.sh parser + SKILL.md (to check the pack's format is actually consumable by the tool it's written for).
Critical Issues
-
The complexity-reviewer's stated ruff invocation doesn't run.
.reviewers/complexity-reviewer.md:12:**Objective floor (Python): McCabe complexity > 10.** Run `ruff check --select C901 --max-complexity 10` against the PR head...I ran this exact command against the repo:
ruff check --select C901 --max-complexity 10fails witherror: unexpected argument '--max-complexity' found. Ruff has no--max-complexityCLI flag —mccabe.max-complexityis apyproject.toml/ruff.tomlsetting only, and this repo'spyproject.tomldoesn't set it. The correct command is justruff check --select C901 .(ruff's mccabe default is already 10 — confirmed by running it here, e.g.openforge/app/routes/__init__.py:8: C901 'authenticate' is too complex (15 > 10)). An agent following the spec literally will hit a shell error on its very first review action for every Python PR. This is the reviewer's headline objective-floor check — it needs to work. -
The PR claims to close a ticket that its own diff shows as still open.
Both commit messages and the PR body say "Closesopenforge_catalog-1v5". But the.beads/issues.jsonlentry this PR adds for that exact ticket (.beads/issues.jsonl:1) is:{"id":"openforge_catalog-1v5", ..., "status":"in_progress", ...}— no
close_reason, noclosed_at. Compare to how this repo's own convention marks a ticket done, e.g.openforge_catalog-4jiin the same file:"status":"closed","closed_at":"...","close_reason":"Completed: ...". Either the "Closes" trailer is wrong (the work isn't actually done from beads' point of view) or the ticket update was forgotten. This is a self-contradiction shipped inside the PR's own diff, not a stylistic nit.
Medium Issues
-
AGENT-REVIEWERS.md's "Default severity per reviewer" table doesn't match the explicit per-item tags in several specs it summarizes. Since the workflow does tell spawned agents to read the full.reviewers/<name>.md(so the explicit inline tags should win in practice), this is a documentation-accuracy problem rather than a live bug, but it's the kind of drift the index exists to prevent:credentials-hygiene-reviewer: index says "P1 by default" (unqualified). The spec (.reviewers/credentials-hygiene-reviewer.md) only tags 3 of 7 patterns P1 (items 1, 2, 4); items 3 and 5 are explicitly P2, items 6 and 7 explicitly P3.serverless-architecture-reviewer: index says "P1 for S3-instead-of-R2, persistent-state features, and connection-pattern violations; P2 otherwise." But the spec explicitly tags item 7 ("Frontend assumptions of a server") P1 — not listed in the index's P1 set — and item 8 ("Unbounded Lambda invocation patterns") P3, not P2 as "otherwise" implies.migration-discipline-reviewer: index says "P2 otherwise," but the spec explicitly tags item 4 (missing docstring) and item 7 (silent data migrations) P3.frontend-conventions-reviewer: index says "P2 otherwise," but the spec explicitly tags item 5 (state anti-patterns) and item 7 (utility duplication) P3.js-async-reviewerandreact-hooks-reviewerhave the same pattern (js-async items 5/6/7 and react-hooks item 8 are explicit P3s not captured by the index's "P2 by default" summary).
Net effect: a reader who trusts the index table alone (rather than opening every spec) will over-promote several intentionally-advisory P3 findings to P2/P1. Recommend either fixing the summaries or adding "see spec for exceptions" next to each.
-
The "Output format" template doesn't account for how the consuming skill actually anchors comments.
AGENT-REVIEWERS.md's output template requires every finding to embedFile: path/to/file.py:LINEin the comment body. Butpr-review-loop's actual posting mechanism (scripts/post-line-comment.sh <PR> <file> <line> <agent-name> "<text>", per that skill'sSKILL.md) already takes file and line as separate arguments that anchor the GitHub comment to that exact location. The template's embeddedFile:/line header is therefore redundant with the anchor the harness already provides — harmless, but the spec doesn't acknowledge it, and a reviewer copying the "File:" line from the quoted example verbatim instead of using the real anchor argument risks the two disagreeing.
Minor Issues
-
One-directional overlap acknowledgment.
.reviewers/react-hooks-reviewer.mditem 5 explicitly notes "(Shared withfrontend-conventions-reviewer, post under whichever found it first.)" for derived-state-in-useStatefindings..reviewers/frontend-conventions-reviewer.mditem 5 describes the same pattern ("derived state stored inuseState+ synced withuseEffect") but doesn't reciprocally referencereact-hooks-reviewer. Since these two reviewers' spawn conditions overlap on almost every touched.tsxfile, it'd be worth stating the cross-reference from both sides rather than one. -
PR description reviewer count is stale. The PR body says "
AGENT-REVIEWERS.md— index of 11 reviewers," but the second commit ("add dedicated SPA reviewers") brought the total to 13 (react-hooks-reviewerandjs-async-reviewerwere added after that sentence was written). Not a file in the shipped pack, but worth a quick edit before merge so the PR record isn't misleading to future readers of git history.
Positive Observations
- Every factual claim I could verify against the repo checked out true:
openforge/db/fixtures/__init__.py'simportlib.resourcesusage, thePgDB.connection()pattern reused uniformly acrossopenforge/app/routes/*.py,version_12.py's exact@SchemaVersionDecorator/SchemaBase/up_impl/down_implshape, the version-15 gap, the CI job commands (pytest tests/ --cov=openforge,npm test,npm run lint:all,ruff check .,ruff format --check .) matching.github/workflows/test.yamlverbatim, frontend test locations/commands matchingpackage.json, and the static-export (next.config.js: output: "export") / single-Lambda (aws_lambda_wsgi) architecture claims being current rather than stale. This pack is clearly grounded in the actual codebase, not generic boilerplate. AGENT-REVIEWERS.md's own structure (# AgentsH1 → H2 reviewer names →# Guidelines→# Context) matches thepr-review-loopskill'sdiscover-agents.shparser exactly — this file will actually parse correctly by the tool that's supposed to consume it, which isn't a given for a hand-authored spec file.- Thoughtful, explicit division of labor between overlapping reviewers in most places:
frontend-conventions-reviewerexplicitly defers complexity thresholds tocomplexity-reviewer;serverless-architecture-reviewerexplicitly defers leak shapes toresource-leak-reviewerwhile keeping acquisition-pattern violations for itself;credentials-hygiene-reviewerexplicitly defers log-interpolation leaks tologging-reviewer. - Strong "Do NOT flag" sections throughout (e.g.
dead-code-reviewercorrectly excludes Flask route decorators, pytest fixtures, and Next.js reserved exports — the exact false-positive traps naive dead-code tools fall into) show real familiarity with this codebase's idioms, not copy-pasted generic advice. - The
.gitignoreaddition (*.har) is correctly justified and timed — it lands ahead of the Thingiverse HAR-capture work described in the linked beads epic, exactly the kind of "prerequisite" hygiene this PR claims to deliver.
Recommendations Summary
Must fix before merge:
- Fix the broken
ruff check --select C901 --max-complexity 10command in.reviewers/complexity-reviewer.md:12— drop--max-complexity 10(ruff's default is already 10) or move it topyproject.tomlconfig if a different threshold is ever wanted. - Resolve the
openforge_catalog-1v5status contradiction — either update the beads entry toclosedwith aclose_reason, or remove the "Closes" claims from the commit messages/PR body if the ticket is intentionally left open for follow-up work.
Should fix:
3. Correct the "Default severity per reviewer" summaries in AGENT-REVIEWERS.md (or annotate them as approximations) so they match the explicit P1/P2/P3 tags actually present in credentials-hygiene-reviewer.md, serverless-architecture-reviewer.md, migration-discipline-reviewer.md, frontend-conventions-reviewer.md, js-async-reviewer.md, and react-hooks-reviewer.md.
4. Clarify that the "Output format" template's File:/line header is descriptive only — the actual comment anchor comes from post-line-comment.sh's <file> <line> arguments, not from text in the body.
Nice to have:
5. Make the react-hooks-reviewer ↔ frontend-conventions-reviewer overlap acknowledgment bidirectional.
6. Update the PR description's "11 reviewers" to 13 before merge.
Overall Assessment
This is a well-researched, codebase-grounded reviewer pack — nearly every factual claim I checked against the repo held up, and the reviewer-to-reviewer scoping shows real thought about avoiding duplicate ownership. The two concrete defects (a non-functional ruff invocation that's the complexity-reviewer's core check, and a "Closes" claim contradicted by the ticket's own shipped status) are both easy, mechanical fixes; once those and the severity-table drift are addressed, this is ready to merge.
Response to Claude ReviewCritical 1 (invalid ruff command in complexity-reviewer.md): Fixed — command corrected to Critical 2 ("Closes openforge_catalog-1v5" vs ticket in_progress): Fixed in process — PR body now says the ticket is closed via Medium 1 (severity table vs per-item spec tags): Fixed — added an explicit precedence rule to the Guidelines: a spec's per-item priority tag wins over the index table; the table gives defaults, not ceilings/floors. Medium 2 (File:/LINE in body redundant with post-line-comment.sh anchoring): Fixed via clarification — kept the Minor 1 (one-directional overlap acknowledgment): Fixed — Minor 2 (stale "11 reviewers" count in PR body): Fixed — body updated to 13 and now lists the SPA reviewers. |
- correct ruff C901 invocation (max-complexity is pyproject config, not a CLI flag; ruff default is already 10) - add severity precedence rule: spec per-item tags win over the index defaults table - document why File: stays in the finding body despite post-line-comment.sh anchoring - make hooks/frontend-conventions overlap acknowledgment mutual - broaden capture hygiene: ignored captures/ dir for raw traffic dumps of any extension (credentials-hygiene-reviewer finding) Part of openforge_catalog-1v5 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Verification Review (Claude)Verified all 6 round-1 findings fixed at 85ada01 (
No new issues introduced by the fix commit — all four changed files ( |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- annotate the skip rule (general-reviewer always matches by construction) - correct ruff-flag catch attribution to PR #223 in ticket aq5 Part of openforge_catalog-aq5 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
* feat: formalize general-reviewer in the agent pack This repo has no Gemini/Cursor, so the pack gains a whole-PR generalist that spawns on every PR, every round: logic correctness verified empirically, design soundness against upcoming tickets, contract fidelity, cross-cutting interactions, docs accuracy. Explicitly non-overlapping with the specialists. Previously run ad-hoc in PRs #224-#226, where this pass produced the highest-value catches (invalid ruff flag, JWT AttributeError, multi-accept AND-vs-ANY). Also documents the no-external-bots reality so future loops don't poll for Gemini. Closes openforge_catalog-aq5 (via bd close on merge) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: address general-reviewer dogfood findings - annotate the skip rule (general-reviewer always matches by construction) - correct ruff-flag catch attribution to PR #223 in ticket aq5 Part of openforge_catalog-aq5 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
Summary
Creates the agent reviewer pack consumed by the
pr-review-loopskill, modeled on 521-studios/wyrd's pack. This is the prerequisite for running the review loop on the upcoming Thingiverse publish/sync epic PRs (openforge_catalog-4kx).AGENT-REVIEWERS.md— index of 13 reviewers with what-it-checks / when-to-spawn lines, per-reviewer file scopes, severity conventions (P1/P2/P3 mapped to beads deferral rules), structured output format, and branch-targeting guidance (test, nevermain)..reviewers/*.md— full per-reviewer specs:complexity(extended to TS/React, wired to CLAUDE.md's And/Or test + one-screen rule),error-handling,test-coverage(pytest + Jest),resource-leak(serverless Postgres connection cap = P1),dead-code,logging,importlib-resources(Lambda zip).migration-discipline(version_NN.py system),serverless-architecture(Lambda size, R2-not-S3, no persistent state, N+1),frontend-conventions(relative/apirule, static-export constraints, hooks),credentials-hygiene(secrets/JWTs/HAR captures — timely for the Thingiverse work).react-hooks(deps, cleanup, fetch races, catalog-scale list rendering),js-async(floating promises,resp.ok, swallowed rejections)..gitignore— adds*.harand an ignoredcaptures/directory (traffic captures carry live credentials).Test plan
Beads:
openforge_catalog-1v5(closed viabd closewhen this merges — GitHub's Closes keyword can't close beads tickets).🤖 Generated with Claude Code