Skip to content

feat(ai-review): add incremental reviews and dry-run benchmarks - #15

Merged
absorbb merged 1 commit into
mainfrom
feat/ai-review-incremental
Oct 16, 2026
Merged

absorbb merged 1 commit into
mainfrom
feat/ai-review-incremental

Conversation

@absorbb

@absorbb absorbb commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Follow-up pushes currently re-review the entire PR and execution settings are hardcoded. Select a compatible previously reviewed ancestor from trusted publisher coverage markers, and expose explicit model/effort/timeout/CLI inputs while preserving the production defaults.

Coverage requires the configured bot identity, matching commit_id, the same full PR merge base, and a profile hash of the prompt, settings, additional focus, and PR title/body. Missing or incompatible coverage, rewrites, and same-head reruns use the full diff. Only validated publication records coverage; dry runs and benchmarks never advance it.

Intentional decisions and tradeoffs

  • Default model/effort/CLI/action pins remain Terra/high/0.149.0/current action commit. Floating CLI versions are rejected; timeouts are bounded to 1–30 minutes per engine.
  • Incremental LEFT locations refer to the previous head, not GitHub's original PR base. Those findings and restored lines absent from the overall PR diff go in the review summary with locations, rather than incorrect inline threads. Oversized summaries are split; only the final accepted review records coverage, and retries deduplicate earlier parts.
  • dry_run validates and archives without minting an App token or posting. benchmark_native implies this behavior and additionally runs native review from a clean worktree and separate Codex home.
  • The optional native run uses the same selected range/model/effort/CLI but built-in guidance and repository rules instead of the custom prompt/PR context. It compares execution setups, not prompt wording alone. Results/events remain artifacts.
  • Default bot identity is jitsu-code-review[bot]; custom callers must configure their actual posting App identity. Plain APPROVE reviews do not count as coverage.
  • Primary usage is collected before the benchmark. Unknown-model pricing is left unavailable instead of reusing Terra rates.

Validation

  • python3 -m unittest discover -s tests -v: 33 tests passed, including fixup selection, rewrite/base/profile fallbacks, marker trust, explicit full review, API failures, incremental summary coordinates, split/retried publication, execution bounds, pricing, and benchmark invocation.
  • actionlint 1.7.7 and git diff --check: passed.
  • Actual CLI 0.149.0 and candidate 0.163.0 help confirm native review options. Pinned action source confirms its API proxy remains available to the optional benchmark.
  • Parallel local finders, independent verification, and final fresh review before push.

The README documents all inputs and a non-publishing newer-CLI benchmark command. Live workflow and candidate benchmark results will be checked on this PR.

@jitsu-code-review jitsu-code-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of 8feeb26eeb4bc5e5d4163631745699659413b7c2

Completed review of the supplied full range; no high-confidence actionable defects found.

@absorbb
absorbb merged commit 3389b5c into main Oct 16, 2026
4 checks passed
@absorbb
absorbb deleted the feat/ai-review-incremental branch October 16, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant