Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 7 additions & 18 deletions .github/workflows/pr-review-by-openhands.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,13 @@
name: PR Review by OpenHands

on:
# Use pull_request_target to allow fork PRs to access secrets when triggered by maintainers
# Security: This workflow runs when:
# 1. A new PR is opened (non-draft), OR
# 2. A draft PR is marked as ready for review, OR
# 3. A maintainer adds the 'review-this' label, OR
# 4. A maintainer requests openhands-agent or all-hands-bot as a reviewer
# Use pull_request_target so maintainers can trigger reviews on fork PRs.
# Security: require an explicit maintainer action before running:
# 1. A maintainer adds the 'review-this' label, OR
# 2. A maintainer requests openhands-agent or all-hands-bot as a reviewer
# Adding labels and requesting reviewers requires write access.
# The PR code is explicitly checked out for review, but secrets are only accessible
# because the workflow runs in the base repository context.
pull_request_target:
types: [opened, ready_for_review, labeled, review_requested]
types: [labeled, review_requested]

permissions:
contents: read
Expand All @@ -21,15 +17,8 @@ permissions:

jobs:
pr-review:
# Run when one of the following conditions is met:
# 1. A new non-draft PR is opened by a non-first-time contributor, OR
# 2. A draft PR is converted to ready for review by a non-first-time contributor, OR
# 3. 'review-this' label is added, OR
# 4. openhands-agent or all-hands-bot is requested as a reviewer
# Note: FIRST_TIME_CONTRIBUTOR and NONE PRs require manual trigger via label/reviewer request.
# Only run after an explicit maintainer action.
if: |
(github.event.action == 'opened' && github.event.pull_request.draft == false && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR' && github.event.pull_request.author_association != 'NONE') ||
(github.event.action == 'ready_for_review' && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR' && github.event.pull_request.author_association != 'NONE') ||
github.event.label.name == 'review-this' ||
github.event.requested_reviewer.login == 'openhands-agent' ||
github.event.requested_reviewer.login == 'all-hands-bot'
Expand All @@ -46,5 +35,5 @@ jobs:
# Review style: roasted (other option: standard)
review-style: roasted
llm-api-key: ${{ secrets.LLM_API_KEY }}
github-token: ${{ secrets.ALLHANDS_BOT_GITHUB_PAT }}
github-token: ${{ github.token }}
lmnr-api-key: ${{ secrets.LMNR_SKILLS_API_KEY }}
27 changes: 12 additions & 15 deletions plugins/pr-review/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# PR Review Plugin

Automated pull request review using OpenHands agents. This plugin provides GitHub workflows that automatically review PRs with detailed, inline code review comments.
Automated pull request review using OpenHands agents. This plugin provides GitHub workflows that maintainers can trigger to review PRs with detailed, inline code review comments.

## Quick Start

Expand All @@ -18,7 +18,7 @@ Then configure the required secrets (see [Installation](#installation) below).

## Features

- **Automated PR Reviews**: Triggered when PRs are opened, marked ready, or when a reviewer is requested
- **Maintainer-Triggered PR Reviews**: Triggered when a maintainer requests the reviewer or adds the `review-this` label
- **Inline Code Comments**: Posts review comments directly on specific lines of code
- **Two Review Styles**:
- `standard` - Balanced code review covering style, readability, and security
Expand Down Expand Up @@ -70,7 +70,7 @@ Add the following secrets in your repository settings (**Settings → Secrets an
| `GITHUB_TOKEN` | Auto | Provided automatically by GitHub Actions |
| `LMNR_SKILLS_API_KEY` | No | Laminar API key (org-level secret; mapped to `LMNR_PROJECT_API_KEY` env var in workflows) |

**Note**: For repositories that need to post review comments from a bot account, use `ALLHANDS_BOT_GITHUB_PAT` instead of `GITHUB_TOKEN`.
**Default**: The example workflow uses the ephemeral `${{ github.token }}`. If you need reviews to come from a dedicated bot account, replace it with a PAT such as `ALLHANDS_BOT_GITHUB_PAT`.

### 3. Customize the Workflow (Optional)

Expand All @@ -97,7 +97,7 @@ Edit the workflow file to customize:

# Secrets
llm-api-key: ${{ secrets.LLM_API_KEY }}
github-token: ${{ secrets.GITHUB_TOKEN }}
github-token: ${{ github.token }}

# Optional: Enable Laminar observability
# lmnr-api-key: ${{ secrets.LMNR_PROJECT_API_KEY }}
Expand All @@ -115,14 +115,12 @@ Create a `review-this` label for manual review triggers:

## Usage

### Automatic Triggers
### Triggers

PR reviews are automatically triggered when:
PR reviews run only after an explicit maintainer action:

1. A new non-draft PR is opened (by non-first-time contributors)
2. A draft PR is marked as ready for review
3. The `review-this` label is added
4. `openhands-agent` or `all-hands-bot` is requested as a reviewer
1. The `review-this` label is added
2. `openhands-agent` or `all-hands-bot` is requested as a reviewer

### Requesting a Review

Expand Down Expand Up @@ -259,7 +257,7 @@ Also update any `sdk-repo` and `sdk-version` inputs to `extensions-repo` and `ex
### Review Not Triggered

1. Check that the workflow file is in `.github/workflows/`
2. Verify the PR author association (first-time contributors need manual trigger)
2. Verify that a maintainer added the `review-this` label or requested the reviewer
3. Ensure secrets are configured correctly

### Review Comments Not Appearing
Expand All @@ -276,12 +274,11 @@ If you see rate limit errors:

## Security

- Uses `pull_request_target` when you need secrets for fork PR reviews; apply strict maintainer-controlled triggers and checkout safeguards
- Uses `pull_request_target` only for explicit maintainer-triggered reviews on fork PRs
- Reviews the PR diff from the GitHub API while checking out the trusted base revision for workspace context
- Keeps GitHub Actions caching disabled in privileged review workflows to avoid cache-poisoning pivots from prompt injection
- Defaults to the ephemeral `${{ github.token }}`; switch to a PAT only if you need a dedicated bot identity
- For lower-trust or comment-only smoke-test setups, prefer `pull_request` to reduce privilege by default
- Only triggers for trusted contributors or when maintainers add labels/reviewers
- PR code is checked out explicitly; secrets are not exposed to PR code
- Credentials are not persisted during checkout

## Contributing

Expand Down
18 changes: 12 additions & 6 deletions plugins/pr-review/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,17 @@ runs:
ref: ${{ inputs.extensions-version }}
path: extensions

- name: Checkout PR repository
# Security: use the trusted base revision for workspace context.
# The review prompt already includes the PR diff from the GitHub API,
# so we do not need to checkout untrusted PR code in this privileged workflow.
- name: Checkout target repository (trusted base)
uses: actions/checkout@v4
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
repository: ${{ github.event.pull_request.base.repo.full_name }}
ref: ${{ github.event.pull_request.base.sha }}
fetch-depth: 0
persist-credentials: false
path: pr-repo
path: target-repo
submodules: recursive

- name: Set up Python
Expand Down Expand Up @@ -133,9 +136,10 @@ runs:
PR_BODY: ${{ github.event.pull_request.body }}
PR_BASE_BRANCH: ${{ github.event.pull_request.base.ref }}
PR_HEAD_BRANCH: ${{ github.event.pull_request.head.ref }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
REPO_NAME: ${{ github.repository }}
run: |
cd pr-repo
cd target-repo
uv run --no-project --with openhands-sdk --with openhands-tools --with lmnr \
python ../extensions/plugins/pr-review/scripts/agent_script.py

Expand All @@ -147,13 +151,15 @@ runs:
path: |
*.log
output/
target-repo/*.log
target-repo/output/
retention-days: 7

- name: Upload Laminar trace info for evaluation
uses: actions/upload-artifact@v4
if: success()
with:
name: pr-review-trace-${{ github.event.pull_request.number }}
path: pr-repo/laminar_trace_info.json
path: target-repo/laminar_trace_info.json
retention-days: 30
if-no-files-found: ignore
13 changes: 11 additions & 2 deletions plugins/pr-review/scripts/agent_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
PR_BODY: Pull request body (optional)
PR_BASE_BRANCH: Base branch name (required)
PR_HEAD_BRANCH: Head branch name (required)
PR_HEAD_SHA: Head commit SHA (optional; used for comment anchoring)
REPO_NAME: Repository name in format owner/repo (required)
REVIEW_STYLE: Review style ('standard' or 'roasted', default: 'standard')
REQUIRE_EVIDENCE: Whether to require PR description evidence showing the code
Expand Down Expand Up @@ -688,15 +689,23 @@ def get_truncated_pr_diff() -> str:


def get_head_commit_sha(repo_dir: Path | None = None) -> str:
"""
Get the SHA of the HEAD commit.
"""Get the commit SHA to anchor review comments on.

Prefer the PR head SHA from the workflow event when available. This keeps
inline comments attached to the untrusted PR revision even when the action
reviews the diff from a trusted base checkout. Fall back to the local HEAD
commit for compatibility with older workflows.

Args:
repo_dir: Path to the repository (defaults to cwd)

Returns:
The commit SHA
"""
pr_head_sha = os.getenv("PR_HEAD_SHA")
if pr_head_sha:
return pr_head_sha

if repo_dir is None:
repo_dir = Path.cwd()
return run_git_command(["git", "rev-parse", "HEAD"], repo_dir).strip()
Expand Down
25 changes: 7 additions & 18 deletions plugins/pr-review/workflows/pr-review-by-openhands.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,13 @@
name: PR Review by OpenHands

on:
# Use pull_request_target to allow fork PRs to access secrets when triggered by maintainers
# Security: This workflow runs when:
# 1. A new PR is opened (non-draft), OR
# 2. A draft PR is marked as ready for review, OR
# 3. A maintainer adds the 'review-this' label, OR
# 4. A maintainer requests openhands-agent or all-hands-bot as a reviewer
# Use pull_request_target so maintainers can trigger reviews on fork PRs.
# Security: require an explicit maintainer action before running:
# 1. A maintainer adds the 'review-this' label, OR
# 2. A maintainer requests openhands-agent or all-hands-bot as a reviewer
# Adding labels and requesting reviewers requires write access.
# The PR code is explicitly checked out for review, but secrets are only accessible
# because the workflow runs in the base repository context.
pull_request_target:
types: [opened, ready_for_review, labeled, review_requested]
types: [labeled, review_requested]

permissions:
contents: read
Expand All @@ -21,15 +17,8 @@ permissions:

jobs:
pr-review:
# Run when one of the following conditions is met:
# 1. A new non-draft PR is opened by a non-first-time contributor, OR
# 2. A draft PR is converted to ready for review by a non-first-time contributor, OR
# 3. 'review-this' label is added, OR
# 4. openhands-agent or all-hands-bot is requested as a reviewer
# Note: FIRST_TIME_CONTRIBUTOR and NONE PRs require manual trigger via label/reviewer request.
# Only run after an explicit maintainer action.
if: |
(github.event.action == 'opened' && github.event.pull_request.draft == false && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR' && github.event.pull_request.author_association != 'NONE') ||
(github.event.action == 'ready_for_review' && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR' && github.event.pull_request.author_association != 'NONE') ||
github.event.label.name == 'review-this' ||
github.event.requested_reviewer.login == 'openhands-agent' ||
github.event.requested_reviewer.login == 'all-hands-bot'
Expand All @@ -46,5 +35,5 @@ jobs:
# Review style: roasted (other option: standard)
review-style: roasted
llm-api-key: ${{ secrets.LLM_API_KEY }}
github-token: ${{ secrets.ALLHANDS_BOT_GITHUB_PAT }}
github-token: ${{ github.token }}
lmnr-api-key: ${{ secrets.LMNR_SKILLS_API_KEY }}
12 changes: 12 additions & 0 deletions tests/test_pr_review_review_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,10 @@ def debug(self, *args, **kwargs):
git_utils.run_git_command = lambda command, repo_dir: "deadbeef"
sys.modules["openhands.sdk.git.utils"] = git_utils

plugin_module = types.ModuleType("openhands.sdk.plugin")
plugin_module.PluginSource = object
sys.modules["openhands.sdk.plugin"] = plugin_module

tools_preset = types.ModuleType("openhands.tools.preset.default")
tools_preset.get_default_condenser = lambda llm: None
tools_preset.get_default_tools = lambda enable_browser=False: []
Expand Down Expand Up @@ -163,3 +167,11 @@ def test_format_thread_includes_rendered_suggestion_text_in_review_context():
assert "- Do **NOT** approve the PR." in formatted
assert "Dependabot ignores the freshness guardrail" in formatted
assert "```suggestion" not in formatted


def test_get_head_commit_sha_prefers_pr_head_sha(monkeypatch):
module = _load_agent_script_module()

monkeypatch.setenv("PR_HEAD_SHA", "abc123")

assert module.get_head_commit_sha() == "abc123"
Loading