Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
48ded10
Add subscription auth mode to PR review action
xingyaoww Apr 24, 2026
b4a9708
Add ACP backend mode to PR review action
xingyaoww Apr 24, 2026
5fd222a
Install Codex CLI for ACP reviews
xingyaoww Apr 24, 2026
a63e2b6
Use user npm prefix for Codex CLI install
xingyaoww Apr 24, 2026
256e025
Avoid public skill clone in PR review action
xingyaoww Apr 24, 2026
4e93db8
Keep public skills enabled for PR reviews
xingyaoww Apr 24, 2026
e627557
Update Codex ACP default for PR reviews
xingyaoww Apr 24, 2026
2051062
Avoid duplicate public skill loading in PR review
xingyaoww Apr 25, 2026
c768ebc
Load top-level review skills in PR review plugin
xingyaoww Apr 26, 2026
10c853e
Merge remote-tracking branch 'origin/main' into feat/pr-review-subscr…
xingyaoww Apr 28, 2026
1e2ab8b
Polish PR review SDK package docs
xingyaoww Apr 28, 2026
462a8e1
Simplify PR review agent auth modes
xingyaoww Apr 28, 2026
fa1a580
Simplify PR review skill loading and ACP sub-agent docs
xingyaoww Apr 28, 2026
57d3d60
Remove public skill reload workaround
xingyaoww Apr 28, 2026
de288ee
Remove bundled ACP CLI installation
xingyaoww Apr 28, 2026
5ef3140
Clarify ACP command and SDK version inputs
xingyaoww Apr 28, 2026
c00e861
Remove ACP prompt timeout input
xingyaoww Apr 28, 2026
a343d6b
Restore ACP prompt timeout input and Codex example
xingyaoww Apr 28, 2026
25bd50e
Document Codex ACP workflow setup
xingyaoww Apr 28, 2026
005212e
Document Codex ACP auth secret setup
xingyaoww Apr 28, 2026
8dc8c17
Clarify Codex device auth secret setup
xingyaoww Apr 28, 2026
832dde6
Use npx Codex ACP command in docs
xingyaoww Apr 28, 2026
6a71574
Document API key authentication for ACP backends
openhands-agent May 3, 2026
19779cd
Rename review-agent-mode to review-agent-kind
openhands-agent May 3, 2026
30470e8
Rename REVIEW_AGENT_KIND to AGENT_KIND
openhands-agent May 3, 2026
55d4cd0
Apply suggestion from @xingyaoww
xingyaoww May 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 80 additions & 6 deletions plugins/pr-review/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Add the following secrets in your repository settings (**Settings → Secrets an

| Secret | Required | Description |
|--------|----------|-------------|
| `LLM_API_KEY` | Yes | API key for your LLM provider |
| `LLM_API_KEY` | Yes for `api-key` mode | API key for your LLM provider |
| `GITHUB_TOKEN` | Auto | Provided automatically by GitHub Actions |
| `LMNR_SKILLS_API_KEY` | No | Laminar API key (org-level secret; mapped to `LMNR_PROJECT_API_KEY` env var in workflows) |

Expand All @@ -81,7 +81,7 @@ Edit the workflow file to customize:
with:
# LLM model(s) - comma-separated for A/B testing
llm-model: anthropic/claude-sonnet-4-5-20250929

# Optional: Custom LLM endpoint
# llm-base-url: https://your-llm-proxy.example.com

Expand All @@ -102,6 +102,77 @@ Edit the workflow file to customize:
# lmnr-api-key: ${{ secrets.LMNR_PROJECT_API_KEY }}
```

#### Experimental: ACP review backend

Use `agent-kind: acp` to run the reviewer through an ACP-compatible
agent server. In this mode, OpenHands still loads review skills and plugin
prompt context, but the ACP server owns model access, authentication, and tool
execution. Install the ACP CLI and configure its authentication in the runner
environment before invoking this action. Sub-agent delegation is disabled in ACP
mode because delegation depends on OpenHands agent runtime details such as
TaskToolSet, agent registration, and tool routing that ACP servers do not expose
consistently.

```yaml
- name: Run PR Review
uses: OpenHands/extensions/plugins/pr-review@main
with:
agent-kind: acp
acp-command: your-acp-server
llm-model: your-acp-model
github-token: ${{ secrets.GITHUB_TOKEN }}
```

**Authentication.** ACP servers typically support two authentication methods.
The simplest is API key authentication: store your provider API key as a
repository secret (e.g. `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`) and pass it to the ACP server as an
environment variable in the workflow step. Most ACP servers will pick up the
standard provider key automatically. Alternatively, subscription-based tools
such as Codex support device-code login, which lets you use a ChatGPT
Plus/Pro subscription without a separate API key - see the example below.

Codex ACP example for a runner that stores Codex auth in a GitHub secret.
Create `CODEX_AUTH_JSON_B64` from a trusted machine where the Codex CLI is
installed, then complete device-code login:

```bash
codex login --device-auth
codex login status
```

After login succeeds, encode the generated auth file:

```bash
base64 -w 0 "$HOME/.codex/auth.json"
```

Store the printed value as the repository or organization secret
`CODEX_AUTH_JSON_B64`. This secret contains your Codex authentication material;
limit it to trusted self-hosted runners and avoid exposing it to workflows that
run untrusted pull request code.

```yaml
Comment thread
xingyaoww marked this conversation as resolved.
- name: Restore Codex auth
env:
CODEX_AUTH_JSON_B64: ${{ secrets.CODEX_AUTH_JSON_B64 }}
run: |
mkdir -p "$HOME/.codex"
printf '%s' "$CODEX_AUTH_JSON_B64" | base64 -d > "$HOME/.codex/auth.json"
chmod 600 "$HOME/.codex/auth.json"

- name: Run PR Review
uses: OpenHands/extensions/plugins/pr-review@main
with:
agent-kind: acp
acp-command: npx -y @zed-industries/codex-acp@0.12.0
llm-model: gpt-5.5
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Cleanup Codex auth
if: always()
run: rm -f "$HOME/.codex/auth.json"
```

### 4. Create the Review Label (Optional)

Create a `review-this` label for manual review triggers:
Expand Down Expand Up @@ -138,14 +209,18 @@ PR reviews are automatically triggered when:

| Input | Required | Default | Description |
|-------|----------|---------|-------------|
| `llm-model` | No | `anthropic/claude-sonnet-4-5-20250929` | LLM model(s), comma-separated for A/B testing |
| `agent-kind` | No | `openhands` | Review backend: `openhands` for the standard SDK Agent or `acp` for an ACP-compatible agent server |
| `llm-model` | No | `anthropic/claude-sonnet-4-5-20250929` | LLM model(s), comma-separated for A/B testing. In ACP mode this is passed to the ACP server when supported. |
| `acp-command` | Yes for `acp` mode | `''` | Command used to start the ACP server. The command must already be available in the runner environment or be runnable through a package manager. Examples: `npx -y @zed-industries/codex-acp@0.12.0`, `codex-acp`, `claude-agent-acp`, `npx -y @agentclientprotocol/claude-agent-acp`. |
| `acp-prompt-timeout` | No | `'1800'` | Timeout in seconds for one ACP prompt turn |
| `llm-base-url` | No | `''` | Custom LLM endpoint URL |
| `review-style` | No | `roasted` | **[DEPRECATED]** Previously chose between `standard` and `roasted` review styles. Now ignored — the styles have been merged into a single unified skill. |
| `require-evidence` | No | `'false'` | Require the reviewer to enforce an `Evidence` section in the PR description with end-to-end proof: screenshots/videos for frontend work, commands and runtime output for backend or scripts, and an agent conversation link when applicable. Test output alone does not qualify. |
| `use-sub-agents` | No | `'false'` | Enable sub-agent delegation for file-level reviews. The main agent acts as a coordinator that delegates per-file review work to `file_reviewer` sub-agents via the SDK TaskToolSet, then consolidates findings into a single PR review. Useful for large PRs with many changed files. **Disabled by default** due to high token costs and potential timeouts (see [#208](https://github.com/OpenHands/extensions/issues/208)). Set to `'true'` to opt in. |
| `use-sub-agents` | No | `'false'` | Enable sub-agent delegation for file-level reviews in `openhands` mode. The main agent acts as a coordinator that delegates per-file review work to `file_reviewer` sub-agents via the SDK TaskToolSet, then consolidates findings into a single PR review. Useful for large PRs with many changed files. **Disabled by default** due to high token costs and potential timeouts (see [#208](https://github.com/OpenHands/extensions/issues/208)). Set to `'true'` to opt in. Ignored in ACP mode. |
| `extensions-repo` | No | `OpenHands/extensions` | Extensions repository |
| `extensions-version` | No | `main` | Git ref (tag, branch, or SHA) |
| `llm-api-key` | Yes | - | LLM API key |
| `openhands-sdk-package` | No | `openhands-sdk` | Package spec passed to `uv --with`; override only when pinning a specific SDK build for testing or rollout control |
| `llm-api-key` | Yes for `openhands` mode | - | LLM API key for the OpenHands agent. Ignored in ACP mode. |
| `github-token` | Yes | - | GitHub token for API access |
| `lmnr-api-key` | No | `''` | Laminar API key for observability |
| `enable-uv-cache` | No | `'false'` | Enable setup-uv's GitHub Actions cache for Python deps. Default `false` for security (see [Caching and Security](#caching-and-security)). |
Expand Down Expand Up @@ -314,4 +389,3 @@ See the main [extensions repository](https://github.com/OpenHands/extensions) fo
## License

This plugin is part of the OpenHands extensions repository. See [LICENSE](../../LICENSE) for details.

65 changes: 58 additions & 7 deletions plugins/pr-review/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,35 @@ branding:
color: blue

inputs:
agent-kind:
description: >
Review agent backend. Use 'openhands' for the standard OpenHands
SDK Agent, or 'acp' to run an ACP-compatible agent server.
required: false
default: openhands
llm-model:
description: >
LLM model to use for the review. Can be a comma-separated list for A/B testing
- one model will be randomly selected per review.
LLM model to use for the review. In ACP mode this is passed to the
ACP server when supported. Can be a comma-separated list for A/B
testing - one model will be randomly selected per review.
Example: 'model-a' or 'model-a,model-b,model-c'
required: false
default: anthropic/claude-sonnet-4-5-20250929
acp-command:
description: >
Command used to start the ACP server when agent-kind is
'acp'. The command must be available in the runner environment
or runnable through a package manager. The value is shell-split
before execution.
Examples: 'npx -y @zed-industries/codex-acp@0.12.0',
'codex-acp', 'claude-agent-acp', or 'npx -y
@agentclientprotocol/claude-agent-acp'.
required: false
default: ''
acp-prompt-timeout:
description: Timeout in seconds for one ACP prompt turn.
required: false
default: '1800'
llm-base-url:
description: LLM base URL (optional, for custom LLM endpoints)
required: false
Expand Down Expand Up @@ -44,9 +66,18 @@ inputs:
description: Git ref to use for extensions (tag, branch, or commit SHA, e.g., v1.0.0, main, or abc1234)
required: false
default: main
openhands-sdk-package:
description: >
Package spec passed to uv for openhands-sdk. Keep the default for
normal use, or override it to pin a specific SDK build for testing
or rollout control.
required: false
default: openhands-sdk
llm-api-key:
description: LLM API key (required)
required: true
description: >
LLM API key. Required when agent-kind is 'openhands'.
Ignored in ACP mode because the ACP server owns authentication.
required: false
github-token:
description: GitHub token for API access (required)
required: true
Expand Down Expand Up @@ -119,11 +150,23 @@ runs:
id: select-model
shell: bash
env:
AGENT_KIND: ${{ inputs.agent-kind }}
ACP_COMMAND: ${{ inputs.acp-command }}
LLM_API_KEY: ${{ inputs.llm-api-key }}
GITHUB_TOKEN: ${{ inputs.github-token }}
run: |
if [ -z "$LLM_API_KEY" ]; then
echo "Error: llm-api-key is required."
if [ "$AGENT_KIND" != "openhands" ] && [ "$AGENT_KIND" != "acp" ]; then
echo "Error: agent-kind must be 'openhands' or 'acp'."
exit 1
fi

if [ "$AGENT_KIND" = "openhands" ] && [ -z "$LLM_API_KEY" ]; then
echo "Error: llm-api-key is required when agent-kind is 'openhands'."
exit 1
fi

if [ "$AGENT_KIND" = "acp" ] && [ -z "$ACP_COMMAND" ]; then
echo "Error: acp-command is required when agent-kind is 'acp'."
exit 1
fi

Expand All @@ -142,8 +185,12 @@ runs:
echo "PR Title: ${{ github.event.pull_request.title }}"
echo "Repository: ${{ github.repository }}"
echo "Extensions Version: ${{ inputs.extensions-version }}"
echo "Agent kind: $AGENT_KIND"
echo "Available models: $MODELS_LIST"
echo "Selected LLM model: $SELECTED_MODEL"
if [ "$AGENT_KIND" = "acp" ]; then
echo "ACP command: ${{ inputs.acp-command }}"
fi
if [ -n "${{ inputs.llm-base-url }}" ]; then
echo "LLM base URL: ${{ inputs.llm-base-url }}"
fi
Expand All @@ -155,6 +202,10 @@ runs:
# ignores any .python-version in the PR repo checkout (which
# could be older *or* newer than 3.12).
UV_PYTHON: '3.12'
OPENHANDS_SDK_PACKAGE: ${{ inputs.openhands-sdk-package }}
AGENT_KIND: ${{ inputs.agent-kind }}
ACP_COMMAND: ${{ inputs.acp-command }}
ACP_PROMPT_TIMEOUT: ${{ inputs.acp-prompt-timeout }}
LLM_MODEL: ${{ steps.select-model.outputs.selected_model }}
LLM_BASE_URL: ${{ inputs.llm-base-url }}
REVIEW_STYLE: ${{ inputs.review-style }}
Expand All @@ -171,7 +222,7 @@ runs:
REPO_NAME: ${{ github.repository }}
run: |
cd pr-repo
uv run --no-project --with openhands-sdk --with openhands-tools --with lmnr \
uv run --no-project --with "$OPENHANDS_SDK_PACKAGE" --with openhands-tools --with lmnr \
python ../extensions/plugins/pr-review/scripts/agent_script.py

- name: Upload logs as artifact
Expand Down
Loading
Loading