Skip to content

Add ACP backend mode to PR review action#210

Merged
xingyaoww merged 26 commits intomainfrom
feat/pr-review-subscription-device-code
May 3, 2026
Merged

Add ACP backend mode to PR review action#210
xingyaoww merged 26 commits intomainfrom
feat/pr-review-subscription-device-code

Conversation

@xingyaoww
Copy link
Copy Markdown
Contributor

@xingyaoww xingyaoww commented Apr 24, 2026

Summary

  • Add an ACP backend mode to the PR review action.
  • Keep the action model simple with two supported agent modes:
    • OpenHands agent: requires llm-api-key and uses the SDK/LiteLLM path.
    • ACP agent: starts the configured acp-command and expects the caller to preinstall and authenticate that ACP server in the runner environment.
  • Keep ACP model access owned by the ACP server while still loading review plugin context, public skills, and repo-level skills through the SDK.
  • Add an SDK package override input for controlled rollout/testing without making direct-reference SDK branches part of the recommended setup.
  • Preserve project skill loading by delegating discovery to the SDK's load_project_skills().

Validation

  • python3 scripts/sync_extensions.py --check
  • python3 -m py_compile plugins/pr-review/scripts/agent_script.py
  • Parsed plugins/pr-review/action.yml and plugins/pr-review/workflows/pr-review-by-openhands.yml with PyYAML
  • uv run --with pytest --with openhands-sdk --with openhands-tools --with lmnr pytest -q tests/test_pr_review_prompt.py tests/test_pr_review_review_context.py (13 passed)
  • Downstream self-hosted workflow validation completed successfully with ACP review mode, project skills, public skills, and a posted PR review

Notes

This PR intentionally does not install Codex, Claude, or any other ACP CLI. ACP setup belongs to the caller workflow or runner image; the action only starts the configured acp-command and connects it to the SDK review flow.

@xingyaoww xingyaoww marked this pull request as ready for review April 28, 2026 14:46
Copy link
Copy Markdown
Contributor

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Good taste - Clean implementation that solves real problems (ChatGPT subscription auth, ACP backend) while maintaining excellent backward compatibility.

[RISK ASSESSMENT]

⚠️ Risk Assessment: 🟡 MEDIUM

This PR introduces new execution paths (ACP mode, subscription auth) with moderate blast radius, but risk is well-mitigated by:

  • Strong backward compatibility (defaults preserve existing api-key + openhands behavior)
  • Comprehensive input validation for new modes
  • Experimental features clearly marked in documentation
  • Successful downstream validation mentioned in PR description
  • Clean separation of concerns between modes

VERDICT:Worth merging - Well-structured feature addition with solid engineering practices and no critical issues.

KEY INSIGHT: The mode-based architecture cleanly extends the PR review action without breaking existing workflows, demonstrating good taste in API evolution.

@xingyaoww xingyaoww changed the title Add subscription auth mode to PR review action Add ACP backend mode to PR review action Apr 28, 2026
@xingyaoww
Copy link
Copy Markdown
Contributor Author

Design update after simplifying the configuration surface:

This action now supports two agent modes only:

  1. review-agent-mode: openhands

    • Uses the standard OpenHands SDK agent path.
    • Requires llm-api-key.
    • Optional llm-base-url still works for LiteLLM/proxy deployments.
  2. review-agent-mode: acp

    • Starts the configured ACP server from acp-command.
    • Does not consume llm-api-key and does not expose a separate action-level LLM auth mode.
    • The ACP server is responsible for model access and authentication. Any subscription/local-login credentials must already be available in the runner environment before this action starts the ACP process.

I removed the previous llm-auth-mode, device-code, and subscription instruction inputs. That keeps this action from becoming a second auth layer and avoids the confusing "LLM off / subscription" configuration path.

@xingyaoww
Copy link
Copy Markdown
Contributor Author

Follow-up cleanup:

  • Removed the custom _load_review_project_skills() helper.
  • PR review now delegates project skill discovery directly to the SDK's load_project_skills(cwd).
  • This keeps skill-loading semantics centralized in the SDK instead of teaching this action to scan extra repository directories itself.
  • Repo-specific review skills should live in SDK-supported project skill locations such as .agents/skills/ / .openhands/skills/, or in always-on repo guidance like AGENTS.md.

I also expanded the ACP/sub-agent explanation: sub-agent delegation is disabled in ACP mode because it depends on OpenHands-specific runtime behavior (TaskToolSet, agent registration, and tool routing). ACP servers do not expose those semantics consistently, so this action should not pretend that delegation is portable across arbitrary ACP implementations.

@xingyaoww
Copy link
Copy Markdown
Contributor Author

Follow-up cleanup: removed the agent_context.load_public_skills = False workaround and its dedicated test.

The action now leaves AgentContext(load_public_skills=True, skills=project_skills) intact and relies on the SDK/Conversation plumbing to handle validation/copy behavior. If duplicate public skill loading becomes a problem, that should be fixed in the SDK rather than by mutating the context flag in this action.

@xingyaoww
Copy link
Copy Markdown
Contributor Author

Follow-up simplification: removed the bundled Codex-specific ACP setup.

The action no longer has codex-cli-package, no longer installs the Codex CLI, and no longer defaults acp-command to a Codex ACP command. In ACP mode, acp-command is required and must already be available in the runner environment.

That makes the boundary provider-neutral:

  • Caller workflow / runner image: install and authenticate the chosen ACP server, whether Codex, Claude, or another ACP CLI.
  • plugins/pr-review action: validate inputs, start acp-command, and wire the ACP server into the SDK PR review flow.

This avoids growing provider-specific install/auth branches inside the action.

@xingyaoww
Copy link
Copy Markdown
Contributor Author

Follow-up: removed the public acp-prompt-timeout input.

There is no existing action-level timeout input to reuse here; the workflow/job timeout-minutes remains the outer timeout controlled by the caller workflow. The ACP prompt timeout is now just an internal 1800s guardrail in the script, so the action API does not grow an ACP-specific timeout knob.

@xingyaoww
Copy link
Copy Markdown
Contributor Author

ACP validation update with Codex ACP after the latest SDK changes:

  • A private self-hosted-runner workflow completed successfully with review-agent-mode: acp and acp-command: codex-acp.
  • The runner restored preconfigured Codex authentication, installed @openai/codex@0.124.0 and @zed-industries/codex-acp@0.12.0, and reported codex-cli 0.124.0.
  • The workflow used the merged SDK main package while waiting for the next PyPI release:
    openhands-sdk @ git+https://github.com/OpenHands/software-agent-sdk.git@main#subdirectory=openhands-sdk.
  • ACP_PROMPT_TIMEOUT was set to 1800.
  • The review action loaded both project and public skill context, including a repository-specific review skill, and the resulting review followed that skill's language requirement.
  • The PR review job completed successfully.

Redacted log excerpt:

codex-cli 0.124.0
review-agent-mode: acp
acp-command: codex-acp
llm-model: gpt-5.5
acp-prompt-timeout: 1800
OPENHANDS_SDK_PACKAGE: openhands-sdk @ git+https://github.com/OpenHands/software-agent-sdk.git@main#subdirectory=openhands-sdk
ACP_COMMAND: codex-acp
ACP_PROMPT_TIMEOUT: 1800
Review agent mode: acp
Using skill trigger: /codereview
Loaded 10 project skills: [..., 'custom-codereview-guide']
Loaded 39 public skills: [..., 'code-review', ...]
Using ACP review agent with command: codex-acp
pr-review: success

Copy link
Copy Markdown
Contributor

@malhotra5 malhotra5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat!

Copy link
Copy Markdown
Contributor

@simonrosenberg simonrosenberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment thread plugins/pr-review/README.md
Comment thread tests/test_pr_review_review_context.py Outdated
@xingyaoww
Copy link
Copy Markdown
Contributor Author

Validation update after the SDK release:

  • Re-ran the ACP PR review workflow against the published openhands-sdk package, without pinning SDK main.
  • The workflow completed successfully with Codex ACP launched through npx.
  • Repository-specific review skills and public skills were both loaded, and the resulting review followed the repository-specific language instruction.

Redacted log excerpt:

review-agent-mode: acp
acp-command: npx -y @zed-industries/codex-acp@0.12.0
OPENHANDS_SDK_PACKAGE: openhands-sdk
ACP_COMMAND: npx -y @zed-industries/codex-acp@0.12.0
ACP_PROMPT_TIMEOUT: 1800
OpenHands SDK v1.19.1
Review agent mode: acp
Using skill trigger: /codereview
Loaded 10 project skills: [..., 'custom-codereview-guide']
Loaded 39 public skills: [..., 'code-review', ...]
Using ACP review agent with command: npx -y @zed-industries/codex-acp@0.12.0
pr-review: success

Add a paragraph explaining that ACP servers support API key auth
(via standard provider environment variables) as the simplest method,
alongside the existing device-code login documentation.

Co-authored-by: openhands <openhands@all-hands.dev>
Align naming with software-agent-sdk's agent_kind convention.

Renames across action.yml, agent_script.py, README.md, and tests.

Co-authored-by: openhands <openhands@all-hands.dev>
Comment thread plugins/pr-review/scripts/agent_script.py Outdated
Simplify the env var / input / config key naming:
- Env var: REVIEW_AGENT_KIND -> AGENT_KIND
- Action input: review-agent-kind -> agent-kind
- Python var / config key: review_agent_kind -> agent_kind

Co-authored-by: openhands <openhands@all-hands.dev>
Comment thread plugins/pr-review/README.md Outdated
@xingyaoww
Copy link
Copy Markdown
Contributor Author

Re-validated the latest PR head after the agent-kind rename.

A private self-hosted-runner workflow completed successfully with the updated action input and the published SDK package.

Redacted log excerpt:

Download action repository 'OpenHands/extensions@feat/pr-review-subscription-device-code' (SHA:55d4cd035a5861191679fe08dcf843ff35f0b353)
agent-kind: acp
acp-command: npx -y @zed-industries/codex-acp@0.12.0
AGENT_KIND: acp
OPENHANDS_SDK_PACKAGE: openhands-sdk
ACP_COMMAND: npx -y @zed-industries/codex-acp@0.12.0
ACP_PROMPT_TIMEOUT: 1800
OpenHands SDK v1.19.1
Agent kind: acp
Using skill trigger: /codereview
Loaded 10 project skills: [..., 'custom-codereview-guide']
Loaded 39 public skills: [..., 'code-review', ...]
Using ACP review agent with command: npx -y @zed-industries/codex-acp@0.12.0
pr-review: success

The resulting review followed the repository-specific language instruction.

@xingyaoww xingyaoww merged commit d97b974 into main May 3, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants