feat(code-review): auto-verify GitHub Actions runner version from CI logs#186
Draft
aivong-openhands wants to merge 1 commit intomainfrom
Draft
feat(code-review): auto-verify GitHub Actions runner version from CI logs#186aivong-openhands wants to merge 1 commit intomainfrom
aivong-openhands wants to merge 1 commit intomainfrom
Conversation
…logs
Add section 9 'GitHub Actions Version Upgrades' to the code review skill
that instructs the agent to proactively verify runner version compatibility
from the PR's own CI job logs, instead of asking the PR author to check
manually.
When a GitHub Action upgrade requires a minimum runner version (e.g.,
docker/login-action v4 requires Actions Runner v2.327.1+ for Node 24),
the reviewer now:
1. Fetches workflow runs for the PR via the GitHub API
2. Extracts the runner version from job logs ('Current runner version:')
3. Compares against the requirement and reports the verified result
4. Falls back to flagging for manual confirmation if CI hasn't run
Also adds 6 tests for the new section and updates existing test for
the renumbered Risk and Safety Evaluation section (9→10).
Co-authored-by: openhands <openhands@all-hands.dev>
all-hands-bot
approved these changes
Apr 17, 2026
Contributor
all-hands-bot
left a comment
all-hands-bot
left a comment
There was a problem hiding this comment.
🟢 Good taste - Pragmatic solution that eliminates manual busywork.
[IMPROVEMENT OPPORTUNITIES]
- [skills/code-review/SKILL.md, Line 105] Clarity: Consider briefly mentioning how to find
job_id(e.g., viagh api repos/{owner}/{repo}/actions/runs/{run_id}/jobs), though the current wording is acceptable since it implies the agent has already navigated to the job.
[RISK ASSESSMENT]
- [Overall PR]
⚠️ Risk Assessment: 🟢 LOW
Documentation-only change with solid test coverage. Adds actionable instructions for automated runner version verification. No breaking changes, no code execution changes. The grep pattern"Current runner version:"depends on GitHub's log format, but this is a pragmatic trade-off that can be updated if the format changes.
VERDICT:
✅ Worth merging: Solves a real problem (referenced in review comment r3094412910) with clear, testable instructions.
KEY INSIGHT:
This automates verification that was previously manual busywork - exactly the kind of incremental improvement that compounds into better code review quality.
Contributor
Author
|
Followup to #167 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
When the code review skill reviews a PR that upgrades a GitHub Action (e.g.,
docker/login-actionv3→v4), it currently flags the runner version requirement from the release notes but asks the PR author to manually verify compatibility. The agent is already looking at the CI workflow job that ran the upgraded action to confirm it passed — it should extract the runner version from that same job's logs and report it directly, removing unnecessary manual work.Summary
skills/code-review/SKILL.md) that instructs the agent to extract the runner version from the CI job it is already inspecting, compare it against the requirement, and report the verified result.Issue Number
Related to review comment r3094412910 on OpenHands/OpenHands#13960.
How to Test
Run the test suite:
All 17 tests pass (6 new + 11 existing).
Notes
plugins/pr-review/skills/code-review/directory is a symlink toskills/code-review/, so the pr-review plugin automatically picks up this change.