Automate external plugin update PR quality checks#2005
Open
aaronpowell wants to merge 5 commits into
Open
Conversation
Automate external plugin update PR review by running skill-validator and install smoke checks against changed entries in plugins/external.json. Sync PR workflow-state labels and upsert a marker-based status comment with source tree links for each changed plugin. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
added
new-submission
Contributor
There was a problem hiding this comment.
Pull request overview
Adds automation to evaluate external plugin update pull requests (those modifying plugins/external.json) by detecting changed entries, running existing quality gates per entry, and syncing PR labels plus a marker-based status comment so reviewers get consistent, actionable signals.
Changes:
- Introduces
External Plugin PR Quality Gatesworkflow to detect changed external plugin entries, run install smoke + skill-validator gates, then sync PR labels and a summarized PR comment. - Adds an
eng/helper script to aggregate per-plugin results and generate source tree links for PR status output. - Documents the PR-based external plugin update path in
CONTRIBUTING.md.
Show a summary per file
| File | Description |
|---|---|
eng/external-plugin-pr-quality-gates.mjs |
Aggregates existing external plugin quality gates for a list of changed plugin entries and produces a summarized result payload. |
CONTRIBUTING.md |
Documents the new PR-based automation path for updating plugins/external.json. |
.github/workflows/external-plugin-pr-quality-gates.yml |
Adds a PR workflow that detects changed entries, runs gates, and syncs labels + a marker comment onto the PR. |
Copilot's findings
- Files reviewed: 3/3 changed files
- Comments generated: 3
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment on lines
+112
to
+115
| sync-pr-state: | ||
| runs-on: ubuntu-latest | ||
| needs: [detect-changed-plugins, run-quality-gates] | ||
| if: always() |
Contributor
Author
There was a problem hiding this comment.
@copilot do we need to add issue: read permissions when adding comments to PRs?
Contributor
There was a problem hiding this comment.
No — we don’t need a separate issues: read. For PR labels/comments, issues: write is sufficient and already implies the read access that actions/github-script needs. I also fixed a missing steps: key in c684617, which was preventing this job from parsing correctly.
Scope write permissions to the PR synchronization job, keep the quality-gate job read-only, and handle no-op and detection-failure states explicitly. Also fix source tree link encoding for refs, SHAs, and plugin paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Checklist
npm startand verified thatREADME.mdis up to date.stagedbranch for this pull request.Description
External plugin update PRs like #2004 currently rely on manual review context, while issue intake already has automated quality gates. This adds equivalent automation to PRs that modify
plugins/external.jsonso reviewers get consistent, actionable state signals.The change introduces a new PR workflow that detects changed external plugin entries, runs the existing smoke install and skill-validator gates for each changed entry, then syncs PR labels to workflow state (
ready-for-review,requires-submitter-fixes,awaiting-review). It also upserts a marker-based PR comment with per-plugin gate results and a direct source tree link (repo + ref/sha + path) for each changed plugin. A small script (eng/external-plugin-pr-quality-gates.mjs) aggregates per-plugin execution and summary output, and CONTRIBUTING guidance now documents this PR update path.Follow-up fixes from review feedback are also included so the workflow matches the intended behavior: repository-level permissions stay read-only by default, PR write access is scoped to the sync job that applies labels/comments, and the sync job keeps the permissions and YAML structure needed to check out
stagedand run successfully.Type of Contribution
Additional Notes
Validated with
npm run build,npm run plugin:validate,npm run skill:validate, and targeted workflow/YAML verification after the review-driven fixes.By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.