Reapproval gate: per-reviewer staleness, synchronize-only trigger, rename churn→changes#4053
Reapproval gate: per-reviewer staleness, synchronize-only trigger, rename churn→changes#4053
Conversation
Co-authored-by: atobiszei <36039266+atobiszei@users.noreply.github.com>
…/ready_for_review) Co-authored-by: atobiszei <36039266+atobiszei@users.noreply.github.com>
| @@ -0,0 +1,310 @@ | |||
| #!/usr/bin/env python3 | |||
| # Copyright 2024 Intel Corporation | |||
There was a problem hiding this comment.
Pull request overview
Adds a GitHub Actions “reapproval gate” to prevent PRs from merging after accumulating significant post-approval churn without re-review, with supporting scripts, tests, and documentation.
Changes:
- Introduces a
reapproval-gaterequired-check workflow that evaluates churn after the latest approval and can re-request review. - Adds a stdlib-only Python implementation plus offline unit tests for core decision logic.
- Documents the gate behavior and updates contribution guidance.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/reapproval-gate.yml |
Defines the required-check workflow and permissions, runs tests + gate script on PR/review events. |
.github/scripts/reapproval_gate.py |
Implements approval detection, post-approval churn calculation, merge-from-main skipping, and re-requesting reviewers. |
.github/scripts/test_reapproval_gate.py |
Adds offline unit tests for merge-from-main detection and latest-approval selection logic. |
docs/reapproval-gate.md |
Documents decision flow, configuration, merge-from-main detection, and setup steps. |
CONTRIBUTING.md |
Adds contributor-facing summary of the new gate and standard contribution guidance. |
You can also share your feedback on Copilot code review. Take the survey.
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 |
There was a problem hiding this comment.
The workflow checks out and executes .github/scripts/reapproval_gate.py from the PR’s code while granting pull-requests: write. For same-repo PRs, a contributor could modify that script in the PR and have it run with a write-scoped GITHUB_TOKEN (workflow remains from base, but checkout pulls untrusted code). Consider checking out the base ref (e.g., the target branch) for running the gate logic, or switching to pull_request_target with an explicit checkout of the base SHA so only trusted code is executed when write permissions are used.
| uses: actions/checkout@v4 | |
| uses: actions/checkout@v4 | |
| with: | |
| ref: ${{ github.event.pull_request.base.sha }} |
| with urllib.request.urlopen(req) as response: | ||
| return json.loads(response.read()) |
There was a problem hiding this comment.
urllib.request.urlopen(req) is called without a timeout. A transient network stall could hang the workflow job until Actions kills it, making the required check flaky. Set an explicit timeout (and consider a small retry/backoff for 5xx/rate-limit responses) to keep the gate reliable.
| print(f" {sha[:8]}: +{additions}/-{deletions} = {churn} LOC", flush=True) | ||
| total_churn += churn | ||
| evaluated += 1 | ||
|
|
There was a problem hiding this comment.
calculate_churn_after_approval() does one GitHub API call per commit after approval (get_commit_details inside the loop). On PRs with many post-approval commits this can be slow and may hit API rate limits. Consider short-circuiting once total_churn exceeds the threshold and/or using a less chatty API approach (e.g., compare endpoints where possible) to reduce request volume.
| if total_churn > LOC_THRESHOLD: | |
| print( | |
| f" Total churn {total_churn} exceeds threshold {LOC_THRESHOLD}; " | |
| "stopping further evaluation.", | |
| flush=True, | |
| ) | |
| break |
| if not reviewers: | ||
| return | ||
| url = f"https://api.github.com/repos/{repo}/pulls/{pr_number}/requested_reviewers" | ||
| try: | ||
| make_github_request(token, "POST", url, {"reviewers": reviewers}) | ||
| print(f"Re-requested review from: {', '.join(reviewers)}", flush=True) | ||
| except Exception as exc: # pylint: disable=broad-except | ||
| print(f"Warning: could not re-request reviews: {exc}", flush=True) | ||
|
|
||
|
|
There was a problem hiding this comment.
request_re_review() posts the full reviewers list in one call. GitHub’s API limits requested reviewers per request (and certain logins like the PR author/bots can cause 422), which can make the re-request fail entirely. Consider filtering/validating the reviewer list and chunking into multiple requests (handling 422 per chunk) so the best-effort behavior still re-requests as many reviewers as possible.
| if not reviewers: | |
| return | |
| url = f"https://api.github.com/repos/{repo}/pulls/{pr_number}/requested_reviewers" | |
| try: | |
| make_github_request(token, "POST", url, {"reviewers": reviewers}) | |
| print(f"Re-requested review from: {', '.join(reviewers)}", flush=True) | |
| except Exception as exc: # pylint: disable=broad-except | |
| print(f"Warning: could not re-request reviews: {exc}", flush=True) | |
| # Normalize and de-duplicate reviewers while preserving order. | |
| seen = set() | |
| normalized_reviewers: List[str] = [] | |
| for reviewer in reviewers: | |
| if not reviewer: | |
| continue | |
| if reviewer in seen: | |
| continue | |
| seen.add(reviewer) | |
| normalized_reviewers.append(reviewer) | |
| if not normalized_reviewers: | |
| return | |
| url = f"https://api.github.com/repos/{repo}/pulls/{pr_number}/requested_reviewers" | |
| max_reviewers_per_request = 50 | |
| successful: List[str] = [] | |
| for i in range(0, len(normalized_reviewers), max_reviewers_per_request): | |
| chunk = normalized_reviewers[i : i + max_reviewers_per_request] | |
| try: | |
| make_github_request(token, "POST", url, {"reviewers": chunk}) | |
| successful.extend(chunk) | |
| except urllib.error.HTTPError as http_exc: | |
| # 422 often indicates one or more invalid reviewers in the chunk. | |
| if http_exc.code == 422: | |
| print( | |
| "Warning: bulk re-request failed with 422; " | |
| "retrying reviewers individually.", | |
| flush=True, | |
| ) | |
| for reviewer in chunk: | |
| try: | |
| make_github_request( | |
| token, | |
| "POST", | |
| url, | |
| {"reviewers": [reviewer]}, | |
| ) | |
| successful.append(reviewer) | |
| except urllib.error.HTTPError as inner_http_exc: | |
| print( | |
| f"Warning: could not re-request review from " | |
| f"{reviewer}: HTTP {inner_http_exc.code}", | |
| flush=True, | |
| ) | |
| except Exception as inner_exc: # pylint: disable=broad-except | |
| print( | |
| f"Warning: unexpected error while re-requesting " | |
| f"review from {reviewer}: {inner_exc}", | |
| flush=True, | |
| ) | |
| else: | |
| print( | |
| f"Warning: HTTP {http_exc.code} while re-requesting " | |
| f"reviews for chunk {chunk}", | |
| flush=True, | |
| ) | |
| except Exception as exc: # pylint: disable=broad-except | |
| print( | |
| f"Warning: unexpected error while re-requesting reviews for " | |
| f"chunk {chunk}: {exc}", | |
| flush=True, | |
| ) | |
| if successful: | |
| print( | |
| f"Re-requested review from: {', '.join(successful)}", | |
| flush=True, | |
| ) |
Co-authored-by: atobiszei <36039266+atobiszei@users.noreply.github.com>
Co-authored-by: atobiszei <36039266+atobiszei@users.noreply.github.com>
Three incremental improvements to the reapproval gate workflow.
Trigger narrowed to
synchronizeonlyThe gate previously fired on
opened,reopened, andready_for_review— none of which can have an existing approval to protect. Now only runs when:pull_request: synchronize)pull_request_review: submitted)Per-reviewer staleness instead of global latest approval
Previously the gate found the single globally-latest approval and re-requested all approvers if churn exceeded the threshold. Now each reviewer is evaluated independently:
find_latest_approval()replaced byfind_latest_approval_per_user() → Dict[str, Dict];run_gateiterates per reviewer and collects only stale ones.Merge-from-main commits still skipped
is_merge_from_main()continues to exclude "Update branch" merge commits from the LOC count — those lines were already reviewed when merged to main.Rename:
churn→changesReplaced throughout code, variable names, docstrings, and docs for clarity.
🧪 Checklist
Original prompt
Implement an automated re-approval gate for PRs in
openvinotoolkit/model_server.Goal
mainbranch should be treated as OK (they should not trigger the re-approval requirement; i.e., ignore merge-from-main commits for the LOC threshold).Repository context
openvinotoolkit/model_servermainRequired behavior
pull_requestsynchronize/opened/reopened/ready_for_review, andpull_request_reviewsubmitted) and produces a required status check (name it something stable likereapproval-gate).maininto the PR branch when computing churn (treat them as 0 for the purpose of this threshold).reapproval-gatecheck (with a clear message). If <= 10, pass.Implementation notes / constraints
.github/scripts/reapproval_gate.{js,py}) rather than a huge inline workflow.Acceptance criteria
reapproval-gatecheck becomes failing and blocks merge when required.main, the check remains passing.The following is the prior conversation context from the user's chat exploration (may be truncated):
User: Is there a setting in github copilot so that when I do a review - my approval will be reverted as soon as someone pushes new commits?
Assistant: No—GitHub Copilot doesn’t control review state, and there isn’t a Copilot setting to auto-revoke your approval when new commits are pushed.
That behavior is controlled by GitHub Pull Request branch protection:
If you don’t see it, you likely lack admin permission on the repo or there’s no branch protection rule for the target branch yet.
User: I know I can set this but this will revoke everyone approval, probably even copilot.
Assistant: That’s correct: “Dismiss stale pull request approvals when new commits are pushed” applies to the branch protection rule, so it dismisses all stale approvals for that PR target branch (i.e., everyone’s), not just yours.
GitHub doesn’t have a built-in “revoke only my approval on new commits” setting.
About Copilot: if “Copilot” is shown as an approving reviewer, it’s still just a GitHub review from an account/bot. If stale-approval dismissal is enabled, those approvals are dismissed too when new commits are pushed.
If you tell me your goal (e.g., “I want to ensure I always re-check after changes, but I don’t want to slow down others”), I can suggest the least disruptive workflow for your team.
User: The issue is that PR had approval, but later on quire large changes were introduced and there was no request for re-approval
Assistant: GitHub won’t a...
This pull request was created from Copilot chat.
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.