Plugin Directory: Serve a release immediately when Gandalf reports no findings#717
Plugin Directory: Serve a release immediately when Gandalf reports no findings#717obenland wants to merge 3 commits into
Conversation
The test environment was missing three things the development environment already had, which between them made whole classes of test impossible: - `update_source` was never created. `after-start.sh` imports `database-tables.sql` for development; the test script did not, so anything touching the update API's table failed. - `wp-content/env-bin` was not mapped, so `database-tables.sql` was not reachable from the test container to import in the first place. - `PLUGINS_TABLE_PREFIX` was undefined. `SVN_Access` reads it at construction, so `API\Base::load_routes()` fatalled and *no* `plugins/v1` route was registered — every REST request in the test environment 404'd. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ngq7zaYBfv2zS8dnjFoUG
6163382 to
2f0a96d
Compare
2f0a96d to
63cb850
Compare
… findings. The release delay exists to give scanners and humans a window to flag a bad release before it reaches sites. Until now the window was spent in full on every release, including the overwhelming majority that a scanner has already looked at and cleared. This closes that loop: when a Gandalf scan completes with no findings, the version it scanned stops waiting and starts being served. `force_release()` and the new path do the same thing — clear a release's delay and write it to `update_source` — so they collapse into one `API_Update_Updater::serve_release_now( $slug, $version, $reason, $user )`. What separates a reviewer force-releasing from a clean scan is recorded in `$reason`, not in the behaviour, and every such release now logs in one greppable format. The version is passed explicitly and re-checked against the plugin's current one. Both callers act on a version they evaluated earlier — a reviewer reading the metabox, a scanner reading a ZIP — and a newer commit can land in between; serving a stale version would skip the wait on a release nobody evaluated, which is precisely what the delay exists to cover. The metabox did this check itself and `force_release()` did not; now there is one implementation. Callback payload validation is restored, as an args schema on the route. WordPress#634 dropped it on the grounds that the scan verdict was advisory, where the worst case was a wrong Slack message. That no longer holds now that the verdict serves releases: `(int) 'abc'` is 0, and a `findings_count` that is absent or malformed would read as a clean scan and publish. We validate what we act on, at our own boundary. `error.kind` is accepted as an open string rather than a fixed enum, so a new failure kind needs no WPORG deploy. Adds coverage for the delay lifecycle and the callback route, including the gate itself, which had none. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019ngq7zaYBfv2zS8dnjFoUG
63cb850 to
6e11272
Compare
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the Core Committers: Use this line as a base for the props when committing in SVN: To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
Luc45
left a comment
There was a problem hiding this comment.
Thanks for putting this together.
I have limited context on the WPORG side, so my review will focus mostly on the Gandalf side.
Nonblocking follow-ups
There's two issues that I think we could tackle in a follow-up, that doesn't have to hold this PR.
- Same reference, different ZIP. A ZIP can change during a scan without changing its version or release ref, so WPORG could accelerate an artifact Gandalf never scanned. Follow up by returning the scanned ZIP digest and comparing it with the current ZIP before acceleration.
- Zero findings, incomplete coverage. Gandalf can complete with zero findings even when
prompt-securitycould not scan everything within its budget; WPORG cannot currently tell. Follow up by returning coverage status so WPORG can withhold acceleration when coverage is incomplete.
Recent Gandalf data shows that about 99% of attempted scans complete, so I’m comfortable handling these separately.
WPORG clarification
Just to be clear, since I’m not familiar with the WPORG side, the way I understand it there are two separate cooldowns:
- The older one, from 2025:
manual-updates-24hr. From the phased rollout work in Meta Trac #8009 and PR #498, once the version is offered through the update API this allows manual updates but disables automatic updates for 24 hours. - The newer one, from 2026:
release_delay. From PR #650, this happens first: the update API continues offering the previous version for the configured delay, leaving time for scanners and reviewers to act.
My understanding is that the intended effect of this PR is only to end release_delay early, which seems expected. I've created this diagram below to wrap my head around how those two cooldowns behave, and how this PR changes it, let me know if this looks accurate to you.
Before this PR
■ release_delay □ manual-updates-24hr
0h 24h 48h
├───────────────────────┼───────────────────────┤
No clean callback ■■■■■■■■■■■■■■■■■■■■■■■■□□□□□□□□□□□□□□□□□□□□□□□□
↑M ↑A
Clean result at +2h (not acted on) ■■■■■■■■■■■■■■■■■■■■■■■■□□□□□□□□□□□□□□□□□□□□□□□□
↑M ↑A
Clean result at +26h (not acted on) ■■■■■■■■■■■■■■■■■■■■■■■■□□□□□□□□□□□□□□□□□□□□□□□□
↑M ↑A
After this PR
■ release_delay □ manual-updates-24hr
0h 24h 48h
├───────────────────────┼───────────────────────┤
No clean callback ■■■■■■■■■■■■■■■■■■■■■■■■□□□□□□□□□□□□□□□□□□□□□□□□
↑M ↑A
Clean callback at +2h ■■□□□□□□□□□□□□□□□□□□□□□□
↑M ↑A
Clean callback at +26h ■■■■■■■■■■■■■■■■■■■■■■■■□□
↑M↑A
Each square represents one hour. ↑M marks when the new version becomes available as a manual update to existing installations; ↑A marks when it becomes eligible for automatic updates.
That is what the code appears to do. Before merging, does this look accurate and intentional to you?
My review is limited to the Gandalf side, so it may be useful to have a WPORG reviewer confirm these points.
…ation when serving early. Serving a release early via serve_release_now() zeroed its release_delay before update_single_plugin() ran, which disabled the guard that anchors release_time to the moment the version goes public. release_time fell back to the commit time, silently shortening (or collapsing) the author-set manual-updates-24hr window that phased_rollout() measures from it. Anchor release_time whenever a version newly enters update_source, not only while a delay is active, so an expedited release still dates from publication. A clean scan arriving after the delay already elapsed now leaves the served version — and its running window — untouched. Extracts get_served_version() to share the update_source lookup that both paths need. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
I'm not sure about the "Clean callback at +26h" section, but the rest looks good. 27cddb6 is meant to make that clearer, anchoring |
The release delay (#650) buys a window for scanners and humans to flag a bad release before it reaches sites. Gandalf (#634) fills the automated half. The two don't talk to each other, so the window is spent in full on every release — including the overwhelming majority a scanner has already cleared.
This closes the loop: when a Gandalf scan completes with no findings, the version it scanned stops waiting and starts being served. Findings still go to Slack for a reviewer. A scan that fails or never reports back leaves the delay to elapse on its own — silence never shortens the wait.
Changes
serve_release_now()replacesforce_release(). Both do the same thing — clear a release's delay and write it toupdate_source— so a reviewer force-releasing and a clean scan now share one path, differing only in the$reasonthey log.The version is checked in one place.
serve_release_now()takes the version explicitly and re-checks it against the plugin's current one. Callers act on a version they evaluated earlier — a reviewer reading the metabox, a scanner reading a ZIP — and a newer commit can land in between; serving a stale version would skip the wait on a release nobody evaluated. The metabox did this check itself andforce_release()didn't; now there's one implementation.The author's
manual-updates-24hrwindow is preserved. Serving early still dates the release from the moment it goes public, not from the commit, sophased_rollout()'s author-set window runs its full length from availability. A clean scan that arrives after the delay already elapsed leaves the served version untouched.Payload validation is restored, as an
argsschema on the route. #634 dropped it while the verdict was advisory, where the worst case was a wrong Slack message. That no longer holds now that the verdict publishes releases:(int) 'abc'is0, so an absent or malformedfindings_countwould read as a clean scan. Failing toward "keep waiting" costs time; failing toward "serve" quietly removes a supply-chain control.Test environment fixes (first commit) —
update_sourcewas never created,env-binwasn't mapped, andPLUGINS_TABLE_PREFIXwas undefined, which fatalledAPI\Base::load_routes()and 404'd every REST request in the test environment.Tests
25 new tests.
Release_Delay_Testcovers the delay lifecycle — including the gate itself, and that serving early preserves the manual-updates window — which had no coverage.Gandalf_Scan_Callback_Testdrives the real REST route end to end.Test plan
findings_count: 0serves the scanned version immediately and zeroesrelease_delay; the audit log names the scan.manual-updates-24hrwindow (auto-updates still gated for 24h from publication).versionno longer matches the plugin's current version is refused.findings_countreturns 400 and does not serve._gandalf_scan_last_errorand leaves the delay running.WPORG_PLUGIN_THEME_RELEASE_DELAYunset (delay0), nothing changes.Notes for review
notify_slack()still passesseverity_countsandverdict_hashthrough unsanitized (verdict_hashbecomes an unbounded post-meta array key).🤖 Generated with Claude Code
https://claude.ai/code/session_019ngq7zaYBfv2zS8dnjFoUG