Skip to content

Plugin Directory: Serve a release immediately when Gandalf reports no findings#717

Open
obenland wants to merge 3 commits into
WordPress:trunkfrom
obenland:feature/gandalf-clean-scan-serves-release
Open

Plugin Directory: Serve a release immediately when Gandalf reports no findings#717
obenland wants to merge 3 commits into
WordPress:trunkfrom
obenland:feature/gandalf-clean-scan-serves-release

Conversation

@obenland

@obenland obenland commented Jul 16, 2026

Copy link
Copy Markdown
Member

The release delay (#650) buys a window for scanners and humans to flag a bad release before it reaches sites. Gandalf (#634) fills the automated half. The two don't talk to each other, so the window is spent in full on every release — including the overwhelming majority a scanner has already cleared.

This closes the loop: when a Gandalf scan completes with no findings, the version it scanned stops waiting and starts being served. Findings still go to Slack for a reviewer. A scan that fails or never reports back leaves the delay to elapse on its own — silence never shortens the wait.

Changes

serve_release_now() replaces force_release(). Both do the same thing — clear a release's delay and write it to update_source — so a reviewer force-releasing and a clean scan now share one path, differing only in the $reason they log.

The version is checked in one place. serve_release_now() takes the version explicitly and re-checks it against the plugin's current one. Callers act on a version they evaluated earlier — a reviewer reading the metabox, a scanner reading a ZIP — and a newer commit can land in between; serving a stale version would skip the wait on a release nobody evaluated. The metabox did this check itself and force_release() didn't; now there's one implementation.

The author's manual-updates-24hr window is preserved. Serving early still dates the release from the moment it goes public, not from the commit, so phased_rollout()'s author-set window runs its full length from availability. A clean scan that arrives after the delay already elapsed leaves the served version untouched.

Payload validation is restored, as an args schema on the route. #634 dropped it while the verdict was advisory, where the worst case was a wrong Slack message. That no longer holds now that the verdict publishes releases: (int) 'abc' is 0, so an absent or malformed findings_count would read as a clean scan. Failing toward "keep waiting" costs time; failing toward "serve" quietly removes a supply-chain control.

Test environment fixes (first commit) — update_source was never created, env-bin wasn't mapped, and PLUGINS_TABLE_PREFIX was undefined, which fatalled API\Base::load_routes() and 404'd every REST request in the test environment.

Tests

25 new tests. Release_Delay_Test covers the delay lifecycle — including the gate itself, and that serving early preserves the manual-updates window — which had no coverage. Gandalf_Scan_Callback_Test drives the real REST route end to end.

Test plan

  • A callback with findings_count: 0 serves the scanned version immediately and zeroes release_delay; the audit log names the scan.
  • A clean scan serves early without shortening the manual-updates-24hr window (auto-updates still gated for 24h from publication).
  • A callback with findings leaves the release waiting and alerts Slack as before.
  • A callback whose version no longer matches the plugin's current version is refused.
  • A callback with a missing, non-numeric, or negative findings_count returns 400 and does not serve.
  • A callback without a valid bearer secret returns 401.
  • A failed scan records _gandalf_scan_last_error and leaves the delay running.
  • Reviewer force-release from the Controls metabox still works, with the reason in the audit log.
  • With WPORG_PLUGIN_THEME_RELEASE_DELAY unset (delay 0), nothing changes.

Notes for review

  • This is a policy change, not just a code change. The delay's stated purpose is a window for "scanners and humans". Ending it on a clean scan removes the human half for the releases Gandalf clears — which is the intent, but worth agreeing on explicitly, since the releases that most need human eyes are the ones a scanner missed. A shorter-but-nonzero delay on clean scans is the alternative.
  • Not addressed: notify_slack() still passes severity_counts and verdict_hash through unsanitized (verdict_hash becomes an unbounded post-meta array key).
  • Not addressed: Plugin Directory: Send plugin updates to Gandalf for advisory scans #633's contract fixtures were dropped along with its tests, so nothing on our side catches drift between the callback we accept and the callback that's sent.

🤖 Generated with Claude Code

https://claude.ai/code/session_019ngq7zaYBfv2zS8dnjFoUG

The test environment was missing three things the development environment
already had, which between them made whole classes of test impossible:

- `update_source` was never created. `after-start.sh` imports
  `database-tables.sql` for development; the test script did not, so anything
  touching the update API's table failed.
- `wp-content/env-bin` was not mapped, so `database-tables.sql` was not
  reachable from the test container to import in the first place.
- `PLUGINS_TABLE_PREFIX` was undefined. `SVN_Access` reads it at construction,
  so `API\Base::load_routes()` fatalled and *no* `plugins/v1` route was
  registered — every REST request in the test environment 404'd.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019ngq7zaYBfv2zS8dnjFoUG
@obenland
obenland force-pushed the feature/gandalf-clean-scan-serves-release branch from 6163382 to 2f0a96d Compare July 16, 2026 21:29
@obenland
obenland requested a review from Luc45 July 16, 2026 21:30
@obenland
obenland force-pushed the feature/gandalf-clean-scan-serves-release branch from 2f0a96d to 63cb850 Compare July 16, 2026 21:34
… findings.

The release delay exists to give scanners and humans a window to flag a bad
release before it reaches sites. Until now the window was spent in full on
every release, including the overwhelming majority that a scanner has already
looked at and cleared. This closes that loop: when a Gandalf scan completes
with no findings, the version it scanned stops waiting and starts being served.

`force_release()` and the new path do the same thing — clear a release's delay
and write it to `update_source` — so they collapse into one
`API_Update_Updater::serve_release_now( $slug, $version, $reason, $user )`.
What separates a reviewer force-releasing from a clean scan is recorded in
`$reason`, not in the behaviour, and every such release now logs in one
greppable format.

The version is passed explicitly and re-checked against the plugin's current
one. Both callers act on a version they evaluated earlier — a reviewer reading
the metabox, a scanner reading a ZIP — and a newer commit can land in between;
serving a stale version would skip the wait on a release nobody evaluated,
which is precisely what the delay exists to cover. The metabox did this check
itself and `force_release()` did not; now there is one implementation.

Callback payload validation is restored, as an args schema on the route. WordPress#634
dropped it on the grounds that the scan verdict was advisory, where the worst
case was a wrong Slack message. That no longer holds now that the verdict
serves releases: `(int) 'abc'` is 0, and a `findings_count` that is absent or
malformed would read as a clean scan and publish. We validate what we act on,
at our own boundary. `error.kind` is accepted as an open string rather than a
fixed enum, so a new failure kind needs no WPORG deploy.

Adds coverage for the delay lifecycle and the callback route, including the
gate itself, which had none.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019ngq7zaYBfv2zS8dnjFoUG
@obenland
obenland force-pushed the feature/gandalf-clean-scan-serves-release branch from 63cb850 to 6e11272 Compare July 16, 2026 21:35
@obenland
obenland marked this pull request as ready for review July 17, 2026 14:28
Copilot AI review requested due to automatic review settings July 17, 2026 14:28

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props obenland, lucasbustamante.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@Luc45 Luc45 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together.

I have limited context on the WPORG side, so my review will focus mostly on the Gandalf side.

Nonblocking follow-ups

There's two issues that I think we could tackle in a follow-up, that doesn't have to hold this PR.

  • Same reference, different ZIP. A ZIP can change during a scan without changing its version or release ref, so WPORG could accelerate an artifact Gandalf never scanned. Follow up by returning the scanned ZIP digest and comparing it with the current ZIP before acceleration.
  • Zero findings, incomplete coverage. Gandalf can complete with zero findings even when prompt-security could not scan everything within its budget; WPORG cannot currently tell. Follow up by returning coverage status so WPORG can withhold acceleration when coverage is incomplete.

Recent Gandalf data shows that about 99% of attempted scans complete, so I’m comfortable handling these separately.

WPORG clarification

Just to be clear, since I’m not familiar with the WPORG side, the way I understand it there are two separate cooldowns:

  • The older one, from 2025: manual-updates-24hr. From the phased rollout work in Meta Trac #8009 and PR #498, once the version is offered through the update API this allows manual updates but disables automatic updates for 24 hours.
  • The newer one, from 2026: release_delay. From PR #650, this happens first: the update API continues offering the previous version for the configured delay, leaving time for scanners and reviewers to act.

My understanding is that the intended effect of this PR is only to end release_delay early, which seems expected. I've created this diagram below to wrap my head around how those two cooldowns behave, and how this PR changes it, let me know if this looks accurate to you.

Before this PR
                                             ■ release_delay      □ manual-updates-24hr
                                        0h                      24h                     48h
                                        ├───────────────────────┼───────────────────────┤
No clean callback                       ■■■■■■■■■■■■■■■■■■■■■■■■□□□□□□□□□□□□□□□□□□□□□□□□
                                                                ↑M                      ↑A
Clean result at +2h (not acted on)      ■■■■■■■■■■■■■■■■■■■■■■■■□□□□□□□□□□□□□□□□□□□□□□□□
                                                                ↑M                      ↑A
Clean result at +26h (not acted on)     ■■■■■■■■■■■■■■■■■■■■■■■■□□□□□□□□□□□□□□□□□□□□□□□□
                                                                ↑M                      ↑A

After this PR
                                             ■ release_delay      □ manual-updates-24hr
                                        0h                      24h                     48h
                                        ├───────────────────────┼───────────────────────┤
No clean callback                       ■■■■■■■■■■■■■■■■■■■■■■■■□□□□□□□□□□□□□□□□□□□□□□□□
                                                                ↑M                      ↑A
Clean callback at +2h                   ■■□□□□□□□□□□□□□□□□□□□□□□
                                          ↑M                    ↑A
Clean callback at +26h                  ■■■■■■■■■■■■■■■■■■■■■■■■□□
                                                                ↑M↑A

Each square represents one hour. ↑M marks when the new version becomes available as a manual update to existing installations; ↑A marks when it becomes eligible for automatic updates.

That is what the code appears to do. Before merging, does this look accurate and intentional to you?

My review is limited to the Gandalf side, so it may be useful to have a WPORG reviewer confirm these points.

…ation when serving early.

Serving a release early via serve_release_now() zeroed its release_delay
before update_single_plugin() ran, which disabled the guard that anchors
release_time to the moment the version goes public. release_time fell back
to the commit time, silently shortening (or collapsing) the author-set
manual-updates-24hr window that phased_rollout() measures from it.

Anchor release_time whenever a version newly enters update_source, not only
while a delay is active, so an expedited release still dates from publication.
A clean scan arriving after the delay already elapsed now leaves the served
version — and its running window — untouched. Extracts get_served_version()
to share the update_source lookup that both paths need.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@obenland

Copy link
Copy Markdown
Member Author

I'm not sure about the "Clean callback at +26h" section, but the rest looks good. 27cddb6 is meant to make that clearer, anchoring manual-updates-24hr to release time as intended, rather than commit time, which it was implicitly. Thanks for that review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants