Repository navigation
test: verify dependency snapshots on clean temporary base - #2495
Closed
groupthinking wants to merge 1 commit into
Closed
groupthinking wants to merge 1 commit into
groupthinking wants to merge 1 commit into
Conversation
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
✅ E2E Test Results: ALL TESTS PASSED
Test Output |
Contributor
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.OpenSSF Scorecard
Scanned Files
|
This was referenced Oct 28, 2026
groupthinking
added a commit
that referenced
this pull request
Oct 29, 2026
## Summary - Fetch the dependency diff from GitHub's compare API after the existing review action, avoiding GitHub's template-object limit on large action outputs. - Save the raw response and a separate record with the exact base/head commits, review outcome, count, and SHA-256 digest. Missing, malformed, snapshot-warned, or graph-inconsistent API responses fail before evidence is saved. For changed npm lockfiles, verify returned package names, versions, and scopes against the exact base/head lockfile revisions. - Treat a change only to the dependency review workflow as a code change with no dependency graph delta. CI tests still validate the workflow. Actual manifest and lockfile changes still run dependency review. - Keep the existing vulnerability and license policy unchanged. Closes #2485 ## Paths changed - `.github/workflows/dependency-review.yml` - `scripts/ci/record_dependency_review.py` - `tests/unit/test_dependency_review_workflow.py` ## Validation - Focused `tests/unit/test_dependency_review_workflow.py` suite — **19 passed** on the final head, including warned empty/partial responses, a warning-free empty response, graph drift rejection, and a real Git rename regression. - `uvx ruff check scripts/ci/record_dependency_review.py tests/unit/test_dependency_review_workflow.py` — passed. - `git diff --check` — passed. ## Hosted validation and limits The current head is `148cbf55fdce412255d2a61780459143521fa4c4`. Its hosted checks are being verified on that exact head; see the live check list for final status. The dependency-review job passes through its no-delta path because this PR changes no dependency manifest or lockfile. Its hosted run tests the code-only path, while a real dependency PR must still validate the evidence recorder against a warning-free comparison. The Vercel Agent Review found that Git rename detection could omit an old manifest path; `--no-renames` and a real Git rename regression test address that finding. The review thread is resolved. An earlier [hosted artifact on superseded head `80fdf2b`](https://github.com/groupthinking/EventRelay/actions/runs/40657379137) contained a raw 276-entry comparison whose bytes matched its recorded digest. GitHub also warned that the base had 3 dependency snapshots and the head had 1. That comparison may have unexpected removals and is **not complete evidence** under the new guard. The prior local trial against #2479 saved 1,390 raw changes before snapshot warnings were checked; it likewise cannot establish a complete comparison. ## Live recorder check and graph scope limitation At the current code head, the local recorder rejects the live, warning-free #2495 comparison **without writing evidence**, because its removed root fast-uri 3.1.7 entry is labeled `runtime` while the exact base lockfile marks it `dev: true`. It also rejects current `main` versus #2452 before writing evidence because GitHub reports a 3-base/1-head snapshot warning. In contrast, it accepts controlled off-main [#2496](#2496): the v0.2.0 comparison has only the intended root fast-uri 3.1.7 removal and 3.1.9 addition, both `development`, and the saved raw bytes match the recorded SHA-256. #2496 was closed unmerged. The older v0.1.6 component-detection action caches packages by PURL and can apply one development flag across manifests. The v0.2.0 action used on #2496 maps each manifest's scope separately; that version is proposed in stacked draft #2492. The new recorder validation reduces the risk of accepting a wrong npm lockfile scope or a change attributed to an unchanged manifest. It does **not** establish complete snapshot parity or accurate non-npm package metadata for real updates against current `main`. Inspect the full comparison, including manifests, packages, scopes, vulnerabilities, and licenses, before a real dependency PR verdict. A fresh compare for #2452 still reports the base 3/head 1 snapshot mismatch. Once this guard is on `main`, that update will require complete matching snapshots before its dependency evidence can pass. The previously approved per-file Python scan timeout and declared-dependency fallback are unchanged. Python fallback can omit transitive dependencies when a requirements scan times out.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Result — failed off-main diagnostic, closed unmerged
The temporary base push run 40673726423 completed and submitted a full-tree snapshot for
15ec6562630fd8362a694f2b872946d6063e3040. The draft PR run 40674075576 submitted a head snapshot forb358718206353a5479554cee1bc36c39762672f6and the dependency-review action displayed only the expectedpackage-lock.jsonfast-uri 3.1.7 → 3.1.9 replacement, with no detected moderate-or-higher vulnerability or denied package.The independent compare API still returned “No snapshots were found for the head SHA” more than five minutes after submission. It returned the two expected change records, but the snapshot warning means the result cannot be accepted as complete evidence. The evidence recorder rejected it and uploaded no artifact. The dependency-review check failed.
Hosted checks finished 23 passed, 12 skipped, 5 failed. Three other failures (
test (Python 3.11),test (Python 3.12), andGenerate and Upload Coverage) came from the deliberate temporary push-trigger widening, which violated the repository guard that limits push workflows tomain/develop; this trigger was solely for the diagnostic branch. The issue-closing check failed by design because the canary does not close #2490.This branch-to-branch test does not establish what will happen against
main; whether the non-default base affects snapshot indexing is unverified. The canary is closed without merging. Issue #2490 and draft fix #2492 remain open, and no warning-free real update againstmainhas been proven.Original diagnostic plan
Diagnostic purpose
Refs #2490 and draft fix #2492. Temporary draft test; do not merge. This tests the full-tree dependency submission proposal against a newly created base SHA, separately from the inconsistent
mainsnapshot state.The base branch at
15ec6562630fd8362a694f2b872946d6063e3040contains #2487's evidence recorder, #2492's full-tree scan change, and a temporary push trigger for this base branch. The head atb358718206353a5479554cee1bc36c39762672f6changes onlypackage-lock.json, updating fast-uri 3.1.7 to 3.1.9 with the same three-line lockfile delta as bot PR #2452. The temporary branch trigger and its unit expectation will not be merged intomain.Validation and required readback
This experiment does not establish that the current
maindependency graph is clean; #2490 remains open until a real update againstmainis verified.