Skip to content

fix(releases): grade a release over working projects first; an affected release for an active pin always surfaces - #988

Merged
runyourempire merged 2 commits into
mainfrom
fix/release-grade-active-pins
Oct 18, 2026
Merged

runyourempire merged 2 commits into
mainfrom
fix/release-grade-active-pins

Conversation

@runyourempire

@runyourempire runyourempire commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This follow-up to #983 fixes a gap in release_grade::concerned_pins that existed before #983.

The bug: the grade picked the largest version gap across every project, and the stack-change stream and the Brief only dropped scratch and dormant projects afterwards. So a scratch tree that was a major version behind made the release "major" for the scratch tree alone. The filter then removed the scratch tree, and the active project that was only a minor behind got no announcement, even when the release was malicious.

The fix:

  • New ReleaseGrade::retain_projects: narrows a grade to the chosen projects before the class and the concerned projects are computed.
  • Stack-change stream (stack_change_releases.rs):
    • It takes the newest stable release per package and computes active_grade(..) once per release.
    • It checks advisories for every release that any active direct pin is behind on, patches included.
  • An affected release is always surfaced for an active project, whatever its size. It is badged security (an id the frontend already reads) and has no Run action.
    • Its sites are every active direct pin behind the release (behind_pins), not just the largest-gap set. Coverage by the advisory lane is judged on those same copies (review B1). With release 4.2.1, project A on 4.1.0 and project B on 4.2.0:
      • a malicious 4.2.1 names both A and B;
      • an advisory that hits 4.2.1 and A's copy but not B's keeps the item, for B as well.
    • Ranking (review S2): a malicious release ranks first at High. A non-malicious affected release ranks and takes its urgency from its own grade (rank_change, a patch counts as a minor), so a runtime major stays Medium and a yanked pin still outranks it.
    • Version line (review S3): it reads npm · pkg 4.2.1 (affected), with no "installed → announced" arrow that could be mistaken for the advisory lane's "installed → fix".
  • Brief (brief_facts_build.rs): a new upgrade_grade narrows to working projects before the Breaking/Yanked test. A release that only a scratch tree is behind on therefore never takes the package's single upgrade slot.

stack_change.rs (claimed by #986): only the version-line and rank/urgency lines in release_citations and release_item change. Those hunks are away from #986's edit to security_item.

Effect on the consumers of release_grade (before and after on fixtures)

Consumer Fixture Before After
Stack-change stream scratch tree on 3.0.0, active project on 4.1.3, release 4.2.0 class Major, only the scratch tree concerned, then filtered out: nothing Minor for the active project, with npm update left-padder
Stack-change stream same, release is malware nothing security item, High, no Run action
Stack-change stream active project on 4.1.3, malicious patch 4.1.4 nothing (a patch is not news) security item, says a refresh would pull it in, no Run action
Stack-change stream A on 4.1.0, B on 4.2.0, malicious 4.2.1 A only A and B
Brief scratch tree pins a yanked 1.4.1, active project on 1.9.0, release 2.0.0 Yanked, scratch tree only, filtered out: no fact Breaking upgrade fact for the active project
Brief scratch tree pins a yanked 1.4.1, active project already on 3.0.0, release 3.0.0 a Yanked candidate that takes the package's slot no candidate
Scorer (pipeline_v2), knowledge gaps, release_changelog any unchanged unchanged

Scores do not change. Only the stack-change stream and the Brief call retain_projects; grade_registry_release and grade_pins are untouched. The scorer's pins already leave out dormant projects before grading, and keep scratch projects as a label by design. PIPELINE_VERSION needs no bump, so this does not interact with #986. Cost, as measured in review: about 49 lookups per pass, roughly 0.2 s, cached.

Follow-ups (not in this PR)

  • A malicious patch that reaches an active project only through a transitive dependency is not reported. A release row asks only the projects that declare the package; the parent walk would have to carry it.
  • The change_label and release_action arms that combine Minor | Security predate this PR. Rust needs an arm for every variant, and an affected release never reaches them because the advisory's title and advice replace them.

Tests

  • cargo test --lib -- scoring:: evidence:: brief_facts ace_commands osv:: briefing knowledge_gaps release_changelog: 1823 passed, 0 failed (35 ignored). After making release_change test-only to fix a clippy dead-code error, stack_change passes 30/30 and clippy is clean. The pre-push gate passed after rebasing onto fix(temporal): find a pin by the matcher's folded name (PEP 503, Cargo -/_), not only exact or lowercase #989.
  • New or changed tests:
    • the A/B case: malware names both, and an advisory that hits only A is kept for B;
    • a project a patch behind is named for malware but not for the clean minor;
    • an affected major keeps Major rank and Medium urgency, its version line reads (affected), and a yanked pin outranks it;
    • a scratch tree a major behind no longer hides an active minor, or malware;
    • a malicious patch surfaces;
    • Brief upgrade_grade: a scratch-only release is no candidate, and a scratch yanked pin does not mask a breaking upgrade. Reverting the narrowing makes this test fail;
    • release_grade: the before and after of narrowing.
  • The transitive-only test is dropped: it passed on the old code too.
  • cargo fmt: clean. cargo clippy --lib -- -D warnings: clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7

runyourempire and others added 2 commits October 18, 2026 22:36
…ed release for an active pin always surfaces

release_grade::concerned_pins picked the largest gap across EVERY project,
and the stack-change stream and the Brief dropped scratch/dormant projects
only afterwards. A scratch tree a major behind made the release "major" for
the scratch tree alone; the filter then removed it, and the active project
a minor behind heard nothing (a malicious release included).

ReleaseGrade::retain_projects narrows the pins before the class and the
concerned set are computed. The stack-change stream grades
active_grade(..) and checks advisories against its direct pins for every
release an active project is behind on; an Affected release is a Security
change whatever its size (a malicious patch inside a caret range is what a
refresh installs first). The Brief narrows before its Breaking/Yanked test.

The scorer (pipeline_v2) and knowledge gaps / release_changelog do not call
retain_projects: their grades and scores are unchanged, so PIPELINE_VERSION
needs no bump.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7
…it; rank by its grade (#988 review)

B1: the sites (and the coverage copies) of an AFFECTED release are every
active direct pin behind it (behind_pins), not the largest-gap set: with
4.2.1, A on 4.1.0 and B on 4.2.0, a malicious 4.2.1 named A alone, and an
advisory hitting 4.2.1 and A's copy counted as covered and dropped B's
warning.

S2: a non-malicious affected release ranks and takes its urgency from its
grade (rank_change; a patch ranks with the minors), so it never outranks a
yanked pin or loses a major's Medium; malware stays High and first.
S3: its version line reads "pkg X (affected)", no installed -> announced
arrow that would read as the fix.
S4: the Brief's narrowing is `upgrade_grade`, tested so that reverting it
fails; the transitive-only test (green on the old code) is dropped.
Nits: active_grade computed once per release (release_change_active);
"newest stable release per package".

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7
@runyourempire
runyourempire force-pushed the fix/release-grade-active-pins branch from e750f8e to d808741 Compare October 18, 2026 12:39
@runyourempire
runyourempire added this pull request to the merge queue Oct 18, 2026
Merged via the queue into main with commit d63fd7c Oct 18, 2026
24 checks passed
@runyourempire
runyourempire deleted the fix/release-grade-active-pins branch October 18, 2026 12:58
runyourempire added a commit that referenced this pull request Oct 18, 2026
…ge of #988 and #992)

#988 landed a stack_change test that builds an AdvisoryWindow literal
before #992 added the cwe_ids field, so the merge-queue build failed
with E0063. Same fixture, empty CWE list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant