fix: report stale structure baselines, and correct three of them (#227) - #234
Merged
Merged
Conversation
The collapse check only ever fires on a DROP, so a baseline sitting far below what a saver really produces is invisible. Adds a symmetric staleness report and, on its first run, it found three. WHAT IT FOUND ON MAIN Starfield Warp baseline 0.0011 measured 0.1423 129.4x Mandelbrot baseline 0.0020 measured 0.0557 27.8x Voronoi baseline 0.0341 measured 0.1557 4.6x All three are leftovers from the rewrites in #212, #210 and #211. Each rewrite made its saver denser, the check only fails on drops, so nothing complained. THIS CHANGES WHAT #227 IS ABOUT Starfield Warp was named in that issue, alongside Plasma, as a saver too dim to protect -- its baseline sat under STRUCTURE_MIN_ABS_DROP. That was wrong. It is not dim: it measures 0.1423, two orders of magnitude above the margin. Its baseline was simply stale, left behind by the saver it replaced (thirty white dashes on navy). Correcting it moves Starfield Warp into full protection and shrinks the known blind spot from two savers to one -- Plasma, whose 0.0001 is genuine. A test now asserts that Plasma is the only sub-margin saver, so this cannot quietly grow again. REPORTED, NOT FAILED A stale baseline is a defect in the check's configuration, not in the frame under test. Failing on it would turn every unrelated PR red until somebody fixed a saver they had not touched. The exit code comes solely from the fail= count, which this does not touch -- verified against scripts/shader-check.mjs. THE RATIO IS CALIBRATED, NOT PICKED 4x. 2x was tried first and flagged two savers whose own measurement is bimodal at frames 5/12: Raymarch Fractal read 0.0082 in one run and 0.0185 in the next, and Reaction Diffusion likewise 2.3x. That is the effect #192 was filed over, and the threshold has to clear the noisiest saver rather than the median. 4x still catches every real case -- Truchet's historical gap was 4.1x, and the three above. A REJECTED APPROACH, MEASURED #227 also asks for the flat-gradient case: a smooth ramp passes for a sub-margin saver, because it has few edges and a wide spread, and so does a healthy Plasma frame. Coarse-stride edge density was tried -- a full-frame ramp differs by stride/width between samples, so it scores zero at strides 1, 8 and 32 alike, while a real image should not. It does separate them, but not by enough to ship: the lowest healthy score is DVD Logo at 0.0026 (stride 8), so any floor that catches a ramp sits within 2.6x of a real saver, and it only catches the slowest gradients -- a steeper one scores normally. Instrumentation removed. That part of #227 stays open.
This was referenced Aug 18, 2026
BernardJen
added a commit
that referenced
this pull request
Aug 18, 2026
…heck (#235) (#251) A flat gradient passed for Plasma. Edge density could not see it -- Plasma's baseline is 0.0001, so a drop to zero is smaller than the absolute margin -- and #229's spread check catches a UNIFORM frame, not a smooth ramp. To both of those statistics a gradient and a healthy Plasma frame are the same picture: near-zero edges, wide luminance spread. WHAT SEPARATES THEM A gradient is MONOTONIC. Its signed slope along a row never reverses; a real image, however smooth, reverses constantly. Measured across all 30 savers: healthy Plasma 0.5948 radial gradient 0.0586 (one reversal per row) linear ramp 0.0000 (none, by construction) Threshold 0.12: a 5.0x margin under the healthy value and 2x above the radial case. TWO STRIDES, AND THE MAX Falling Sand reads 0.5833 at stride 4 and 0.0000 at stride 16 -- its grains are finer than the coarse step, so one stride alone would call a sparse saver monotonic. Taking the max of strides 4 and 16 fixes that. GATED TO SUB-MARGIN SAVERS, WHICH IS PLASMA ALONE Not applied set-wide, and that is a deliberate limit rather than laziness. The lowest undulation among savers with a non-zero baseline is Julia Set at 0.1544, which clears 0.12 by only 1.3x. A set-wide floor would therefore sit within a third of a healthy saver -- the same thin margin that made the coarse-stride approach unshippable in #234, and no more acceptable here. VERIFIED BY FAULT INJECTION, NOT BY ELIMINATION Each case names the rule that fired: linear ramp fail=5 "frame is monotonic: undulation 0.0000 is below 0.12" radial gradient fail=5 "frame is monotonic: undulation 0.0586 is below 0.12" uniform grey fail=5 caught by the #229 spread rule, as intended clean ok=150 fail=0, Plasma undulation 0.5948 A SELF-INFLICTED ERROR WORTH RECORDING The first measurement read 0.0000 for all thirty savers, which looked like a finding and was a bug: the block sat AFTER lums.sort(), and a sorted array is monotonic by definition. The file already warned about this for edge density. The computation is now before the sort, with a comment saying why it must be. AND A HARNESS BUG FOUND WHILE VERIFYING The failure reason never reached CI output. scripts/shader-check.mjs extracts console lines with a per-line regex against Chrome's INFO:CONSOLE(n] "..." format, so a MULTI-LINE message is dropped entirely -- its first line has no closing quote and the rest have no prefix. The FAIL log was multi-line, so every failed run has been printing "FAILING SAVERS: <name>" with no reason attached, and the reason could only be recovered by reading the page. Now emitted on one line. That bug is why this commit can say which rule caught which fault instead of inferring it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the baseline-staleness half of #227. Found three real problems on its first run.
What it found on
mainAll three are leftovers from the rewrites in #212, #210 and #211. Each rewrite made its
saver denser; the collapse check only fires on a drop, so nothing complained.
This changes what #227 is about
I named Starfield Warp in that issue, alongside Plasma, as a saver too dim for the check
to protect — its baseline sat under
STRUCTURE_MIN_ABS_DROP. That was wrong.It isn't dim. It measures 0.1423, two orders of magnitude above the margin. Its
baseline was simply stale, left behind by the saver it replaced — thirty white dashes on
navy. Correcting it moves Starfield Warp into full protection and shrinks the known
blind spot from two savers to one: Plasma, whose 0.0001 is genuine.
A test now asserts Plasma is the only sub-margin saver, so this cannot quietly grow
again.
Reported, not failed
A stale baseline is a defect in the check's configuration, not in the frame under
test. Failing on it would turn every unrelated PR red until somebody fixed a saver they
had never touched. The exit code comes solely from the
fail=count — verified by readingscripts/shader-check.mjs, which matches onlySHADERCHECK_DONE ok=(\d+) fail=(-?\d+).The ratio is calibrated, not picked
4×. I tried 2× first and it flagged two savers whose own measurement is bimodal at
frames 5/12: Raymarch Fractal read 0.0082 in one run and 0.0185 in the next (2.3×),
Reaction Diffusion likewise 2.3×. That is exactly the effect #192 was filed over, and a
threshold has to clear the noisiest saver rather than the median.
4× still catches every real case — Truchet's historical gap was 4.1×, and the three above.
A rejected approach, measured rather than argued
#227 also asks for the flat-gradient case: a smooth ramp passes for a sub-margin saver,
because it has few edges and a wide spread — and so does a healthy Plasma frame.
I tried coarse-stride edge density. A full-frame ramp differs by
stride/widthbetweentwo samples, so on a 1200-wide frame it scores zero at strides 1, 8 and 32 alike, while
a real image should not. It does separate them — but not by enough to ship:
a ramp sits within 2.6× of a real saver — one saver getting slightly sparser turns CI
red;
would not reliably catch the Physarum-class bug it was aimed at.
Instrumentation removed. That part of #227 stays open, with the negative result recorded
so nobody re-derives it.
Verification
main(7c5aa56).npm run lintclean.npm test— 407 passed (400 + 7 new staleness tests).npm run shadercheckbefore the baseline fix: 3 stale reported,fail=0.npm run shadercheckafter: stale list empty,ok=150 fail=0.Mandelbrot 0.0557 vs 0.055, Voronoi 0.1558 vs 0.15.
Not verified
bimodal by more than 4×, it will produce a spurious warning — visible, and not CI-breaking.