fix: correct the inverted EEVEE engine id and close the coverage gap that hid it - #204
Merged
Merged
Conversation
eevee_engine_id() keyed on >= (4, 2, 0) and returned 'BLENDER_EEVEE_NEXT'
for every version at or above it. Blender 5.x reclaimed the plain
'BLENDER_EEVEE' id and no longer offers 'BLENDER_EEVEE_NEXT', so the
--output path died on 5.1 and 5.2:
TypeError: bpy_struct: item.attr = val: enum "BLENDER_EEVEE_NEXT" not
found in ('BLENDER_EEVEE', 'BLENDER_WORKBENCH', 'CYCLES')
Smoke stayed green because smoke never passes --output.
The correct mapping is already witnessed in the tree: examples/swatch-grid
asserts both the chosen id and the other era's id against the running
build. Use its form and name it as the source, per the copied-not-imported
convention the hygiene helpers already follow.
shipping-crate is the pilot piece the other 27 were modelled on, which is
how one inverted ternary propagated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: TMHSDigital <154358121+TMHSDigital@users.noreply.github.com>
Second copy of the same inversion, found by auditing every engine-id mapping in the tree rather than by grepping for the one known bad file. Audit method: extract every expression that yields a BLENDER_EEVEE* id and evaluate it twice, against a stubbed bpy.app.version of (4, 5, 11) and (5, 2, 1), then compare both results against the canonical mapping. 83 expressions across 85 files; two returned 'BLENDER_EEVEE_NEXT' for 5.2 — shipping-crate and this one. Everything else was already correct. Like shipping-crate, its --output path raised TypeError on 5.1 and 5.2 and smoke could not see it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: TMHSDigital <154358121+TMHSDigital@users.noreply.github.com>
The render path is invisible to CI by construction: smoke runs the default
check path, and --output is exercised only at authoring time, on one
version, by the author. That is how an inverted engine id shipped twice.
tests/smoke/run_smoke.py already asserts the mapping against the running
build, but against its own copy of the helper. It cannot see the other
copies, and showcase and example scripts are standalone by convention, so
each carries its own. Eighty-four mappings across eighty-three files.
This checker closes that gap at zero render cost. It walks examples/,
showcase/, templates/, snippets/, scripts/ and tests/, and for every
function that resolves an EEVEE id it compiles and *calls* the function
twice against a stubbed bpy.app.version of (4, 5, 11) and (5, 2, 1);
conditional expressions yielding an id are evaluated the same way.
AST rather than regex, deliberately. A line-oriented regex misreads the
if/return form as two independent branches and flags
snippets/version-branch-skeleton.py, which is correct. Calling the code is
the only reading that cannot be fooled by formatting.
Resolver detection is narrow on purpose: only a body of returns and
version branches qualifies, so a main() that merely mentions the id is not
compiled and does not need the module's imports.
Three deliberately inverted mappings exist — swatch-grid, turntable and
gn-sdf-remesh each assert that the *other* era's id is rejected by the
build. Those now carry an explicit `# engine-id-exempt:` marker, so the
intent is stated in the source rather than inferred by the checker.
Canary-proven. With the shipping-crate inversion reintroduced:
ERROR: showcase/shipping-crate/shipping_crate.py:103 eevee_engine_id()
5.2.1->BLENDER_EEVEE_NEXT (want BLENDER_EEVEE)
RC=1
Reverted:
engine-id checks passed: 84 mapping(s) across 83 file(s), 3 exempt.
RC=0
Runs in validate.yml beside the other tests/ checkers. No smoke cost, no
blender-smoke.yml change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: TMHSDigital <154358121+TMHSDigital@users.noreply.github.com>
showcase/README.md said nothing about either gate, and no showcase piece has ever shipped one — all 44 contact sheets and all 4 asset sheets in docs/gallery/ belong to examples. The gates live under CLAUDE.md "Quality Gates for Example Runs", so the next author had to guess. Both are now required, each on its own evidence rather than on symmetry. Contact sheets have caught a real defect in showcase work: the first sheets for crate-stack and stone-archway showed both wedge pools reading as cool grey bands instead of the warm pool the house style calls for, and both pieces were relit. Nothing else in the pipeline puts the still beside its peers. Asset sheets cover what nothing else here covers. Showcase is entirely game props, which is exactly the scope docs/VISUAL-STYLE.md names for the gate. Budgets measure geometry conformance; the contact sheet measures staged presentation; neither removes the scene. The recorded evidence is socket-attach-points, which passed every measurable floor on its first draft (edge90 0.000, ten materials, no default names) and was still judged bad by eye and rebuilt — "the floors scored the bevels, not the design". That is precisely the failure a showcase budget cannot see. Also records the exit-code collision: check_asset_quality returns 11, which showcase numbering already spends on the collider ceiling, so call sites remap rather than sharing a code. Existing pieces are not retrofitted here; the backlog is filed separately. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: TMHSDigital <154358121+TMHSDigital@users.noreply.github.com>
Five falsifiers in the previous showcase run had to be retuned because they tripped an earlier check instead of the budget they were built for. A falsifier that exits non-zero on an earlier budget is red for the wrong reason and proves nothing about its target, and the exit code is right there to catch it. Documents the rule in CONTRIBUTING.md and showcase/README.md, with --off-circle replacing --flat-arch as the worked example: a lintel is 0.62 m shorter than the arch, so it failed the bounding box (exit 8) and never reached the intrados circle fit (exit 19). Widening the bbox tolerance would have destroyed a real budget to rescue a bad falsifier. The rule is explicit that the fix is the model or the falsifier, never the band. tests/check_falsifier_targets.py enforces it in two modes, one checker rather than a parallel harness: - static (default, no Blender, runs in validate.yml): reads each piece's falsifier table and asserts the flags are real argparse flags, the declared exit codes appear in that piece's exit-code table, every falsifier names a target budget, and no falsifier-shaped flag in the script is undocumented. 160 falsifiers across 25 pieces. - --run BLENDER: executes each falsifier and asserts the observed exit equals the declared one. This is the mode that catches an ill-aimed falsifier. One Blender launch per falsifier, about 150 s for the tree on one version, so it is an authoring and cron tool rather than a per-PR smoke step. No blender-smoke.yml change. Table parsing is header-aware, not positional: wooden-ladder's four-column "Falsifier | Budget violated | Exit | Measured failure" is as valid as the three-column form, and an earlier positional regex silently read it as having no exit column at all. The bidirectional variant — requiring each exit-code row to name its falsifier — was tried and dropped. It would have forced a format change across 23 READMEs to state something the falsifier table's own target column already carries. cart, hay-bale and stone-well predate the convention and have no table. They are listed explicitly in the checker and reported on every run; a new piece without a table is an error rather than an entry in that list. Canary-proven at runtime. Declaring --off-circle as targeting the bbox budget: MISMATCH stone-archway --off-circle: want 8, got 19 RC=1 Reverted: ok stone-archway --off-circle: want 19, got 19 falsifier-target checks passed: 167 falsifier(s) across 25 pieces. RC=0 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: TMHSDigital <154358121+TMHSDigital@users.noreply.github.com>
The estimate said roughly 150 s. Running the full sweep measured 276 s for 160 falsifiers across 25 showcase pieces on Blender 5.2.1, with zero mismatches — every declared exit code is the one the falsifier actually produces. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: TMHSDigital <154358121+TMHSDigital@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A live bug, the coverage gap that hid it, and two convention gaps the last loop surfaced. Six commits, one per item.
Binaries:
.scratch/blender-4.5.11-windows-x64/blender.exe(4.5.11 LTS),.scratch/blender-5.1.2-windows-x64/blender.exe(5.1.2),.scratch/blender-5.2.1-windows-x64/blender.exe(5.2.1 LTS).Part A1/A2 — the inverted engine id, twice
showcase/shipping-cratecarriedeevee_engine_id()keyed on>= (4, 2, 0), returningBLENDER_EEVEE_NEXTfor everything above it. Blender 5.x reclaimed the plainBLENDER_EEVEEid and no longer offersBLENDER_EEVEE_NEXT. Reproduced verbatim on 5.1.2:After the fix, on 5.1.2:
shipping-crate exit=0,iron-cauldron exit=0.The audit found a second copy.
showcase/iron-cauldronhad the identical inversion. It was not in the brief; it was found by auditing every mapping rather than grepping for the one known file.Audit method
Not a regex. Every expression that yields a
BLENDER_EEVEE*id was extracted and evaluated twice, against a stubbedbpy.app.versionof(4, 5, 11)and(5, 2, 1), then compared against the canonical mapping fromexamples/swatch-grid:A line-oriented regex is not good enough here: it reads the
if/returnform insnippets/version-branch-skeleton.pyas two unrelated branches and flags a file that is correct.The table
84 mappings across 83 files. Everything not listed below evaluates correctly on both eras.
showcase/shipping-crate/shipping_crate.py7cce965)showcase/iron-cauldron/iron_cauldron.pyb0a23c4)examples/swatch-grid/swatch_grid.pywrong = …)# engine-id-exempt:examples/turntable/turntable.pywrong = …)# engine-id-exempt:examples/gn-sdf-remesh/gn_sdf_remesh.pywrong = …)# engine-id-exempt:snippets/version-branch-skeleton.pyif/returnform, not a ternary)tests/smoke/run_smoke.pytests/smoke/tmpl_render.pyThe three deliberate inversions are examples that assert the other era's id is rejected by the running build. They now say so in source rather than relying on the reader.
Both fixes carry the provenance header the copied-helper convention already uses for the hygiene combinatorics, naming
examples/swatch-grid.get_eevee_engine_idas the source.Report item 2 — a committed still is inconsistent with its code
Checked because the brief asked. The answer is yes, and it is broader than these two pieces. Filed as #200; not fixed here.
Method: render
--outputto lossless PNG on 5.2.1, compare against the committed hero.Control — how much the webp encode alone accounts for:
So ~0.5 % is the noise floor. Measured:
crate-stack(hero rendered yesterday, direct to webp)wooden-ladderiron-cauldronanvilshipping-crateIt is not renderer era. The obvious hypothesis — old stills rendered on 4.5 with EEVEE Next — is ruled out. The same piece rendered on both binaries agrees with itself to within the noise floor:
Committed heroes are also systematically brighter (mean luminance +0.0104 on shipping-crate) and ~1.85× larger on disk. That points at a separate PNG→webp conversion step at a different quality and colour handling —
.scratch/still carriesto_webp.py,to_webp_h.py,ads_webp.py.This does not undermine the engine-id fix: the
TypeErroris independently reproducible. Both are true.Part A3 — coverage, with the numbers
Measured render cost
showcase/shipping-crate --output, whole process including the check path:Check-only baseline: 1 762 / 2 115 / 1 625 ms.
Chosen: option 2, the static mapping check — and why not the canary
The cheap canary does not cover the class, and the covering canary is not cheap:
blender-smoke.ymlsays so itself: "Cycles (CPU) so this is reliable on GPU-less runners", andrun_smoke.py: "an EEVEE GPU render aborts the process (no EGL)".So a render canary buys nothing for the motivating class at either price.
tests/check_engine_id.pycovers it completely, for free, across all 84 mappings — including the 82 a canary on one piece would never touch. The remainder of the render path is #201, with these numbers.No
blender-smoke.ymlchange. The checker runs invalidate.yml.A3 canary, verbatim
Inversion reintroduced in shipping-crate:
Reverted:
Part B — both gates required, each on its own evidence
Applied the rule in the brief rather than a preference.
Contact sheets: required. They have caught a real defect in showcase work. The first sheets for
crate-stackandstone-archwayshowed both wedge pools reading as cool grey bands instead of the warm pool the house style calls for; both pieces were relit as a direct result. Nothing else in the pipeline puts a still beside its peers.Asset sheets: also required — the brief's escape clause applies. Reading the definition shows they catch something budgets cannot.
docs/VISUAL-STYLE.md§ Asset quality scopes the gate to "asset-type examples (game props, kits)", and showcase is entirely game props. Budgets measure geometry conformance; the contact sheet measures staged presentation; neither removes the scene. The recorded evidence is decisive:A showcase budget cannot see that failure. The asset sheet is the only gate here that can, so marking it "not applicable" would have been the comfortable answer rather than the correct one.
Also recorded:
check_asset_qualityreturns 11, which showcase numbering already spends on the collider-triangle ceiling. Call sites remap rather than sharing a code.Not retrofitted here. 26 of 28 pieces lack a contact sheet, 28 of 28 lack an asset sheet — #202, which is explicitly blocked on #200, because building a contact sheet from a drifted hero calibrates against the wrong image.
Part C — falsifiers must fail the budget they target
Documented in
CONTRIBUTING.mdandshowcase/README.md, with--off-circlereplacing--flat-archas the worked example, and the rule stated plainly: fix a collision by changing the model or the falsifier, never by widening a band.tests/check_falsifier_targets.py, one checker in two modes rather than a parallel harness:validate.yml) — flags are real argparse flags, declared exit codes exist in that piece's exit-code table, every falsifier names a target budget, no falsifier-shaped flag is undocumented. 160 falsifiers across 25 pieces.--run BLENDER— executes each falsifier and asserts the observed exit equals the declared one. This is the mode that catches an ill-aimed falsifier.Table parsing is header-aware, not positional:
wooden-ladder's four-columnFalsifier | Budget violated | Exit | Measured failureis as valid as the three-column form, and a positional regex read it as having no exit column at all.One design dropped on evidence. Requiring each exit-code row to name its falsifier would have forced a format change across 23 READMEs to restate what the falsifier table's own target column already carries. Replaced with the structural requirement that every falsifier row names a target budget — which the existing tables already satisfy.
Applied to existing pieces
25 of 28. Three —
cart,hay-bale,stone-well— predate the convention and have no table; they are named in the checker, reported on every run, and filed as #203. A new piece without a table is an error, not an entry in that list.Full runtime sweep, 5.2.1
Every one of the 160 declared exit codes is the code the falsifier actually produces. No existing piece was mis-aimed.
Part C canary, verbatim
--off-circlere-declared as targeting the bbox budget (exit 8):Reverted:
Constraints honoured
No budget band, exit-code value, or witness changed.
gallery_framing.py,LICENSE,VERSION,CHANGELOG.md,release.ymlandpages.ymluntouched.blender-smoke.ymluntouched — the chosen coverage did not need it.Issues filed
--outputrender path has no CI coverage beyond the engine idcart,hay-baleandstone-well🤖 Generated with Claude Code