fix(GATE-OP-PARITY-MANIFEST): refuse a throwing golden by name instead of aborting the pass (#776)

[localai-bot]

The gate had stopped gating (issue #776)

test_op_parity THREW json.exception.type_error.302 out of the CPU golden pass instead of failing an assertion. A throw at manifest-read time aborts the whole test case, so every golden after the offender goes unchecked — including the walker's no runner for op refusal, which is the check that caught #559's missing runner arm. That is the deliverable here: restoring the gate, not clearing the red line.

Linked in three places: this body, .agents/issue-index.md, and tests/parity/goldens/README.md, which is where the walker's contract is declared. No spec is owed — this is the in-flow bug-fix path, the same one #755 took.

main moved under this branch mid-gate and #840 relocated the issue table out of roadmap_v1.md into the append-only index, so the merge commit takes roadmap_v1.md from origin/main wholesale (byte-identical to it) and reapplies the #776 row as an append to .agents/issue-index.md. The focused gate was rebuilt and rerun on the merge result.

Which manifest, and how it was found

tests/parity/goldens/minimax_music3_oracle/manifest.json — an oracle capture with no "op" key at all. Found by auditing all 66 committed manifests (it is the only one that is not an op golden), then attributed decisively: rebuilding the pre-fix source (043e56862^) reproduced the exact CI line, and moving that one directory out of the tree turned the same binary green.

tests/parity/test_op_parity.cpp:1989: ERROR: test case THREW exception: [json.exception.type_error.302] type must be string, but is null
[doctest] test cases: 10 |  9 passed | 1 failed
[doctest] assertions: 70 | 70 passed | 0 failed
[doctest] Status: FAILURE!

# same binary, minimax_music3_oracle/ moved aside:
[doctest] assertions: 123 | 123 passed | 0 failed
[doctest] Status: SUCCESS!

70 assertions against 123. The throw was hiding a third of the gate.

That half is already fixed — and it is not the whole hole

#776 duplicates #755 for its stated symptom. #755 was filed at 13:08 UTC, #776 at 16:36 UTC, and 043e56862 (PR #761) closed #755 at 16:44 UTC — eight minutes later. So the malformed manifest is repaired on main already, and test_op_parity is green there. Reporting only that would have been accurate and useless.

What #761 closed was the walker's INPUT set. It did not close the walker's EXCEPTION surface. Reproducing #776's shape on the fixed tree shows the difference. Setting one tensor dtype to null in rmsnorm_f32_8x128:

tests/parity/test_op_parity.cpp:2177: ERROR: test case THREW exception: [json.exception.type_error.302] type must be string, but is null
  logged: case rmsnorm_f32_8x128
[doctest] assertions: 37 | 37 passed | 0 failed

Identical exception, identical failure mode, 45 committed goldens never reached — on a tree where #776 was supposed to be fixed. Two throw sites remained: json::parse on the manifest, and any runner reading a field out of it.

The fix

Both sites funnel through GuardGoldenStage, which converts a std::exception into a FAIL_CHECK naming goldens/<case>/manifest.json, quotes the original exception so the diagnosis survives, and continues to the next golden. The dispatch chain moves into RunGoldenCase so its continue statements become return false — that is what lets one case be caught without unwinding the pass. The sibling #755 audit case gets the same guard, since it reads every committed manifest and must not stop at the first unparsable one either.

std::exception is the deliberate width: it covers json's type_error and parse_error and the std::runtime_error that LoadTensor raises for a shape or dtype-size mismatch — none of which name a file the caller already knows. doctest's TestFailureException is not derived from std::exception (third_party/doctest/doctest.h:2563), so a REQUIRE or FAIL inside a runner still aborts as before. That is load-bearing rather than incidental: the no runner for op refusal is a FAIL, and a guard that swallowed it would have made this red go away while quietly removing the gate.

Mutation evidence

Tree restored byte-for-byte after each (823 files, md5sum -c clean).

Mutation	Result
rmsnorm_f32_8x128 tensor dtype: null	ERROR: goldens/rmsnorm_f32_8x128/manifest.json: the runner for op "rmsnorm" threw "[json.exception.type_error.302] ..." — 45 op cases still ran, 152 assertions still checked (was: THREW, 37)
Manifest that is not valid JSON	ERROR: goldens/mutation_unparsable/manifest.json: parsing manifest.json threw "[json.exception.parse_error.101] ..." — 46 op cases still ran, no THREW anywhere
{"op": null}	ERROR: goldens/mutation_null_op/manifest.json declares a non-string "op" (null) — pass continues
{"op": "no_such_runner_op_559"}	FATAL ERROR: no runner for op 'no_such_runner_op_559' — add one before committing goldens

The last row is the one that matters most. The #559 guard still bites, and still aborts, exactly as it did before. Without it, the first three rows would only prove the red was silenced.

A new test case drives the same properties without touching committed evidence: the raw type_error.302 names no file, the guarded form names the case and preserves the original what(), a case after the offender still runs, and TestFailureException propagates through the guard rather than being swallowed.

Green

• Focused, clean tree: 13 test cases, 153 assertions, SUCCESS — 46 op cases ran, 1 non-op skip, identical to the pre-change counts, so the RunGoldenCase extraction did not change what is gated.
• sanitize-cpu (address,undefined) lane, built with the job's own flags and run under its env (VT_POOL_BYPASS=1, ASAN_OPTIONS, UBSAN_OPTIONS): SUCCESS. The main RED on sanitize-cpu: test_op_parity throws json type_error.302 — a golden manifest field is null where a string is required #776 half of that lane's red is closed. test_nemotron_h_scaffold (main RED on sanitize-cpu: ForwardNemotronHForCausalLM static_casts a LoadedModel that is not a NemotronHLoadedModel #775) is a separate agent's fix and still red there.
• scripts/agent-preflight.sh: green.
• test_cpu_x86_llamacpp_floor failed once under contention from the other agents on this box and passes standalone (10/10) — the documented load-sensitivity, not a real failure.

The unguarded-read shape elsewhere

Surveyed deliberately. Within test_op_parity.cpp there are ~167 unguarded m.at(...) / m[...] reads inside runners; they are the residual this change covers generically rather than site-by-site, since the guard catches whatever any of them throws and names the manifest. Elsewhere in tests/, the other json::parse call sites read a single named fixture owned by their own test case — a throw there fails one case whose fixture is already named, not a walker over an open input set. The walker-shaped risk, where one bad file silences everything behind it, was unique to RunGoldenPass and is now closed. No follow-up issue is owed.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [Claude Code]