Navigation menu icons: correct the stale "not scriptable" claim, and make row 8 judge the menu

CHANGELOG.md

@@ -16,6 +16,60 @@ three commits past it), and a bug report can name a release instead of a sha nob
 Sections dated before 2026-09-19 predate the cycle and stay as they are.
 
 ## Unreleased
+- learn(ui-loop, learned-detection-gaps): **the screenshot harness is an instrument, and nobody
+  scores it.** Two harness defects on the PRD benchmark's Arm A produced screenshots that were
+  about to be written up as app defects. (1) The harness loaded each page at 1280 and then called
+  `setViewportSize({width:390})` for the second shot; Atlas's off-canvas sidebar region does not
+  re-collapse on resize, so **all seven** phone-width shots came back with the navigation drawer
+  open over a dimmed page — and an entire review round's verdicts were wrong in both directions,
+  the 1280 shots called broken when they were fine. Re-shot from a context *created* at 390, the
+  same pages render a collapsed hamburger and full-width content. (2) At 390 the overview grid is
+  Atlas `hide-phone`: the desktop control measures 0×0 and the control a user taps lives **outside**
+  the grid container, so a container-scoped selector passes at 1280 and times out silently at 390.
+  `ui-loop.md` gains "The screenshot harness is an instrument too" (one width per run set at
+  context creation, never resize a loaded page; target the visible twin, not the container; and
+  the tell — a defect on 7 of 7 screens at exactly one width is the camera, a real layout defect
+  hits one or two). `learned-detection-gaps.md` gains the matching operating rule: rung 6, which
+  the ladder calls "the only rung that verifies *behaviour*", is only as true as the harness that
+  reads it. The high-code arm of the same benchmark never hit either defect because its harness
+  made a fresh context per screen per width — a harness difference, not a platform difference, and
+  it nearly became a finding about Mendix. — PRD benchmark, Arm A (Maurits Visser)
+- learn(learned-detection-gaps): **two new register rows from the PRD benchmark.** `Title = null`
+  as an XPath retrieve constraint passes `check --references`, passes exec, and is **evaluated by
+  mxcli's own OQL engine**, which returns rows — so even a read-back looks right; real mxbuild
+  rejects it with CE0161. The lesson is sharper than the fix (`not(Title)`): mxcli's query engine
+  is more permissive than the model loader, so "I ran the query and it worked" is not evidence a
+  constraint is legal. Second row: a layout migrated with `ALTER PAGES … WHERE LAYOUT = X` is
+  silently reverted by any later `create or replace page` that hardcodes the old `Layout:`, because
+  that statement rewrites the page wholesale. It is green through **every rung including live
+  runtime** — the app loads, every page renders, every journey passes — and the app ships two
+  navigation shells at once. Found when 4 of 7 screens turned out to have no navigation at all,
+  just a bare hamburger, after a UI fix loop re-ran five older page scripts. The migration is not
+  done until the `Layout:` literal is fixed in the source scripts. — PRD benchmark, Arm A (Maurits Visser)
+- fix(module-review): **rubric row 8 judged the nav *bar* but never the nav *menu*.** The row
+  already caught an un-skinned default nav bar and a sidebar design shipped as a top-bar app; it
+  said nothing about the navigation profile's own items. The menu is chrome on every screen and
+  belongs to no page, so a page-by-page LOOK pass structurally never reaches it — Arm A of the PRD
+  benchmark passed a full module review and two UI sweeps with five bare text labels and no icons.
+  Row 8 now names the menu items (labels, icons, order, grouping) as a finding class, says to judge
+  them once explicitly on the first screen opened, and points at `learned-sidebar-collapse-icons.md`
+  so the corrected icon capability is reachable from the review rubric and not only from a build
+  skill. — PRD benchmark, Arm A (Maurits Visser)
+- fix(learned-sidebar-collapse-icons): **the skill told sessions that menu-item icons cannot be
+  scripted — they can, and have been able to for some time.** The file asserted that
+  `navigation.create` "has no menu-item icon token" and that per-item icons "have to be assigned
+  by hand in Studio Pro". `mxcli syntax navigation.create` on **v0.22.0** documents
+  `MENU ITEM 'Label' PAGE Module.Page ICON Atlas_Core.Atlas."name"`, the quoting rule for
+  hyphenated Atlas names, and `SHOW / DESCRIBE ICON COLLECTION` to browse the 366 stock icons.
+  Cost of the stale claim: the PRD low-code-vs-high-code benchmark's Arm A shipped five text-only
+  navigation items through a full build and two UI sweeps, because the skill said icons were not
+  scriptable so nobody re-probed — the user's own verdict on the shipped nav was "lots of text ugly
+  stuff". Corrected in place with the probe command, a worked example, and a note that this is the
+  **capability-probe rule** failing precisely where it hurts most: a general prior written into a
+  skill file is the most convincing general prior there is. Also split the two halves that were
+  conflated — `ICON` places the glyph (MDL), icon-only-when-collapsed is theme CSS (and the class
+  names must be read from your own `themesource/atlas_core/`, never copied from another project).
+  — PRD benchmark, Arm A (Maurits Visser)
 - docs(existing-app-change): **Stage 0 in the change-an-existing-app mode now asks in plain words.** "What do you want to work on", "do you have input documents", "what else does it touch" replace slice / blast radius in the skill, intake Q4–Q5, the gate message, the pipeline walk and the routing row; migration keeps its own vocabulary — Maurits Visser
 - new(existing-app-change): **map the app first, ask what to do with the findings, and let a project park until the change is named.** Stage 0 in this mode now opens with the app analysis (`app-facts.sh` + `app-report.sh`, `skills/app-analysis.md`), run without asking since it is read-only and takes about a minute, followed by the question the user owns: fix / log / accept per top finding. Kickoff no longer opens with "which slice?"; the slice and its blast radius (read from the map's tangles and edges, not recomputed) come at Stage 0b, when the change arrives. `artifact-manifest.tsv` owes the map in this mode (`app-report`, Stage 0); gate-check reports a mapped project with no change as Stage 0 `PENDING` instead of a permanent FAIL, and its Stage 1 hint names Path D instead of an extractor. `existing-app-assurance.md` Track A starts from the same report, so an audit that turns into a change does not redo it. Fixture: `test-bug03-gates.sh` T12. Field run: a live client workflow app — 28 modules, one tangle of 7 of 9 own modules, 213 loop microflows, mapped in 76 s; gate-check read it PENDING/parked with 0 needing attention — Maurits Visser
 - docs(pipeline-walks): **`docs/pipeline-walks.html` — a process diagram per entry mode, with the scripts run at every step.** Shared spine, migration, requirements-driven (incl. the docs-ready fast path), greenfield, change-an-existing-app (opening with the app-mapping step: `SHOW STRUCTURE`, `graph-report`, `lint`, `report`, security matrix, `marketplace diff`), à-la-carte tracks A/A2/B, and the Stage 5 BUILD→GATE→PROVE→LOOK→CONFIRM loop, each as a mermaid flowchart plus a stage/what/scripts table. Linked from the README entry-modes paragraph — Maurits Visser

bin/init-project.sh

@@ -334,6 +334,15 @@ This project uses the shared toolkit at \`$TOOLKIT_ROOT\`. For ANY pipeline work
 3. **Every gate question is asked in chat, then the turn ends and waits** (runbook §1).
    \`ASSUMED\` is earned by asking — the user said "you decide" — never by skipping.
 4. **Before calling a stage done:** \`$TOOLKIT_ROOT/bin/gate-check.sh <project-root> <stage>\`.
+5. **Before any page MDL:** read \`$TOOLKIT_ROOT/skills/ui-preflight-pages.md\` in full —
+   its *text*, not a reference to it. If page drafting is delegated, the dispatch prompt
+   carries the text or the subagent reads the file as its first action.
+   **After any page MDL:** \`bash bin/check-page-shell.sh\`, then
+   \`node $TOOLKIT_ROOT/project-bin/page-fidelity.js <wireframe> <Page> <script>\`, then open
+   the rendered page at 1280px and 390px and write what you saw to
+   \`design/ui-reviews/ui-review-<date>.html\`. The fidelity score does not discharge the
+   look obligation — it scored 100% on two screens that matched under half their wireframe
+   (benchmark run \`mendix-run1b\`, 2026-09-19).
 
 EOF
   echo "Created: CLAUDE.local.md (runbook-first wiring + baseline routing)"

contrib/inbox/2026-09-19-look-pass-never-runs-benchmark-run1.md

@@ -0,0 +1,101 @@
+# LOOK pass, ui-loop and page-fidelity never executed on a full Stage-5 build — second occurrence
+
+**From:** lowcode-vs-highcode benchmark, Mendix arm run 1 (orchestrator session)
+**Date:** 2026-09-19
+**Kind:** process
+**Field evidence:** A 3-module, 7-screen requirements-driven build (Mendix 11.14, mxcli 0.22.0, toolkit @ 2d9a0a2) closed with `design/ui-reviews/` empty, no `docs/PAGE-FIDELITY.tsv`, no `.claude/loop/`, `bin/page-fidelity.js` present but never run, and `PROJECT.md` still reading "gates passed: none yet (derived by gate-check on <scaffold date>)". The human noticed by looking at the screenshots. Same chain as `contrib/inbox/2026-09-09-ui-loop-never-runs-page-fidelity.md`, which was still unpromoted.
+**Proposed target:** `skills/conversion-runbook.md` §1b (definition of done), `bin/init-project.sh` (install instruments), agent-stub dispatch / `iterative-build-loop.md`, and triage of the 2026-09-09 note together with this one.
+
+---
+
+## What we did
+
+- Stages 0–4 with the toolkit (intake, BRDs, blueprint, wireframes S1–S7, ds.css, build plan).
+- Stage 5 built by a lead agent that was itself a subagent, plus 14 generic worker agents in 5
+  batches, each with a per-file brief naming the MDL script and the wireframe. Role-agent stubs
+  (`mdl-agent`, `review-agent`, …) were scaffolded but not dispatched — a subagent lead cannot
+  spawn subagents, and the run rules excluded them anyway.
+- Every stage was declared done in the run log. `gate-check.sh` was never re-run after scaffold.
+- Result: pages structurally off their wireframes (missing status chips, unformatted amounts,
+  absent filter chips, empty persona selector, clipped collapsed nav, raw "No items found" empty
+  states), on top of the scaffold `signal` theme the spec's "platform default" rule left in place.
+
+## Why the five UI obligations never fired (finding, reproduced by reading the artifacts)
+
+1. **The obligation check is only as good as the last `gate-check.sh` run.** `obligations.tsv`
+   row `look` would have printed PENDING for all three modules — but nothing in the build loop
+   requires gate-check before "done", so it printed nothing. Green-by-absence happened one level
+   up: absence of the *check*, not absence of the artifact.
+2. **"A citation is not a read" — again.** `ui-preflight-pages.md`, `ui-loop.md`,
+   `module-review.md`, `page-fidelity.js` and `check-page-shell.sh` are routed through
+   `mdl-agent.md` / `review-agent.md` stubs and the CLAUDE.local.md situational table only. The
+   lead's own always-on block (mxtk wiring: RESUME.md, exec-approval, no strays) has no LOOK line,
+   no fidelity step, no gate-check-before-done. Workers got briefs, not skills. So the text was
+   in nobody's context at page-writing time.
+3. **Instruments installed, never run.** `init-project.sh` copied `page-fidelity.js`
+   and `check-page-shell.sh` into the project `bin/`; nothing in the loop required executing them.
+4. **Inbox not drained.** The 2026-09-09 note predicted this exactly. Ten days in the queue.
+
+## Should these be part of the pipeline? (position, not hypothesis)
+
+They are. The gap is enforcement, not content. Proposals, smallest first:
+
+- **A.** In the runbook's Live Checklist Protocol (§1b), make the last item of every Stage-5
+  module checklist literally: `gate-check.sh <root> 5` ran, output pasted, zero PENDING
+  obligations for this module. A stage reposted without it is not done.
+- **B.** `init-project.sh` (already installs both instruments) additionally writes the `docs/PAGE-FIDELITY.tsv` header
+  so the artifact check sees a stub and the
+  human sees zeros instead of nothing.
+- **C.** Put a 6-line "before any page MDL / after any page MDL" block into the *always-on*
+  section of the generated `CLAUDE.local.md` (not the situational table): read
+  `ui-preflight-pages.md`; after exec, screenshot, run page-fidelity, append the row. This is
+  the inline-the-dispatch rule from CLAUDE.md authoring rule 2 applied to the lead itself.
+- **D.** Hypothesis: the `look` obligation should also accept a *fidelity row ≥ threshold plus
+  a screenshot file* as evidence, not only `ui-review-*.html`, so single-session builds without a
+  review agent can still satisfy it honestly.
+- **E.** Triage discipline: an inbox note older than N days that names a pipeline gap blocks the
+  next `init-project.sh` on that machine with a warning listing it. (Hypothesis; may be too noisy.)
+
+## What we are doing on the project
+
+Re-running the UI pass on both benchmark arms under a spec amendment: element-checklist
+alignment per wireframe (present / placed / behaves), ≥95% target, header/topbar mandatory,
+platform-default theme retained. Fidelity measured mechanically, LOOK pass with screenshots,
+`gate-check.sh` before close. Outcome will be filed as a follow-up to this note.
+
+## Follow-up (same day, same author) — proposal D is field-refuted; A/C landed
+
+Run 1b closed on both arms and the outcome is filed at
+`lowcode-vs-highcode-benchmark/results/mendix-run1b/` and `results/highcode-run1b/`.
+
+**D is withdrawn, not deferred.** It proposed that the `look` obligation accept a fidelity row
+≥ threshold plus a screenshot instead of a review artifact. Scoring the same commit both ways
+shows the score cannot stand in for the pass in either direction:
+
+| Screen | Fidelity (text match) | Element checklist (present / placed / behaves) |
+|---|---:|---:|
+| S1 Requests overview | 100% | 58% |
+| S4 Approval queue | 100% | 40% |
+| S3 Request detail | 55% | 69% |
+
+`page-fidelity.js` greps identifiers out of the page MDL and matches them against the
+wireframe's text; it cannot see nesting or placement. S1 and S4 named every element and laid
+them out wrongly. S3 fails the grep on formatting the instrument has no expression for. A
+threshold over this number would have passed the two worst screens in the run.
+
+**A and C landed instead**, as commit `e466e4c` in this repo:
+- `skills/conversion-runbook.md` §1b **rule 8** — the last checklist item of every stage and
+  every module build is `bin/gate-check.sh`, run and pasted, zero `PENDING` or each one named
+  and waived (proposal A).
+- `bin/init-project.sh` — a before/after page-MDL item in the generated `CLAUDE.local.md`, so
+  the pre-flight text reaches the agent that writes pages (proposal C).
+- `skills/ui-preflight-pages.md` — "a fidelity score is not a LOOK pass", carrying the table
+  above. This is what replaced D.
+
+**B is moot**: `project-bin/page-fidelity.js` already writes the `PAGE-FIDELITY.tsv` header when
+it creates the file, so a missing score is not a missing table. **E is untouched** — still a
+hypothesis, still possibly too noisy.
+
+Non-Mendix control, so this reads as a pipeline gap rather than a platform one: Arm B's
+high-code baseline was 43% (approvals) and 56% (admin) against the same wireframes, and reached
+100% only after the same deliberate LOOK loop.

skills/conversion-runbook.md

@@ -339,6 +339,23 @@ Rules — these apply to every stage and every per-module build loop:
    asked well and proved well and closed nothing — see the template section for the measured
    register.)
 
+8. **The last checklist item of every stage and every module build is the gate script,
+   run and pasted.** `bin/gate-check.sh <project-root> <stage>` — and for a module build the
+   obligation lines for that module — go in the chat as output, with zero `PENDING`, or with
+   each remaining `PENDING` named and waived (`--waive <obligation> --reason "..."`). A stage
+   whose gate script never ran is not done, however complete its checklist looks. This is the
+   item that makes the obligation check bite: it reports a pass nobody performed, but only on
+   a run, and nothing before this rule required one.
+
+   **Why (measured, 2026-09-19, `lowcode-vs-highcode-benchmark` runs `mendix-run1` and
+   `mendix-run1b`).** A full Stage-5 build closed with all seven screens built, a green mxbuild
+   gate, and 11 of 37 acceptance rows confirmed. `gate-check.sh` had run exactly once, at
+   scaffold time. The `look` obligation, the wiring sweep, journeys and coherence were all
+   `PENDING` and nobody saw it, because nothing asked. When the gate was finally run as a
+   deliberate second pass, the seven screens scored 0, 18, 40, 40, 58, 58 and 69 percent against
+   their own wireframes on an element checklist — one screen matched none of its wireframe's
+   elements at all. Every one of those defects was visible to a gate that never ran.
+
 The final full-checklist repost before a gate doubles as the gate's evidence: the user should
 be able to approve the gate by reading that one message. The close-out block is what the
 reader approves the *transition* on — checklist for the work, block for the hand-over.