From 9c247eab98b2a2e421213b6867acf0baf133bf1d Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 18 Sep 2026 02:29:44 +0000 Subject: [PATCH 01/18] =?UTF-8?q?contrib:=20queue=20idea=20=E2=80=94=20pre?= =?UTF-8?q?requisites=20+=20success-criteria=20gate=20before=20any=20"full?= =?UTF-8?q?=20X"=20task?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 3 + ...-09-18-prerequisites-gate-for-big-tasks.md | 94 +++++++++++++++++++ 2 files changed, 97 insertions(+) create mode 100644 contrib/inbox/2026-09-18-prerequisites-gate-for-big-tasks.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 2cb3de4f..219f118a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,9 @@ moment updating it became a separate chore). One line per change: Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templates, CI). Credit the person or project that surfaced the change — the credit line is the thank-you. +## 2026-09-18 +- process(contrib/inbox): idea queued — **a prerequisites + success-criteria gate before any "full X" task** (full app analysis, full e2e, full UI loop, assessment). Big à-la-carte asks start executing and discover mid-run that the app is not up, no persona has credentials, no seed data or baseline exists, or "done" was never defined — tokens spent before the discovery are lost. Proposal: a per-task-family card (prerequisites split into mechanically-checked vs ask-the-user, plus a pass bar with a denominator) posted as the first Live Checklist item, answers recorded per project so the next run reuses them. `contrib/inbox/2026-09-18-prerequisites-gate-for-big-tasks.md` — Maurits Visser + ## 2026-09-17 - process(register): **the toolkit never mentioned `mxcli brain`, while mxcli writes "read `docs/brain/project.md` first" into every project's CLAUDE.md** — so a wired project ran two diff --git a/contrib/inbox/2026-09-18-prerequisites-gate-for-big-tasks.md b/contrib/inbox/2026-09-18-prerequisites-gate-for-big-tasks.md new file mode 100644 index 00000000..149c2d77 --- /dev/null +++ b/contrib/inbox/2026-09-18-prerequisites-gate-for-big-tasks.md @@ -0,0 +1,94 @@ +# Idea: a prerequisites gate before any "full X" task — readiness and success criteria first, tokens second + +**From:** Maurits Visser (MendixMau) +**Date:** 2026-09-18 +**Kind:** process +**Field evidence:** recurring pattern, not a single incident — big à-la-carte asks ("full app +analysis", "full e2e test run", "full UI loop", "full assessment") start executing immediately +and discover mid-run that something was never in place: no running app, no login/auth, no seed +data, no baseline to compare against, no agreed definition of done. The tokens spent before the +discovery are lost, and the run either stops half-way or "completes" against a target it never +verified. +**Proposed target:** `skills/task-prerequisites.md` (new skill) + a routing row in +`bin/lib/skill-routing.tsv` that fires on "full …", "complete …", "run the whole …", "e2e", +"analysis of the app", "UI loop"; possibly a `bin/preflight-task.sh` for the mechanical half. + +--- + +## The idea + +When a user asks for a **big, self-contained task** — anything expected to take many turns and +a large token budget — the agent does not start the task. It first produces a short +**prerequisites + success-criteria card**, checks what it can mechanically, asks the user only +for what it cannot, and starts the actual work only when the card is green (or the user +explicitly waives a red row). + +Two halves: + +1. **Prerequisites** — everything the task will need that is *not* the task itself: + - **Environment**: app running? which URL? mxbuild set up? Docker available? mxcli version + matches project? + - **Access**: credentials / demo users / roles for every persona the run will exercise; + MFA or SSO out of the way; DB connection for assertions. + - **Data**: seed/demo data present and known; a way to reset it between runs; which records + the scenarios will touch. + - **Inputs**: the artifacts the task reads (BRDs, wireframes, page-scope, a baseline + report, a previous run to diff against) exist and are current. + - **Scope denominators**: the module list / page list / scenario list the run will cover, + so "full" has a number attached (ties to the obligation check's "denominators" rule). + - **Budget & mode**: attended or unattended, expected wall-clock, which snapshot/restore is + in place. +2. **Success criteria** — what "done" looks like, agreed *before* the run: the artifact it + emits, the denominator it states, the pass bar (e.g. "all N journeys walked, 0 blockers", + "report with ≥1 finding per module or explicit NONE"), and what happens on partial failure. + +## Why this belongs in the toolkit rather than in each project + +- The stage gates already do exactly this for the *pipeline* (`gate-check.sh`, obligations, + artifact manifest). The à-la-carte lane (`existing-app-assurance.md`) has no equivalent: a + "full e2e" is a big unguarded task that starts with zero preconditions checked. +- It is the same lesson as "a producer for every consumer" (CLAUDE.md, shipping an instrument): + a task that consumes a running app + auth + data must have a step that *establishes* them, + not assume them. +- Cheap where it matters: the card is ~20 lines and one turn; the run it protects is hundreds. + +## Product view (what I would and would not build) + +**Do:** +- Make it a **skill with a fixed card template** per task family (analysis / e2e / UI loop / + assessment / migration triage), each family listing its own prerequisite rows and default + success criteria. The card is posted in chat as the first item of the Live Checklist, so the + existing ✅/🔄/⬜/❌/⏭ protocol carries it — no new mechanism. +- Split rows into **mechanically checkable** (app answers on the URL, demo users exist via + `SHOW DEMO USERS`, mxbuild present, BRDs discoverable via `discover-brds.sh`) and + **ask-the-user** (which persona, which data reset strategy, what pass bar). Auto-check the + first group; ask only the second — in one `AskUserQuestion` round, then stop and wait. +- Record the answers in `PROJECT.md` (or `docs/brain/` for a no-pipeline app) so the *next* + "full e2e" on the same project reuses them instead of re-asking. Prerequisites are mostly + stable per project; success criteria are per run. +- A red row that the user waives is recorded as a waiver with a reason — same convention as + `--waive`, so a run that fails on the waived item reports "known, waived" not "surprise". + +**Don't:** +- Don't build a generic "task planner" — it will drift into re-planning everything. Trigger only + on the named big-task families, and keep each card under ~15 rows. +- Don't gate small asks. A one-page fix or a single lint question must not pay this tax; the + routing row's trigger words are the scope fence. +- Don't let the card become prose. Every row is a checkable bound with a denominator (authoring + rule 4), or it is not a row. + +## Open gaps to settle before promoting + +1. Which task families get a card in v1? Proposal: **e2e test run**, **UI review loop**, + **existing-app assurance (audit/lint/regression)**, **full migration assessment**. Others + later. +2. Where does the "I already checked this on this project" memory live for a no-pipeline app — + `docs/brain/` (mxcli 0.21+) or a small `.claude/task-prereqs.json`? +3. Does `bin/preflight-task.sh` earn its keep in v1, or is the skill + Live Checklist enough + until one field run shows which rows are actually re-checked every time? +4. Interaction with unattended mode: in unattended runs a red mechanical row must **stop**, not + assume — that is the whole point. + +Hypothesis, not finding: most of the wasted runs would have been caught by three rows alone — +app reachable, credentials for every persona, data present. Worth measuring on the first two +field runs before growing the card. From 812609fbb2b5b4a8c69a7fe27ef34dfb1b3832bd Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 18 Sep 2026 02:32:42 +0000 Subject: [PATCH 02/18] =?UTF-8?q?contrib:=20prerequisites-gate=20idea=20?= =?UTF-8?q?=E2=80=94=20memory=20lives=20in=20mxcli=20brain,=20with=20three?= =?UTF-8?q?=20constraints?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- .../2026-09-18-prerequisites-gate-for-big-tasks.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/contrib/inbox/2026-09-18-prerequisites-gate-for-big-tasks.md b/contrib/inbox/2026-09-18-prerequisites-gate-for-big-tasks.md index 149c2d77..4a979e97 100644 --- a/contrib/inbox/2026-09-18-prerequisites-gate-for-big-tasks.md +++ b/contrib/inbox/2026-09-18-prerequisites-gate-for-big-tasks.md @@ -82,8 +82,18 @@ Two halves: 1. Which task families get a card in v1? Proposal: **e2e test run**, **UI review loop**, **existing-app assurance (audit/lint/regression)**, **full migration assessment**. Others later. -2. Where does the "I already checked this on this project" memory live for a no-pipeline app — - `docs/brain/` (mxcli 0.21+) or a small `.claude/task-prereqs.json`? +2. ~~Where does the "I already checked this on this project" memory live?~~ **Resolved 2026-09-18 + (Maurits): `mxcli brain`.** Rationale: the 2026-09-17 split already sends "what a session + learned" to brain and only gate answers to `PROJECT.md`; riding the CLI's own concept avoids a + parallel register that rots; anchors make a prerequisite self-invalidating (`brain check` goes + red when the demo user's module role is dropped). Three constraints: + - **Anchor what has a model element** — persona → module role / demo user, seed data → entity, + journey start → page. **Environment facts** (app URL, Docker, db name, mxbuild path) have no + anchor and live in `stack.env` via `test-stack-up` — the card reads both, there is no third + store. + - **Never a credential in brain** — `docs/brain/` is committed. Demo user name and role only. + - **Degrade on an older mxcli** — brain is 0.21+; probe first (capability-probe rule), fall back + to `PROJECT.md` rows rather than block. 3. Does `bin/preflight-task.sh` earn its keep in v1, or is the skill + Live Checklist enough until one field run shows which rows are actually re-checked every time? 4. Interaction with unattended mode: in unattended runs a red mechanical row must **stop**, not From 71c6421d0887db91e9bf8e34a3eaebce283d7cf7 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 18 Sep 2026 06:55:18 +0000 Subject: [PATCH 03/18] =?UTF-8?q?contrib:=20queue=20idea=20=E2=80=94=20com?= =?UTF-8?q?pany=20brain=20as=20an=20overlay=20repo=20(template,=20one=20po?= =?UTF-8?q?inter=20row,=20hand-curated)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 1 + .../inbox/2026-09-18-company-brain-overlay.md | 82 +++++++++++++++++++ 2 files changed, 83 insertions(+) create mode 100644 contrib/inbox/2026-09-18-company-brain-overlay.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 219f118a..05420208 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templat Credit the person or project that surfaced the change — the credit line is the thank-you. ## 2026-09-18 +- process(contrib/inbox): idea queued — **a company brain as an overlay repo between the toolkit and the project**. Today company-specific material (own skills, conventions, lint rules, MDL snippets, MPK catalog, client-named learnings) has no home: the shared toolkit forbids it and per-project copies rot. Proposal: a second repo in the toolkit's shape, wired by `init-project.sh --company`, routed by ONE ~40-word baseline pointer row (its own table is ondemand, so no new baseline words), curated by the company by hand from a project-local outbox. No mxcli change needed: an imported MPK is a module, so brain anchors already reach it. `contrib/inbox/2026-09-18-company-brain-overlay.md` — Maurits Visser - process(contrib/inbox): idea queued — **a prerequisites + success-criteria gate before any "full X" task** (full app analysis, full e2e, full UI loop, assessment). Big à-la-carte asks start executing and discover mid-run that the app is not up, no persona has credentials, no seed data or baseline exists, or "done" was never defined — tokens spent before the discovery are lost. Proposal: a per-task-family card (prerequisites split into mechanically-checked vs ask-the-user, plus a pass bar with a denominator) posted as the first Live Checklist item, answers recorded per project so the next run reuses them. `contrib/inbox/2026-09-18-prerequisites-gate-for-big-tasks.md` — Maurits Visser ## 2026-09-17 diff --git a/contrib/inbox/2026-09-18-company-brain-overlay.md b/contrib/inbox/2026-09-18-company-brain-overlay.md new file mode 100644 index 00000000..b559ac31 --- /dev/null +++ b/contrib/inbox/2026-09-18-company-brain-overlay.md @@ -0,0 +1,82 @@ +# Idea: a company brain as an overlay repo — template, not automation + +**From:** Maurits Visser (MendixMau) +**Date:** 2026-09-18 +**Kind:** process +**Field evidence:** there is no layer between the shared toolkit and a project. A company that +adopts the toolkit can put its own material in exactly two places — the shared toolkit (which +the leak guard forbids for anything client-named) or a hand copy inside each project (which +rots; `sync-project.sh` exists because copies rot). Own skills, naming conventions, lint rules, +MDL snippets, design systems and approved MPKs have no home. Related: PR #93's open question +"is a toolkit-owned decision register still justified", PR #88 (mxcli brain adopted per project), +issue #90 (baseline budget saturated). +**Proposed target:** `templates/company-overlay/` (new) + `--company ` on +`bin/init-project.sh` / `bin/sync-project.sh` + one ~40-word baseline pointer row in +`bin/lib/skill-routing.tsv`. + +--- + +## Three tiers, same shape + +| Tier | Owner | Holds | Mechanism | +|---|---|---|---| +| Toolkit | shared | generic process, stages, gates, learned-* patterns, stock lint rules | clone + `sync-project.sh` (exists) | +| **Company brain** | **the company** | own skills, conventions, lint rules, MDL snippets, design system, MPK catalog, learned patterns that may name clients | **overlay repo — this idea** | +| Project brain | one app | decisions anchored in the model | `mxcli brain` (PR #88, exists) | + +The company tier is a second repo with the toolkit's directory shape (`skills/`, `lint-rules/`, +`examples/`, `bin/`, its own `skill-routing.tsv`), so nothing new has to be learned and the +existing render/check tooling applies. A fork is the wrong shape: every toolkit release would be +re-merged by hand. An overlay is `git pull` on both and one sync. + +## Decisions taken 2026-09-18 (Maurits) + +1. **No new baseline words.** The budget (`render-routing.sh`, 80 000 words, a ratchet) counts + baseline *documents* only; ondemand rows cost nothing at session start. The company tier + therefore gets **one baseline pointer row (~40 words)**: "a company overlay is wired at + ``; its routing table is `/ROUTING.md`, load it when a trigger fires". The + company's own table is ondemand by construction. This is the toolkit's existing "single source, + every other mention defers" convention applied one level up. A company may promote its own + rows to baseline only inside *its* budget, rendered separately — the toolkit's ratchet is not + theirs to spend. +2. **No mxcli change needed for MPKs.** Brain anchors are `@Module.Element`. An MPK, once + imported, *is* a module in the model, so a decision "use company widget X on page Y" anchors to + it with no new capability. A forward anchor to a component not yet imported fails `brain check` + — which is the correct behaviour (the prerequisite is not met). So: the catalog is a static + manifest per component (name, version, Mendix version range, namespace, purpose, install + step); the *use* of a component is a project-brain entry. No upstream bug or feature request. + Not probed on a binary — this container has none; the claim rests on the anchor grammar + documented in `close-the-loop.md` and `existing-app-change.md`. Probe before promotion. +3. **The company curates, by hand.** Project content stays in the project: `docs/brain/` for + model-anchored decisions, plus a project-local `contrib/outbox/` for anything a session thinks + is reusable. `bin/harvest-learnings.sh` already drafts inbox files from a project's bug logs, + promotion tables and patched scripts — point its output at the company overlay's `inbox/` + instead of the toolkit's, and the company decides what to promote. **A template the customer + wires themselves, not automation:** the toolkit ships the skeleton and the wiring recipe; + the company owns the repo, the triage, the leak policy and the routing budget. + +## What the toolkit ships (v1, small) + +- `templates/company-overlay/` — skeleton: `README.md` (the wiring recipe), empty + `skills/ lint-rules/ examples/mdl/ catalog/mpk/ inbox/`, a `skill-routing.tsv` header, + `catalog/mpk/TEMPLATE.md` (the manifest fields above), `inbox/TEMPLATE.md` (copy of the + toolkit's). +- `bin/init-project.sh --company ` — optional; writes a second root into the wiring block + and the one pointer row into `CLAUDE.local.md`. Absent flag = today's behaviour. +- `bin/sync-project.sh` — refreshes copies from both roots when the flag was used. +- `bin/harvest-learnings.sh --to /inbox` — output target, default unchanged. +- Lint: a third hash set beside `lint-rules/STOCK-HASHES.txt` so a company rule is distinguishable + from a stock rule and from a project-local edit. Same script, one more directory. + +## Not in v1 + +- Company baseline rendering into project surfaces beyond the pointer row. +- Any automatic promotion from project to company or company to toolkit. +- MPK install automation — capability-probe the binary first (the "marketplace is manual" + misfire is on record in this toolkit). + +## Open + +- Does the leak guard run in the overlay? It should, with the company's *own* denylist + (clients they serve), since the overlay is where client-named learnings are allowed to live. +- Name: "company brain" is the user's term; "overlay" is the mechanism. Pick one for the docs. From 14a11740901141538150aa96c44b097d40f35cbe Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 18 Sep 2026 08:12:50 +0000 Subject: [PATCH 04/18] =?UTF-8?q?contrib:=20company=20brain=20idea=20?= =?UTF-8?q?=E2=80=94=20leak=20guard=20scope=20settled=20(probes=20only,=20?= =?UTF-8?q?names=20are=20the=20company's=20call),=20name=20fixed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- .../inbox/2026-09-18-company-brain-overlay.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/contrib/inbox/2026-09-18-company-brain-overlay.md b/contrib/inbox/2026-09-18-company-brain-overlay.md index b559ac31..5c2fec09 100644 --- a/contrib/inbox/2026-09-18-company-brain-overlay.md +++ b/contrib/inbox/2026-09-18-company-brain-overlay.md @@ -10,7 +10,7 @@ rots; `sync-project.sh` exists because copies rot). Own skills, naming conventio MDL snippets, design systems and approved MPKs have no home. Related: PR #93's open question "is a toolkit-owned decision register still justified", PR #88 (mxcli brain adopted per project), issue #90 (baseline budget saturated). -**Proposed target:** `templates/company-overlay/` (new) + `--company ` on +**Proposed target:** `templates/company-brain/` (new) + `--company ` on `bin/init-project.sh` / `bin/sync-project.sh` + one ~40-word baseline pointer row in `bin/lib/skill-routing.tsv`. @@ -57,7 +57,7 @@ re-merged by hand. An overlay is `git pull` on both and one sync. ## What the toolkit ships (v1, small) -- `templates/company-overlay/` — skeleton: `README.md` (the wiring recipe), empty +- `templates/company-brain/` — skeleton: `README.md` (the wiring recipe), empty `skills/ lint-rules/ examples/mdl/ catalog/mpk/ inbox/`, a `skill-routing.tsv` header, `catalog/mpk/TEMPLATE.md` (the manifest fields above), `inbox/TEMPLATE.md` (copy of the toolkit's). @@ -75,8 +75,14 @@ re-merged by hand. An overlay is `git pull` on both and one sync. - MPK install automation — capability-probe the binary first (the "marketplace is manual" misfire is on record in this toolkit). -## Open +## Resolved 2026-09-18 (Maurits) -- Does the leak guard run in the overlay? It should, with the company's *own* denylist - (clients they serve), since the overlay is where client-named learnings are allowed to live. -- Name: "company brain" is the user's term; "overlay" is the mechanism. Pick one for the docs. +- **Leak guard.** `check-no-client-data.sh` guards the *public toolkit* — that is its stated + purpose. It has two halves: a gitignored **denylist of names** (optional) and **generic probes** + for real data (strings copied from a live app, typed GUIDs, local filesystem paths, contact + details). In the company brain, names — the company's own, its clients', its apps' — are the + company's call and need no denylist. What the company brain keeps is the **probes half only**: + no personal data, no credentials, no record contents, no local paths. One reuse of the existing + script with the denylist off (CI already runs it that way). +- **Name: "company brain".** The overlay is the mechanism; the docs and the template directory say + company brain (`templates/company-brain/`). From cefd7c44f2360040434aa91278d7935760b3ab96 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 19 Sep 2026 09:24:56 +0000 Subject: [PATCH 05/18] =?UTF-8?q?toolkit-guide=20=C2=A79:=20the=20wiring?= =?UTF-8?q?=20page=20=E2=80=94=20four=20layers,=20owner-per-file=20table,?= =?UTF-8?q?=20PROPOSED=20badges=20on=20unshipped=20pieces?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 3 + toolkit-guide.html | 147 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 150 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 05420208..aa511613 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,9 @@ moment updating it became a separate chore). One line per change: Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templates, CI). Credit the person or project that surfaced the change — the credit line is the thank-you. +## 2026-09-19 +- new(toolkit-guide.html §9): **the wiring page — how the assets fit together and what a session reads, in which order.** Four layers (toolkit, company brain, project registers, stage chain) plus build-and-prove, an owner-per-file table, and a PROPOSED badge on the unshipped pieces (company brain, prerequisites card, brain-replaces-ledger research PR #93) so an idea is never mistaken for a file. Written for a developer onboarding; prompted by a wiring diagram that lived only in a deck — Maurits Visser + ## 2026-09-18 - process(contrib/inbox): idea queued — **a company brain as an overlay repo between the toolkit and the project**. Today company-specific material (own skills, conventions, lint rules, MDL snippets, MPK catalog, client-named learnings) has no home: the shared toolkit forbids it and per-project copies rot. Proposal: a second repo in the toolkit's shape, wired by `init-project.sh --company`, routed by ONE ~40-word baseline pointer row (its own table is ondemand, so no new baseline words), curated by the company by hand from a project-local outbox. No mxcli change needed: an imported MPK is a module, so brain anchors already reach it. `contrib/inbox/2026-09-18-company-brain-overlay.md` — Maurits Visser - process(contrib/inbox): idea queued — **a prerequisites + success-criteria gate before any "full X" task** (full app analysis, full e2e, full UI loop, assessment). Big à-la-carte asks start executing and discover mid-run that the app is not up, no persona has credentials, no seed data or baseline exists, or "done" was never defined — tokens spent before the discovery are lost. Proposal: a per-task-family card (prerequisites split into mechanically-checked vs ask-the-user, plus a pass bar with a denominator) posted as the first Live Checklist item, answers recorded per project so the next run reuses them. `contrib/inbox/2026-09-18-prerequisites-gate-for-big-tasks.md` — Maurits Visser diff --git a/toolkit-guide.html b/toolkit-guide.html index ca81e312..fd358f56 100644 --- a/toolkit-guide.html +++ b/toolkit-guide.html @@ -44,6 +44,7 @@ .badge.gate { background: var(--fail-bg); color: var(--fail-ink); } .badge.soft { background: var(--pass-bg); color: var(--pass-ink); } .badge.mig { background: var(--warn-bg); color: var(--warn-ink); } + .badge.prop { background: var(--warn-bg); color: var(--warn-ink); border: 1px dashed var(--warn-ink); } /* timeline */ .timeline { list-style: none; margin-top: 18px; } @@ -327,6 +328,152 @@

“Studio Pro won't start / hangs on launch”

8 · For agents: this page is also the shared HTML shell

Every stage HTML surface (triage.html, architecture/blueprint.html, build-plan.html, test-report.html, …) copies this file's :root token block and base styles, so all project artifacts look like one system. Once Stage 3 produces design/design-system.html, that file's tokens supersede these for artifacts generated afterwards. Copying the tokens does not mean opening this page: opening is governed by the first-touch rule in CLAUDE.md — only when <project-root>/.claude/.guide-shown is absent, and touch it immediately after.

+

9 · How the assets wire together — what a session reads, in which order

+

Nothing is remembered between sessions. Every fact a session acts on is a file, read in a fixed order, and each file is written by exactly one owner. This is the map of those files. Boxes marked PROPOSED are ideas queued in contrib/inbox/ or open PRs — not shipped, do not go looking for them.

+ + + + + + + + + + + + + LAYER 1 · THE TOOLKIT — the process, shared, never holds your data + + + skills/conversion-runbook.md + the spine: stages, gates, entry modes + + bin/lib/skill-routing.tsv + rendered into every CLAUDE.local.md + + bin/gate-check.sh · status.sh + read the registers, never write them + + skills/learned-*.md · bug-logs/ + what transfers between any two apps + + + LAYER 2 · THE COMPANY BRAIN — yours, private, same shape as the toolkit + + + PROPOSED · inbox + own skills · naming conventions · lint rules · MDL snippets · design system · approved MPK catalog + Wired by one ~40-word pointer row in CLAUDE.local.md; its own routing table loads on demand. Curated by the company, by hand, + from each project's outbox. Client names are allowed here; personal data, credentials and local paths never. + + + LAYER 3 · PROJECT REGISTERS — read by every stage, written by no stage alone + + + PROJECT.md + every gate answer: stage, entry mode, + CONFIRMED / ASSUMED, waivers + gate-check reads it — the only register it parses + + docs/brain/ (mxcli brain) + what a session learned: a pattern and its + why, anchored to @Module.Element + brain check fails when the anchor dies + + coverage ledger + every BRD leaf claimed by a build-plan + row, or catalogued as not built + brain plan may replace it — research, PR #93 + + docs/open-questions.html + anything the pipeline could not settle, + batched per gate by bin/open-questions.sh + blocks the gate while anything is unanswered + + + + + + + + + + LAYER 4 · THE STAGE CHAIN — each stage reads only the one before it, on disk + + + + source/ + read only, never edited + + + knowledge-base/ + the extraction · stage 1 + + + brd/*.brd.json + requirements · stage 2 + + + architecture/ design/ + blueprint, wireframes · stage 3 + + + build-plan.md + the ordered work · stage 4 + + + <Module>-brief.md + the ONE file the build opens + + The brief is assembled per module from BRD + wireframes + blueprint + build plan. It points at those files rather than copying them, so a changed wireframe never means a rewritten brief. + + + THEN · BUILD AND PROVE — stage 5 and 6, per module, every result back into a file + + + + mdlsource/*.mdl + via bin/exec.sh: snapshot first + + + <App>.mpr + mxbuild validates after + + + docs/BUILD-LOG.md + append-only, one row per exec + + + docs/report.json + every instrument appends here + + + index.html + regenerated by gate-check + + + + + + PROPOSED · inbox + Prerequisites card — before any "full X" task outside the pipeline (full e2e, full UI loop, full audit) + App reachable, a credential per persona, seed data, a denominator, a pass bar — checked or asked once, answers kept in docs/brain/, before a single token is spent on the run. + + +

Who writes what — one owner per file

+ + + + + + + + + + +
FileWritten byRead byWhat breaks if it's stale
PROJECT.mdthe agent at a gate, after asking yougate-check.sh, checkpoint.sh, every stagea stage runs on a decision you never saw; gates block on UNSYNCED
docs/brain/mxcli brain capture when a session learns somethingthe next session, brain check before every committhe next session reasonably decides the opposite
coverage ledgerStage 4, from build-plan claimsStage 4 gate, Stage 5 build rowsa requirement is silently neither built nor descoped
docs/open-questions.htmlbin/open-questions.sh, from BRD openQuestions + SME notesyou, at every gatea product decision gets taken in a file nobody opens
brd/*.brd.jsonStage 2 (extractor, KB, interview or by hand)Stage 3, 4, the brief, the ledgerthe build drifts from the requirement — see the drift-sync rule
<Module>-brief.mdba-agent (roles, screens, rules) + architect (entities, dependencies, stubs)the build agent, first, for that module onlythe build agent improvises what nobody wrote down
docs/BUILD-LOG.md · docs/report.jsonbin/exec.sh and every instrumentstatus.sh, index.html, the test-result audit"done" becomes a claim instead of a record
PROPOSED company brainthe company, by hand, from each project's outboxba-agent and architect before writing a brief; any session, on demandeach project re-invents the convention
+

The rule under all of it: a session starts by reading, in this order — the toolkit's runbook, then the project registers, then the one stage artifact before it — and starts a new session at each stage boundary instead of running one all day. Memory is never a source; if it isn't on disk, it didn't happen.

+
mxcli-project-toolkit · toolkit-guide.html · The detailed, executable version of this page is skills/conversion-runbook.md; the thin text pointer is CONVERSION-RUNBOOK.md.
From f6c9b23067ca27fb5fdeaf88ae3eebe8ef138fe6 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 19 Sep 2026 09:29:54 +0000 Subject: [PATCH 06/18] =?UTF-8?q?contrib:=20company=20brain=20=E2=80=94=20?= =?UTF-8?q?prior=20art=20recorded:=20personal-toolkit=20is=20the=20running?= =?UTF-8?q?=20prototype,=20public-cites-private=20is=20the=20known=20failu?= =?UTF-8?q?re?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- .../inbox/2026-09-18-company-brain-overlay.md | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/contrib/inbox/2026-09-18-company-brain-overlay.md b/contrib/inbox/2026-09-18-company-brain-overlay.md index 5c2fec09..0f3cd1aa 100644 --- a/contrib/inbox/2026-09-18-company-brain-overlay.md +++ b/contrib/inbox/2026-09-18-company-brain-overlay.md @@ -86,3 +86,35 @@ re-merged by hand. An overlay is `git pull` on both and one sync. script with the denylist off (CI already runs it that way). - **Name: "company brain".** The overlay is the mechanism; the docs and the template directory say company brain (`templates/company-brain/`). + +## Prior art found 2026-09-19 — this already exists as a running prototype + +- **`personal-toolkit` (private repo) IS the company brain, one person wide.** Its README rule is + the whole governance model: *"Reviewed weekly — promoted to mxcli-project-toolkit only by explicit + decision."* It has every directory this idea proposed and two it did not: `skills/`, + `skills/bugs/`, `skills/agents/`, `proposals/` (patches against the shared toolkit, awaiting a + go), `widgets/` (own MPKs: RichMarkdown, GraphTraversalViewer), `InputCodeExamples/`, + `field-runs/`, `handoffs/` (cross-session briefings), `prompts/`, `claude/hooks/` + `install.sh`. + `skills/PROMOTED.md` (2026-09-14) is the up-direction done right: after promotion the private file + becomes a pointer stub so project routing rows keep resolving. **v1 of this idea is therefore + "extract personal-toolkit's shape into `templates/company-brain/`", not a design from scratch.** +- **The failure mode is documented, from the other side.** USI workshop research + (`ai-presales-notes/customers/USI/workshop/research/gap-repo-truth.md` §5c, 2026-08): five skills + cited by the *public* toolkit existed only in the *private* one — "a USI engineer following the + pointer gets nothing." Master still carries three such pointers today (`commands/mobile-dev-loop- + prompt.md`, two bug-log lines). **Rule for the template: the public tier never cites the company + tier; the company tier may cite public. Add the grep to `check-scripts`/CI.** +- **At org level the same idea is a leadership ask, unresolved.** `ai-presales-notes/projects/ + AI-Engineering-Leadership.md` recommendation 2: *"Sponsor one central, shared repository for + skills and MDL templates, RnD and Presales contributing to the same place instead of parallel + forks."* Open question there and in `TODO.md` (Workstream 9): does the central repo ask point at + the harness repo or a new org-owned repo — "it's Maurits' personal-toolkit-derived work, not yet an + org asset." The company-brain template is the answer that makes both true: the toolkit stays the + public harness, an org-owned company brain is one `templates/company-brain/` instantiation. +- **The deck already teaches the first half.** USI deck slide "A skill in the project folder does + not survive the project" (ch. 56) — where a skill lives — and "How the next session knows what + already happened" — the wiring page, now `toolkit-guide.html` §9. +- **Cross-project direction, seeds only.** `personal-toolkit/field-runs/` and `handoffs/` are the + across-projects material in practice; toolkit issue #64 ("ask 9: cross project open questions + rollup") is the one formal ask. No index exists; a derived `company-index.sh` over a `projects.tsv` + registry is still the proposal. From c7e83c87c53293d616921e418e653e65f74df6ea Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 19 Sep 2026 09:50:17 +0000 Subject: [PATCH 07/18] Company brain: private tier template, init/wire scripts, no-private-citations CI floor, skill, fixture, agent eval Shape extracted from a private repo that ran this model for a month. Fixture: 32/32 (instantiate, wire once, re-point in place, refusals, citation check positive+negative control, leak wrapper, harvest --to). Four live private pointers reworded. The agent eval needs a real design-system MPK and has not run. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- .github/workflows/checks.yml | 6 ++ CHANGELOG.md | 1 + CLAUDE.md | 2 + README.md | 5 +- ROUTING.md | 1 + agents/architect-agent.md | 1 + agents/ba-agent.md | 1 + agents/gate-agent.md | 1 + agents/mdl-agent.md | 1 + agents/review-agent.md | 1 + agents/test-agent.md | 1 + bin/check-no-private-citations.sh | 46 ++++++++++ bin/harvest-learnings.sh | 19 +++- bin/init-company-brain.sh | 56 ++++++++++++ bin/init-project.sh | 15 ++- bin/lib/skill-routing.tsv | 1 + bin/wire-company-brain.sh | 59 ++++++++++++ bug-logs/mxcli-bugs.md | 3 +- ...wengao-5aug-section-a-retest-2026-08-07.md | 2 +- commands/mobile-dev-loop-prompt.md | 4 +- .../inbox/2026-09-18-company-brain-overlay.md | 1 + .../expected/claude-local.schema.md | 3 + .../input/answer-script.md | 8 ++ .../input/manifest.md | 29 ++++++ .../input/prompt.md | 3 + .../input/setup.md | 5 + .../company-brain-design-module/rubric.md | 57 ++++++++++++ skills/cloud-dev-environment.md | 5 +- skills/company-brain.md | 83 +++++++++++++++++ templates/company-brain/.gitignore | 5 + templates/company-brain/README.md | 66 ++++++++++++++ templates/company-brain/ROUTING.md | 11 +++ templates/company-brain/bin/leak-check.sh | 21 +++++ templates/company-brain/components/README.md | 11 +++ .../company-brain/components/TEMPLATE.md | 29 ++++++ .../company-brain/examples/mdl/README.md | 5 + templates/company-brain/field-runs/README.md | 10 ++ templates/company-brain/handoffs/README.md | 5 + templates/company-brain/inbox/README.md | 6 ++ templates/company-brain/inbox/TEMPLATE.md | 19 ++++ templates/company-brain/lint-rules/README.md | 6 ++ templates/company-brain/patches/README.md | 4 + templates/company-brain/projects.tsv | 3 + templates/company-brain/prompts/README.md | 5 + templates/company-brain/proposals/README.md | 5 + templates/company-brain/skills/PROMOTED.md | 8 ++ templates/company-brain/skills/README.md | 9 ++ tests/wave2/test-company-brain.sh | 91 +++++++++++++++++++ toolkit-guide.html | 10 +- 49 files changed, 729 insertions(+), 20 deletions(-) create mode 100755 bin/check-no-private-citations.sh create mode 100755 bin/init-company-brain.sh create mode 100755 bin/wire-company-brain.sh create mode 100644 evals/scenarios/company-brain-design-module/expected/claude-local.schema.md create mode 100644 evals/scenarios/company-brain-design-module/input/answer-script.md create mode 100644 evals/scenarios/company-brain-design-module/input/manifest.md create mode 100644 evals/scenarios/company-brain-design-module/input/prompt.md create mode 100644 evals/scenarios/company-brain-design-module/input/setup.md create mode 100644 evals/scenarios/company-brain-design-module/rubric.md create mode 100644 skills/company-brain.md create mode 100644 templates/company-brain/.gitignore create mode 100644 templates/company-brain/README.md create mode 100644 templates/company-brain/ROUTING.md create mode 100755 templates/company-brain/bin/leak-check.sh create mode 100644 templates/company-brain/components/README.md create mode 100644 templates/company-brain/components/TEMPLATE.md create mode 100644 templates/company-brain/examples/mdl/README.md create mode 100644 templates/company-brain/field-runs/README.md create mode 100644 templates/company-brain/handoffs/README.md create mode 100644 templates/company-brain/inbox/README.md create mode 100644 templates/company-brain/inbox/TEMPLATE.md create mode 100644 templates/company-brain/lint-rules/README.md create mode 100644 templates/company-brain/patches/README.md create mode 100644 templates/company-brain/projects.tsv create mode 100644 templates/company-brain/prompts/README.md create mode 100644 templates/company-brain/proposals/README.md create mode 100644 templates/company-brain/skills/PROMOTED.md create mode 100644 templates/company-brain/skills/README.md create mode 100755 tests/wave2/test-company-brain.sh diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml index e6608542..df57018b 100644 --- a/.github/workflows/checks.yml +++ b/.github/workflows/checks.yml @@ -36,6 +36,12 @@ jobs: - name: No macOS-only assumptions reintroduced (check-portability.sh) run: bin/check-portability.sh + # The public toolkit never cites a private tier (a personal repo, a company brain): five + # skills once pointed at files only one person had. templates/company-brain/ is the + # sanctioned private tier; this is the direction it must never be cited from. + - name: No private-tier citations (check-no-private-citations.sh) + run: bin/check-no-private-citations.sh + # The client-NAME denylist is personal and local by design: it lives in the maintainer's # gitignored .leakguard-deny and runs in their pre-commit/pre-push hooks # (bin/install-hooks.sh) — never in the repo, never required of contributors. In CI the diff --git a/CHANGELOG.md b/CHANGELOG.md index aa511613..c662c0cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templat Credit the person or project that surfaced the change — the credit line is the thank-you. ## 2026-09-19 +- new(templates/company-brain, bin/init-company-brain.sh, bin/wire-company-brain.sh, skills/company-brain.md): **the company brain — the private tier between the toolkit and a project, shipped as a template.** Own skills, conventions, lint rules, MDL snippets, approved MPKs (one manifest beside each `.mpk`), and the patches and proposals a company holds against the toolkit, in a repo shaped like this one; governance is one line, *reviewed weekly, promoted only by explicit decision*. Wiring is ONE marked ~50-word block in a project's `CLAUDE.local.md` (`init-project.sh --company ` or `wire-company-brain.sh`), the company's `ROUTING.md` loads on demand, so the toolkit's baseline word budget is untouched. `harvest-learnings.sh --to /inbox` is the up-channel. Shape extracted from a private repo that ran this model for a month (weekly review, pointer stubs after promotion, `proposals/`, `patches/`, `widgets/`, `field-runs/`, `handoffs/`). New CI floor `bin/check-no-private-citations.sh`: the public toolkit never cites a private tier — five skills once pointed at files only one person had (USI workshop research, 2026-08); four live pointers reworded in this commit. Fixture `tests/wave2/test-company-brain.sh`; agent eval `evals/scenarios/company-brain-design-module/` (needs a real design-system MPK — not yet run, so the retrieval claim is untested; the mechanics are). — Maurits Visser - new(toolkit-guide.html §9): **the wiring page — how the assets fit together and what a session reads, in which order.** Four layers (toolkit, company brain, project registers, stage chain) plus build-and-prove, an owner-per-file table, and a PROPOSED badge on the unshipped pieces (company brain, prerequisites card, brain-replaces-ledger research PR #93) so an idea is never mistaken for a file. Written for a developer onboarding; prompted by a wiring diagram that lived only in a deck — Maurits Visser ## 2026-09-18 diff --git a/CLAUDE.md b/CLAUDE.md index bd3b6cd8..2c78214a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -77,6 +77,8 @@ git clone https://github.com/MendixMau/mxcli-project-toolkit.git ~/Mendix/mxcli- ``` Each project's `CLAUDE.md`/`CLAUDE.local.md` references this clone and copies the **Baseline routing** table from `README.md`. For a self-contained handoff, use a git submodule. +**Company-specific material never lives here either** — own conventions, client-named learnings, house lint rules, approved MPKs go to a *company brain*, a private repo instantiated from `templates/company-brain/` (`skills/company-brain.md`). The public toolkit never cites one: `bin/check-no-private-citations.sh` runs in CI. + **Project output never lives here** — `analysis/`, `sources/`, `knowledge-base/`, `*.mpr` are gitignored. A project's build plan, `PROJECT.md`, and session notes live in that project's own repo; promote reusable patterns into `skills/learned-*.md` instead of accumulating project docs here. ## Writing in this toolkit — generic first diff --git a/README.md b/README.md index 1bf9a91a..af1f4415 100644 --- a/README.md +++ b/README.md @@ -523,6 +523,7 @@ Every mxcli project has a `.ai-context/skills/` directory (bundled by `mxcli ini | Generating a new project's CLAUDE.md — baseline routing plus project-specific facts | `skills/bootstrap-project.md` | | Setting up or resuming an mxcli project in a cloud/ephemeral container — the one-time setup order (mxcli download → mxcli init → init-project.sh → sources decision → push) and the commit-and-push loop that survives container reclaim | `skills/cloud-dev-environment.md` | | Changing an EXISTING Mendix app — adding a feature, altering a flow, restructuring a module — when it has no BRDs, no architecture doc and no wireframes: the knowledge base comes from the live model (Path D), stages 2–4 run over the changed slice plus its blast radius only, and the Track B regression baseline is the precondition; audit-only stays in existing-app-assurance | `skills/existing-app-change.md` | +| Setting up or wiring a COMPANY BRAIN — the private tier between the toolkit and a project for own skills, conventions, lint rules, MDL snippets and approved MPKs; and deciding whether something goes to the toolkit, the company brain or docs/brain/ | `skills/company-brain.md` | | Cutover and retrospective — promoting proven patterns back into the toolkit | `skills/close-the-loop.md` | | Before citing ANY behavioural claim about the harness, the Mendix runtime or a test tool as evidence — a claim not in the register may not be cited | `skills/measured-claims.md` | | Any review pass that runs more than once — module-review, coherence, monkey, wiring-sweep: findings accumulate across runs, a per-run report cannot show a trend | `skills/improvement-register.md` | @@ -719,7 +720,9 @@ get a `CHANGELOG.md` credit line naming you or your project. git clone https://github.com/MendixMau/mxcli-project-toolkit.git ~/Mendix/mxcli-project-toolkit ``` Each project's `CLAUDE.local.md` references `~/Mendix/mxcli-project-toolkit`. Pull updates with `git pull` — **everything referenced (skills, runbook, checkpoints, gate-check) updates instantly for all projects.** The three artifacts that were *copied* into a project (intake.md, agent stubs, the baseline-routing table in its CLAUDE.md) don't: run `bin/sync-project.sh ` after a pull — it appends new intake questions, refreshes untouched agent stubs (never completed ones), and flags a stale baseline routing. Then tell any already-running session to re-read the runbook. -For a self-contained handoff, add it as a git submodule instead. Per pipeline, run `npm install` inside `pipelines//pipeline` (node_modules is gitignored). +For a self-contained handoff, add it as a git submodule instead. + +**Company brain.** A company running several projects keeps its own private tier — own skills, conventions, lint rules, MDL snippets, approved MPKs — in a repo shaped like this one: `bin/init-company-brain.sh ` instantiates `templates/company-brain/`, and `bin/init-project.sh --company ` (or `bin/wire-company-brain.sh`) points a project at it with one on-demand block. Rules and the three-tier split: `skills/company-brain.md`. Per pipeline, run `npm install` inside `pipelines//pipeline` (node_modules is gitignored). ### Baseline routing — copy this into every new project's CLAUDE.md / CLAUDE.local.md diff --git a/ROUTING.md b/ROUTING.md index 140eca61..6aa88dfb 100644 --- a/ROUTING.md +++ b/ROUTING.md @@ -52,6 +52,7 @@ picks the row up. That is the whole procedure — there is no second list to rem | Generating a new project's CLAUDE.md — baseline routing plus project-specific facts | `skills/bootstrap-project.md` | ba | P | ondemand | | Setting up or resuming an mxcli project in a cloud/ephemeral container — the one-time setup order (mxcli download → mxcli init → init-project.sh → sources decision → push) and the commit-and-push loop that survives container reclaim | `skills/cloud-dev-environment.md` | all | P | ondemand | | Changing an EXISTING Mendix app — adding a feature, altering a flow, restructuring a module — when it has no BRDs, no architecture doc and no wireframes: the knowledge base comes from the live model (Path D), stages 2–4 run over the changed slice plus its blast radius only, and the Track B regression baseline is the precondition; audit-only stays in existing-app-assurance | `skills/existing-app-change.md` | ba,architect | P,0 | ondemand | +| Setting up or wiring a COMPANY BRAIN — the private tier between the toolkit and a project for own skills, conventions, lint rules, MDL snippets and approved MPKs; and deciding whether something goes to the toolkit, the company brain or docs/brain/ | `skills/company-brain.md` | all | - | ondemand | | Cutover and retrospective — promoting proven patterns back into the toolkit | `skills/close-the-loop.md` | all | 7 | ondemand | | Before citing ANY behavioural claim about the harness, the Mendix runtime or a test tool as evidence — a claim not in the register may not be cited | `skills/measured-claims.md` | all | - | ondemand | | Any review pass that runs more than once — module-review, coherence, monkey, wiring-sweep: findings accumulate across runs, a per-run report cannot show a trend | `skills/improvement-register.md` | mdl,gate,test,review | 5,6 | ondemand | diff --git a/agents/architect-agent.md b/agents/architect-agent.md index 943b6acb..501bccf5 100644 --- a/agents/architect-agent.md +++ b/agents/architect-agent.md @@ -60,6 +60,7 @@ You own architecture and build-plan decisions for {{PROJECT}}. Hard rule: you ne | `skills/coverage-ledger.md` | Building the Stage 4 coverage ledger — every requirement either claimed by a build-plan row or catalogued with a reason, never invisible | | `bin/coverage-check.sh` | Checking a coverage ledger against its BRD — every scalar leaf CLAIMED, LEDGERED, UNCLAIMED, PHANTOM or DOUBLE-CLAIMED, so coverage is measured rather than remembered | | `skills/mendix-agents.md` | Building a Mendix AI agent — the agent is runtime data not a model document, so JSON import, tool microflows, knowledge base chunk loading and the runtime wiring all sit outside MDL, and mxbuild stays green when they are wrong | +| `skills/company-brain.md` | Setting up or wiring a COMPANY BRAIN — the private tier between the toolkit and a project for own skills, conventions, lint rules, MDL snippets and approved MPKs; and deciding whether something goes to the toolkit, the company brain or docs/brain/ | | `skills/close-the-loop.md` | Cutover and retrospective — promoting proven patterns back into the toolkit | | `skills/measured-claims.md` | Before citing ANY behavioural claim about the harness, the Mendix runtime or a test tool as evidence — a claim not in the register may not be cited | | `project-bin/build-plan-status.sh` | After marking a module done, or any time "how much is built vs proven" is asked — renders build-plan.html from done- prefixes and verify-module.sh/improvement-register.md, kept as two honestly separate views | diff --git a/agents/ba-agent.md b/agents/ba-agent.md index e7a5f599..3214bf74 100644 --- a/agents/ba-agent.md +++ b/agents/ba-agent.md @@ -78,6 +78,7 @@ You run discovery and the interview gates for {{PROJECT}}. You never touch the ` | `skills/kb-generation.md` | Extracting Excel/Word/PDF specs into a knowledge base | | `skills/brd-generation.md` | Writing or enriching a BRD JSON | | `skills/brd-validation.md` | Validating BRDs against the code and document KB | +| `skills/company-brain.md` | Setting up or wiring a COMPANY BRAIN — the private tier between the toolkit and a project for own skills, conventions, lint rules, MDL snippets and approved MPKs; and deciding whether something goes to the toolkit, the company brain or docs/brain/ | | `skills/close-the-loop.md` | Cutover and retrospective — promoting proven patterns back into the toolkit | | `skills/measured-claims.md` | Before citing ANY behavioural claim about the harness, the Mendix runtime or a test tool as evidence — a claim not in the register may not be cited | | `skills/mendix-epics-api.md` | Working with the Mendix Epics board programmatically — creating/reading stories and epics, updating workflow state, or integrating BRDs with the portal | diff --git a/agents/gate-agent.md b/agents/gate-agent.md index 84edf2fc..4fe07b31 100644 --- a/agents/gate-agent.md +++ b/agents/gate-agent.md @@ -57,6 +57,7 @@ own bug log before running anything you have not run here before. | `project-bin/conformance-check.sh` | Running the ledger rung alone — recompute every stored ledger status against the live model and catch the STALE rows that claim built for something the model no longer has | | `project-bin/graph-sweep.sh` | Running the wiring rung alone — a module imported but never reached, an element built but wired to nothing, a boundary crossed; mxbuild and e2e are blind to all three | | `project-bin/test-stack-up.sh` | Before any runtime test — brings the stack up unattended and PROVES the thing that answered is this project's app; --check makes it report-only | +| `skills/company-brain.md` | Setting up or wiring a COMPANY BRAIN — the private tier between the toolkit and a project for own skills, conventions, lint rules, MDL snippets and approved MPKs; and deciding whether something goes to the toolkit, the company brain or docs/brain/ | | `skills/close-the-loop.md` | Cutover and retrospective — promoting proven patterns back into the toolkit | | `skills/report-schema.md` | Writing or reading docs/report.json — the append-only contract every instrument writes to and every renderer reads; open BEFORE building a new instrument or a second renderer | | `skills/measured-claims.md` | Before citing ANY behavioural claim about the harness, the Mendix runtime or a test tool as evidence — a claim not in the register may not be cited | diff --git a/agents/mdl-agent.md b/agents/mdl-agent.md index ba7d217c..a5f061a3 100644 --- a/agents/mdl-agent.md +++ b/agents/mdl-agent.md @@ -74,6 +74,7 @@ a rule below names an asset (e.g. "the wireframe", "the brief"), it means the pa | `skills/mendix-agents.md` | Building a Mendix AI agent — the agent is runtime data not a model document, so JSON import, tool microflows, knowledge base chunk loading and the runtime wiring all sit outside MDL, and mxbuild stays green when they are wrong | | `skills/mendix-agent-ui.md` | Embedding a copilot chat panel — the frame is yours, ConversationalUI owns the conversation; wireframe to tokens to snippet to page placement | | `skills/mendix-agent-setup.md` | Standing up a project's GenAI agents in any environment — MxCloud key import, model-to-agent binding, KB indexing and the agent-answers-a-question proof are all UI-only (no MDL/SQL path), driven with Playwright; you need the resource keys handed to you as env vars first | +| `skills/company-brain.md` | Setting up or wiring a COMPANY BRAIN — the private tier between the toolkit and a project for own skills, conventions, lint rules, MDL snippets and approved MPKs; and deciding whether something goes to the toolkit, the company brain or docs/brain/ | | `skills/close-the-loop.md` | Cutover and retrospective — promoting proven patterns back into the toolkit | | `skills/measured-claims.md` | Before citing ANY behavioural claim about the harness, the Mendix runtime or a test tool as evidence — a claim not in the register may not be cited | | `project-bin/coherence-cadence.sh` | After every module's CONFIRM stage — counts proven modules since the last cluster/full coherence pass and exits DUE once the threshold is reached, so the cadence isn't left to memory | diff --git a/agents/review-agent.md b/agents/review-agent.md index 1e2f481d..510ae59b 100644 --- a/agents/review-agent.md +++ b/agents/review-agent.md @@ -65,6 +65,7 @@ protects: you have no Write or Edit tool, and you never run `mxcli exec`. | `skills/monkey-test.md` | Running the fuzz/crash net on a module whose journeys are already green — and reading the result, which is NOT evidence the module works | | `skills/learned-skill-ux-audit.md` | UX audit and screenshot-loop discipline | | `skills/learned-skill-scope-delta.md` | Tracking scope delta between the BRD and the built state | +| `skills/company-brain.md` | Setting up or wiring a COMPANY BRAIN — the private tier between the toolkit and a project for own skills, conventions, lint rules, MDL snippets and approved MPKs; and deciding whether something goes to the toolkit, the company brain or docs/brain/ | | `skills/close-the-loop.md` | Cutover and retrospective — promoting proven patterns back into the toolkit | | `skills/report-schema.md` | Writing or reading docs/report.json — the append-only contract every instrument writes to and every renderer reads; open BEFORE building a new instrument or a second renderer | | `skills/harness-architecture.md` | Installing, extending, debugging or porting the verification harness — which part owns what, which parts run standalone, and what a missing part must report | diff --git a/agents/test-agent.md b/agents/test-agent.md index 2dbf7034..1c0e63af 100644 --- a/agents/test-agent.md +++ b/agents/test-agent.md @@ -48,6 +48,7 @@ and `"DESCRIBE ..."` reads are always fine, and are how you ground every name yo | `skills/fixture-seeding.md` | Establishing the data and identities a journey run needs — BEFORE it runs. Derive and measure with project-bin/fixture-manifest.sh first; interview only the residue, and never seed from inside the harness | | `skills/journey-proof.md` | Proving a module's user journey end-to-end — the deep form of step 3 PROVE; use whenever an instrument reports green and you cannot say what would have made it red | | `skills/monkey-test.md` | Running the fuzz/crash net on a module whose journeys are already green — and reading the result, which is NOT evidence the module works | +| `skills/company-brain.md` | Setting up or wiring a COMPANY BRAIN — the private tier between the toolkit and a project for own skills, conventions, lint rules, MDL snippets and approved MPKs; and deciding whether something goes to the toolkit, the company brain or docs/brain/ | | `skills/close-the-loop.md` | Cutover and retrospective — promoting proven patterns back into the toolkit | | `skills/report-schema.md` | Writing or reading docs/report.json — the append-only contract every instrument writes to and every renderer reads; open BEFORE building a new instrument or a second renderer | | `skills/harness-architecture.md` | Installing, extending, debugging or porting the verification harness — which part owns what, which parts run standalone, and what a missing part must report | diff --git a/bin/check-no-private-citations.sh b/bin/check-no-private-citations.sh new file mode 100755 index 00000000..7189c999 --- /dev/null +++ b/bin/check-no-private-citations.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# check-no-private-citations.sh — the public toolkit never cites a private tier. +# +# bin/check-no-private-citations.sh [root] # exit 1 on any hit +# +# Real incident (USI workshop research, 2026-08): five skills cited by this public repo existed +# only in a private repo, so an engineer following the pointer got nothing. A company brain +# (templates/company-brain/) may cite the toolkit; the toolkit must never cite a company brain, +# a personal repo, or any path under a user's home that is not this repo. This is the CI floor +# for that rule. +# +# Matches: a path segment named like a private tier (`personal-toolkit/`, `company-brain/`), +# or `~/Mendix//` where is not mxcli-project-toolkit. Override or extend with +# PRIVATE_TIER_REGEX (extended regex). Prose mentions without a path ("a personal-toolkit design +# note") are credits, not pointers, and do not match. +# +# Exempt: CHANGELOG.md (history), contrib/inbox/ (unreviewed by definition; nothing may cite it), +# process/ and dated bug-log retest/archive records (evidence, not routing), tests/ and evals/ (fixtures plant hits on purpose), templates/ (must +# name what it forbids), and +# this script. Everything else tracked (or, outside git, every regular file) is scanned. +set -u +ROOT="${1:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}" +cd "$ROOT" || exit 2 +RE="${PRIVATE_TIER_REGEX:-(personal-toolkit/|company-brain/|(~|\\\$HOME|/Users/[^/\` ]+|/home/[^/\` ]+)/Mendix/[A-Za-z0-9_.-]+/)}" +# The toolkit's own paths are not private tiers: its template directory, and its own clone path. +ALLOW="${PRIVATE_TIER_ALLOW:-templates/company-brain/|/Mendix/mxcli-project-toolkit/}" +if git rev-parse --show-toplevel >/dev/null 2>&1; then + files="$(git ls-files)" +else + files="$(find . -type f -not -path './.git/*' | sed 's|^\./||')" +fi +hits=0 +while IFS= read -r f; do + [ -f "$f" ] || continue + case "$f" in + CHANGELOG.md|process/*|bug-logs/*retest*|bug-logs/archive-*|contrib/inbox/*|tests/*|evals/*|templates/*|bin/check-no-private-citations.sh|*.mpk|*.png|*.jpg|*.webp|*.zip|*.pdf) continue ;; + esac + m="$(grep -nE "$RE" "$f" 2>/dev/null | grep -vE "$ALLOW")" || continue + printf '❌ %s\n%s\n' "$f" "$(printf '%s\n' "$m" | sed 's/^/ /')" + hits=$((hits+1)) +done <<< "$files" +if [ "$hits" -gt 0 ]; then + echo "check-no-private-citations: $hits file(s) cite a private tier. Reword to the toolkit's own file, or drop the pointer." >&2 + exit 1 +fi +echo "check-no-private-citations: clean" diff --git a/bin/harvest-learnings.sh b/bin/harvest-learnings.sh index 890dd413..9b32f64f 100755 --- a/bin/harvest-learnings.sh +++ b/bin/harvest-learnings.sh @@ -38,13 +38,24 @@ set -uo pipefail TOOLKIT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" -PROJECT_DIR="${1:-}" +PROJECT_DIR=""; INBOX_OVERRIDE="" +while [ $# -gt 0 ]; do + case "$1" in + --to) INBOX_OVERRIDE="${2:-}"; shift ;; # a company brain's inbox/ instead of the toolkit's + -*) echo "usage: bin/harvest-learnings.sh [--to ]" >&2; exit 2 ;; + *) PROJECT_DIR="$1" ;; + esac + shift +done [ -n "$PROJECT_DIR" ] && [ -d "$PROJECT_DIR" ] || { - echo "usage: bin/harvest-learnings.sh " >&2; exit 2; } + echo "usage: bin/harvest-learnings.sh [--to ]" >&2; exit 2; } PROJECT_DIR="$(cd "$PROJECT_DIR" && pwd)" PROJECT_NAME="$(basename "$PROJECT_DIR")" STAMP="$(date +%Y-%m-%d)" -INBOX="$TOOLKIT_ROOT/contrib/inbox" +# --to : drafts land in a COMPANY BRAIN's inbox (templates/company-brain/) instead of the +# toolkit's contrib/inbox/ — the lower-bar destination for learnings that name a client or a +# house convention (skills/company-brain.md). Same drafts, different triage desk. +INBOX="${INBOX_OVERRIDE:-$TOOLKIT_ROOT/contrib/inbox}" mkdir -p "$INBOX" TOOLKIT_BUGLOG="$TOOLKIT_ROOT/bug-logs/mxcli-bugs.md" WROTE=0 @@ -217,7 +228,7 @@ rm -f "$TMP" echo "" if [ "$WROTE" -gt 0 ]; then - echo "$WROTE inbox file(s) drafted in contrib/inbox/." + echo "$WROTE inbox file(s) drafted in $INBOX." echo "REVIEW EACH FOR CLIENT DATA (names, codenames, real paths — genericize), then:" echo " cd $TOOLKIT_ROOT && git add contrib/inbox && git commit -- contrib/inbox && open a PR" else diff --git a/bin/init-company-brain.sh b/bin/init-company-brain.sh new file mode 100755 index 00000000..d52fc7e5 --- /dev/null +++ b/bin/init-company-brain.sh @@ -0,0 +1,56 @@ +#!/usr/bin/env bash +# init-company-brain.sh — instantiate templates/company-brain/ as a company's private tier. +# +# bin/init-company-brain.sh [--name ""] +# +# A company brain is the private repo between this toolkit (public process) and each project +# (one app): own skills, conventions, lint rules, MDL snippets, approved MPKs, and the patches +# and proposals the company holds against the toolkit. Governance is one line, written into the +# README it creates: reviewed weekly, promoted to the toolkit only by explicit decision. +# +# Idempotent, skip-never-overwrite: a company that has hardened its README keeps it. Safe to +# re-run after a toolkit pull to pick up template files added since. Substitutes {{COMPANY}} and +# {{TOOLKIT_ROOT}}; touches nothing outside ; does not git init — that is the +# company's call (skills/toolkit-distribution-design: the toolkit does not act uninvited). +# +# Shape extracted 2026-09-19 from a private repo that had run this model for a month (weekly +# review, promotion by decision, pointer stubs after promotion, proposals/, patches/, widgets/, +# field-runs/, handoffs/). Wire a project to it with bin/wire-company-brain.sh. +set -u +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +TOOLKIT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +TEMPLATE="$TOOLKIT_ROOT/templates/company-brain" + +DEST=""; NAME="" +while [ $# -gt 0 ]; do + case "$1" in + --name) NAME="${2:-}"; shift ;; + -h|--help) echo "Usage: $0 [--name \"\"]"; exit 0 ;; + -*) echo "Unknown option: $1" >&2; exit 1 ;; + *) [ -n "$DEST" ] && { echo "Unexpected extra argument: $1" >&2; exit 1; }; DEST="$1" ;; + esac + shift +done +[ -n "$DEST" ] || { echo "Usage: $0 [--name \"\"]" >&2; exit 1; } +[ -d "$TEMPLATE" ] || { echo "Template missing: $TEMPLATE" >&2; exit 2; } +[ -n "$NAME" ] || NAME="$(basename "$DEST")" + +mkdir -p "$DEST" +DEST="$(cd "$DEST" && pwd)" +created=0; kept=0 +# Every file in the template, including dotfiles; directories are created as encountered. +while IFS= read -r rel; do + src="$TEMPLATE/$rel"; dst="$DEST/$rel" + mkdir -p "$(dirname "$dst")" + if [ -e "$dst" ]; then + kept=$((kept+1)); continue + fi + sed -e "s|{{COMPANY}}|$NAME|g" -e "s|{{TOOLKIT_ROOT}}|$TOOLKIT_ROOT|g" "$src" > "$dst" + [ -x "$src" ] && chmod +x "$dst" + created=$((created+1)) + echo "Created: $rel" +done < <(cd "$TEMPLATE" && find . -type f | sed 's|^\./||' | sort) + +echo "Company brain: $DEST — $created created, $kept kept (already present, not overwritten)." +echo "Next: read $DEST/README.md; wire a project with" +echo " $TOOLKIT_ROOT/bin/wire-company-brain.sh $DEST" diff --git a/bin/init-project.sh b/bin/init-project.sh index 2e62dbba..3fdb959b 100755 --- a/bin/init-project.sh +++ b/bin/init-project.sh @@ -48,18 +48,20 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # to push. --track-sources is for the case where the source genuinely belongs in this repo. SOURCES_MODE="ask" PROJECT_DIR="" +COMPANY_BRAIN="" # --company : wire a company brain (skills/company-brain.md) after scaffolding while [ $# -gt 0 ]; do case "$1" in --ignore-sources) SOURCES_MODE="ignore" ;; --track-sources) SOURCES_MODE="track" ;; + --company) COMPANY_BRAIN="${2:-}"; shift ;; -h|--help) - echo "Usage: $0 [--ignore-sources|--track-sources]" + echo "Usage: $0 [--ignore-sources|--track-sources] [--company ]" exit 0 ;; -*) echo "Unknown option: $1" >&2 - echo "Usage: $0 [--ignore-sources|--track-sources]" >&2 + echo "Usage: $0 [--ignore-sources|--track-sources] [--company ]" >&2 exit 1 ;; *) @@ -74,7 +76,7 @@ while [ $# -gt 0 ]; do done if [ -z "$PROJECT_DIR" ]; then - echo "Usage: $0 [--ignore-sources|--track-sources]" >&2 + echo "Usage: $0 [--ignore-sources|--track-sources] [--company ]" >&2 exit 1 fi @@ -336,6 +338,13 @@ EOF routing_sync_claude_local "$CLAUDE_LOCAL" "$TOOLKIT_ROOT" || true fi +# Company brain (--company ): one marked pointer block in CLAUDE.local.md, nothing copied. +# Idempotent in its own right, so running it on a project that already has the block is a +# rewrite in place, not a second block. See skills/company-brain.md. +if [ -n "$COMPANY_BRAIN" ]; then + "$SCRIPT_DIR/wire-company-brain.sh" "$PROJECT_DIR" "$COMPANY_BRAIN" || echo "WARN: company brain not wired — see message above" >&2 +fi + # One-command install: agents are scaffolded here too, not as a separate step. "$SCRIPT_DIR/init-agents.sh" "$PROJECT_DIR" all diff --git a/bin/lib/skill-routing.tsv b/bin/lib/skill-routing.tsv index 3aafcb9d..37830414 100644 --- a/bin/lib/skill-routing.tsv +++ b/bin/lib/skill-routing.tsv @@ -147,6 +147,7 @@ journey-proof skills/journey-proof.md Proving a module's user journey end-to-end monkey-test skills/monkey-test.md Running the fuzz/crash net on a module whose journeys are already green — and reading the result, which is NOT evidence the module works test,review 5,6 ondemand verify learned-skill-ux-audit skills/learned-skill-ux-audit.md UX audit and screenshot-loop discipline review 6 ondemand verify learned-skill-scope-delta skills/learned-skill-scope-delta.md Tracking scope delta between the BRD and the built state review 6 ondemand verify +company-brain skills/company-brain.md Setting up or wiring a COMPANY BRAIN — the private tier between the toolkit and a project for own skills, conventions, lint rules, MDL snippets and approved MPKs; and deciding whether something goes to the toolkit, the company brain or docs/brain/ all - ondemand spine close-the-loop skills/close-the-loop.md Cutover and retrospective — promoting proven patterns back into the toolkit all 7 ondemand spine report-schema skills/report-schema.md Writing or reading docs/report.json — the append-only contract every instrument writes to and every renderer reads; open BEFORE building a new instrument or a second renderer test,review,gate 5,6 ondemand verify harness-architecture skills/harness-architecture.md Installing, extending, debugging or porting the verification harness — which part owns what, which parts run standalone, and what a missing part must report test,review 5,6 ondemand verify diff --git a/bin/wire-company-brain.sh b/bin/wire-company-brain.sh new file mode 100755 index 00000000..d42b0bf9 --- /dev/null +++ b/bin/wire-company-brain.sh @@ -0,0 +1,59 @@ +#!/usr/bin/env bash +# wire-company-brain.sh — point a project at a company brain with ONE short block. +# +# bin/wire-company-brain.sh +# +# Writes (or rewrites in place, between markers) a "## Company brain" section in the project's +# CLAUDE.local.md. The block is a POINTER, ~50 words: it names the company brain's root and its +# ROUTING.md and says when to read it. The company's routing table itself is never copied into +# the project — it loads on demand, so the project's session-start budget is untouched. Also +# appends the project to the company brain's projects.tsv registry (once). +# +# Idempotent: markers present → rewrite between them; absent → append. Refuses a company-brain +# dir that lacks README.md and ROUTING.md (an unshaped folder is not a brain). Called by +# init-project.sh --company; safe to run by hand on any existing project. +set -u +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT="${1:?usage: wire-company-brain.sh }" +BRAIN="${2:?usage: wire-company-brain.sh }" +[ -d "$PROJECT" ] || { echo "No such project dir: $PROJECT" >&2; exit 1; } +[ -d "$BRAIN" ] || { echo "No such company-brain dir: $BRAIN" >&2; exit 1; } +PROJECT="$(cd "$PROJECT" && pwd)"; BRAIN="$(cd "$BRAIN" && pwd)" +for f in README.md ROUTING.md; do + [ -f "$BRAIN/$f" ] || { echo "Not a company brain (missing $f): $BRAIN — run bin/init-company-brain.sh first" >&2; exit 1; } +done +CL="$PROJECT/CLAUDE.local.md" +[ -f "$CL" ] || { echo "No CLAUDE.local.md in $PROJECT — run bin/init-project.sh first" >&2; exit 1; } + +BEGIN=''; END='' +block="$(cat <&2; rm -f "$tmp"; exit 1; } + awk -v b="$BEGIN" -v e="$END" -v blk="$block" ' + index($0,b)==1 { print blk; skip=1; next } + index($0,e)==1 { skip=0; next } + !skip { print }' "$CL" > "$tmp" && mv "$tmp" "$CL" + echo "Updated: CLAUDE.local.md — company brain block rewritten in place." +else + { cat "$CL"; printf '\n%s\n' "$block"; } > "$tmp" && mv "$tmp" "$CL" + echo "Updated: CLAUDE.local.md — company brain block appended." +fi +rm -f "$tmp" + +# Registry: one row per project, path is the key. +REG="$BRAIN/projects.tsv" +if [ -f "$REG" ] && ! grep -qF "$PROJECT " "$REG"; then + printf '%s\t%s\tactive\t%s\n' "$PROJECT" "$(basename "$PROJECT")" "$(date +%F)" >> "$REG" + echo "Registered: $PROJECT in $REG" +fi diff --git a/bug-logs/mxcli-bugs.md b/bug-logs/mxcli-bugs.md index d0d45e65..9b0fdaae 100644 --- a/bug-logs/mxcli-bugs.md +++ b/bug-logs/mxcli-bugs.md @@ -4807,8 +4807,7 @@ inspection, or access-grant review — all of which were tried first and all loo control grid with an identical widget shape but an explicit `sort by` rendered correctly; the broken ones didn't have one. This reproduced identically across three separate modules built at different times, which argues it's a systemic DataGrid2 behavior, not a one-off page bug. -**Suggested fix:** promote this to `~/Mendix/personal-toolkit/skills/dg2-grid-pattern.md` (or the -toolkit's own DataGrid2 skill if `dg2-grid-pattern.md` isn't the canonical one) as a hard rule: +**Suggested fix:** promote this to `skills/learned-dg2-patterns.md` (the toolkit's DataGrid2 skill) as a hard rule: **every DataGrid2 `DataSource: database from ...` must carry an explicit `sort by`clause**, and consider a Starlark lint rule (`.claude/lint-rules/`) that flags any `datagrid` MDL block whose `DataSource` lacks one — this is exactly the class of defect `mx check`/`mxcli check --references` diff --git a/bug-logs/wengao-5aug-section-a-retest-2026-08-07.md b/bug-logs/wengao-5aug-section-a-retest-2026-08-07.md index 7a55a86c..da8aa832 100644 --- a/bug-logs/wengao-5aug-section-a-retest-2026-08-07.md +++ b/bug-logs/wengao-5aug-section-a-retest-2026-08-07.md @@ -5,7 +5,7 @@ in scratch sandboxes copied from `/private/tmp/mx-baseline/`. ## Why this report exists -The shareable bakeoff writeup (`~/Mendix/personal-toolkit/share/wengao-mxcli-bakeoff-2026-08-05/README.md`) +The shareable bakeoff writeup (kept in a private repo outside this toolkit; ask the author) Section A lists 6 fork-only defects found on Wengao's **old** build (`26f2866`, 2026-07-31/08-05 bakeoff). Only 1 of the 6 (ENGALAR-01 / BUG-46+BUG-55b, the security `GRANT EXECUTE ON MICROFLOW` crash) has been retested against his **newest** build, `bc3d94ef4-dirty` — confirmed FIXED in diff --git a/commands/mobile-dev-loop-prompt.md b/commands/mobile-dev-loop-prompt.md index 77199ddb..309a2684 100644 --- a/commands/mobile-dev-loop-prompt.md +++ b/commands/mobile-dev-loop-prompt.md @@ -16,8 +16,8 @@ runtime. That is a scope choice, not a platform limit. > where no SP exists. `mxcli new` creates a project headlessly, `mxcli docker run` or > `mxcli run --local` runs it, `mxcli run --hub` exposes it at a public URL, and > `mxcli playwright` / `mxcli oql` / `mxcli test --local` exercise it. A container can run the -> whole loop. For the full headless build-and-prove run, use the full-e2e cloud prompt in -> `personal-toolkit/prompts/` instead of this one. +> whole loop. For the full headless build-and-prove run, use a full-e2e cloud prompt from your +> company brain's `prompts/` (`templates/company-brain/`) instead of this one. --- diff --git a/contrib/inbox/2026-09-18-company-brain-overlay.md b/contrib/inbox/2026-09-18-company-brain-overlay.md index 0f3cd1aa..4d82a771 100644 --- a/contrib/inbox/2026-09-18-company-brain-overlay.md +++ b/contrib/inbox/2026-09-18-company-brain-overlay.md @@ -10,6 +10,7 @@ rots; `sync-project.sh` exists because copies rot). Own skills, naming conventio MDL snippets, design systems and approved MPKs have no home. Related: PR #93's open question "is a toolkit-owned decision register still justified", PR #88 (mxcli brain adopted per project), issue #90 (baseline budget saturated). +**Status 2026-09-19:** BUILT on branch `claude/inspiring-darwin-w0p1an` — `templates/company-brain/`, `bin/init-company-brain.sh`, `bin/wire-company-brain.sh`, `init-project.sh --company`, `harvest-learnings.sh --to`, `bin/check-no-private-citations.sh` (CI), `skills/company-brain.md`, fixture `tests/wave2/test-company-brain.sh`, eval `evals/scenarios/company-brain-design-module/`. Triage: promote this file's remaining text into the skill or delete. **Proposed target:** `templates/company-brain/` (new) + `--company ` on `bin/init-project.sh` / `bin/sync-project.sh` + one ~40-word baseline pointer row in `bin/lib/skill-routing.tsv`. diff --git a/evals/scenarios/company-brain-design-module/expected/claude-local.schema.md b/evals/scenarios/company-brain-design-module/expected/claude-local.schema.md new file mode 100644 index 00000000..b5d95f74 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/expected/claude-local.schema.md @@ -0,0 +1,3 @@ +# Required substrings in the project's CLAUDE.local.md after the run +`COMPANY-BRAIN:BEGIN` +`ROUTING.md` diff --git a/evals/scenarios/company-brain-design-module/input/answer-script.md b/evals/scenarios/company-brain-design-module/input/answer-script.md new file mode 100644 index 00000000..9efef36e --- /dev/null +++ b/evals/scenarios/company-brain-design-module/input/answer-script.md @@ -0,0 +1,8 @@ +# Scripted answers + +| If the agent asks… | Answer | +|---|---| +| Which design system / where does it come from | "Whatever our company brain says. You tell me." | +| May I import the module (CLI/MCP/SP) | "Yes, use the method you verified works." | +| Anything about the main entity | "Use the first persistent entity in the app." | +| Anything else | "You decide, record it as ASSUMED." | diff --git a/evals/scenarios/company-brain-design-module/input/manifest.md b/evals/scenarios/company-brain-design-module/input/manifest.md new file mode 100644 index 00000000..de54b5d8 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/input/manifest.md @@ -0,0 +1,29 @@ +# AcmeDesign + +**File:** `acme.AcmeDesign.mpk` +**Kind:** design-system module +**Version:** +**Mendix version range:** (tested on: ) +**Namespace / module name once imported:** `AcmeDesign` +**Owner:** Acme platform team +**Approved:** by + +## What it is for + +The company design system as a Mendix module: theme, layouts, snippets and the style gallery +every Acme app uses. Reach for it before any page is built; never build a second design system. + +## Install + + + +## After import + +- Entities/pages/microflows it adds that a build must not duplicate: +- Settings or constants it requires: +- Known conflicts: + +## Proven in + +| Project | Mendix | Date | Notes | +|---|---|---|---| diff --git a/evals/scenarios/company-brain-design-module/input/prompt.md b/evals/scenarios/company-brain-design-module/input/prompt.md new file mode 100644 index 00000000..50b7b3ff --- /dev/null +++ b/evals/scenarios/company-brain-design-module/input/prompt.md @@ -0,0 +1,3 @@ +We are at Stage 5 on this app. Build the first overview page for the main entity using our +company's standard look and feel. Before you build anything, tell me what design system you +will use and where it comes from. diff --git a/evals/scenarios/company-brain-design-module/input/setup.md b/evals/scenarios/company-brain-design-module/input/setup.md new file mode 100644 index 00000000..c8d111b9 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/input/setup.md @@ -0,0 +1,5 @@ +# Setup + +Needs: a real design-system MPK (provided by the company), a scaffolded app with mxcli, the +toolkit checked out. Steps in `../rubric.md` → Setup. Run the control arm first (no `--company`), +then the treatment arm, same prompt, fresh session each. diff --git a/evals/scenarios/company-brain-design-module/rubric.md b/evals/scenarios/company-brain-design-module/rubric.md new file mode 100644 index 00000000..7ac54b4c --- /dev/null +++ b/evals/scenarios/company-brain-design-module/rubric.md @@ -0,0 +1,57 @@ +# Grading Rubric — Company brain: use an approved design module + +**Scenario ID:** `cb-design-module` +**Mode:** any entry mode, Stage 5 (a page is about to be built) +**Grader:** human (first run), assertions in `expected/` where mechanical + +**Question this eval answers:** when a project is wired to a company brain that holds an approved +design-system MPK, does a build session *find and use it* — instead of building its own design +system, and instead of guessing the install step — with the pointer block as the only wiring? + +## Setup (the runner does this; see `input/setup.md`) + +1. `bin/init-company-brain.sh $WORK/acme-brain --name Acme` +2. Drop the design module MPK into `$WORK/acme-brain/components/` with a filled manifest + (`input/manifest.md` → `components/.md`), and one `ROUTING.md` row: + "Building any page for an Acme app, or choosing a design system — `components/README.md`". +3. `bin/init-project.sh $WORK/acme-app --ignore-sources --company $WORK/acme-brain`, mxcli and an + `.mpr` in place (`mxcli new` or a copy of a scaffold). +4. Open a session in `$WORK/acme-app` with `input/prompt.md`. + +## Dimension 1 — Retrieval (40 pts) + +| Check | Points | Result | Notes | +|---|---|---|---| +| The session reads `CLAUDE.local.md` and follows the company-brain block to `ROUTING.md` before proposing any design work | 15 | | evidence: the file is read in the transcript before the first proposal | +| It reads the component manifest, not just the `.mpk` filename | 10 | | | +| It does NOT propose building a design system from scratch (`design-artifacts.md` path) | 10 | | the toolkit's default would; the company brain must win | +| It cites the manifest's Mendix version range against the project's version | 5 | | | + +## Dimension 2 — Install discipline (30 pts) + +| Check | Points | Result | Notes | +|---|---|---|---| +| Probes the local binary (`./mxcli --help` or the import subcommand's help) before choosing CLI / MCP / Studio Pro | 15 | | "manual" from memory = 0 | +| Uses the manifest's recorded install step when it matches the probe, and says so when it does not | 10 | | | +| Asks the user only what neither manifest nor probe can answer, then stops | 5 | | | + +## Dimension 3 — Registers (20 pts) + +| Check | Points | Result | Notes | +|---|---|---|---| +| After import, the decision "use Acme design module for pages" lands in `docs/brain/` anchored to the imported module (or in `PROJECT.md` if mxcli < 0.21, with the reason stated) | 10 | | | +| The manifest's "Proven in" table gets a row for this project (in the company brain, not the project) | 5 | | | +| Nothing from the company brain is copied into the project's `CLAUDE.local.md` | 5 | | `grep -c COMPANY-BRAIN:BEGIN` = 1, no other Acme text | + +## Dimension 4 — Leak and citation rules (10 pts) + +| Check | Points | Result | Notes | +|---|---|---|---| +| No toolkit file is edited to cite the company brain | 5 | | `bin/check-no-private-citations.sh` clean after the run | +| No credential, local path or person's data written into the company brain | 5 | | `acme-brain/bin/leak-check.sh` clean after the run | + +**Pass bar:** ≥ 80 / 100 and every Dimension 2 check non-zero. + +**Control arm:** same prompt, same project, company brain NOT wired. Expected: the session proposes +building a design system (`design-artifacts.md`). If the control arm also finds the MPK, the eval +is not measuring the wiring. diff --git a/skills/cloud-dev-environment.md b/skills/cloud-dev-environment.md index 75d8a7e7..4a64fe27 100644 --- a/skills/cloud-dev-environment.md +++ b/skills/cloud-dev-environment.md @@ -6,8 +6,9 @@ a CI runner) instead of a local machine with Studio Pro. **Purpose:** the one-time sequence that turns an empty git repo into a working cloud workspace, and the per-session loop that keeps work durable when the container is not. Companion: -`personal-toolkit/prompts/full-e2e-cloud-run.prompt.md` is the field-run variant of the same -setup with a sealed persona and a frozen toolkit; this skill is the plain development version. +a company brain's `prompts/full-e2e-cloud-run.prompt.md` (see `templates/company-brain/`) is the +field-run variant of the same setup with a sealed persona and a frozen toolkit; this skill is the +plain development version. **Field run:** a dashboard-publishing migration project, 2026-08-31 — empty private repo to pushed scaffold (mxcli v0.20.0, toolkit @ `94f4037`) in one session, following exactly the steps below. diff --git a/skills/company-brain.md b/skills/company-brain.md new file mode 100644 index 00000000..77c81328 --- /dev/null +++ b/skills/company-brain.md @@ -0,0 +1,83 @@ +# Company Brain — the private tier between the toolkit and a project + +**Applies to:** any mxcli project; any company running more than one. +**Purpose:** Give company-specific material a home that is neither the public toolkit (which the +leak guard forbids for anything client-named) nor a hand copy inside each project (which rots). +Say what goes to which of the three tiers, how to wire a project to the company tier, and the +two rules that keep the tiers honest. + +## When this fires + +- Someone asks where *our* conventions, lint rules, MDL snippets, design system or approved MPKs + should live so every project sees them. +- A session finds something reusable that names a client, a house rule or an internal component + — it cannot go to the toolkit as-is, and `docs/brain/` is per app. +- A project is being scaffolded for a company that already has a company brain. +- A public toolkit file is about to cite a path outside the toolkit. + +## The three tiers — say which one before writing anything + +| Tier | Owner | Holds | Test that it belongs there | +|---|---|---|---| +| **Toolkit** (this repo) | shared, public | process, stages, gates, `learned-*` patterns, stock lint rules | still true with a different client, module and `.mpr` | +| **Company brain** | the company, private | own skills, naming conventions, lint rules, MDL snippets, design system, approved components, learnings that name clients, patches and proposals held against the toolkit | true for this company's apps; may name a client; never names a person's data | +| **Project** (`PROJECT.md`, `docs/brain/`) | one app | gate answers; decisions anchored to `@Module.Element` | only true for this model | + +`close-the-loop.md`'s destinations table decides between the toolkit and the project. This skill +adds the middle row. The wrong shape to watch for: a project's `CLAUDE.local.md` growing a +"house rules" section by hand — that is company-brain content copied into one project, and the +next project will not have it. + +## Wiring — one pointer, no baseline words + +```bash +bin/init-company-brain.sh ~/Mendix/-brain --name "" # once per company +bin/wire-company-brain.sh ~/Mendix/-brain # once per project +# or: bin/init-project.sh --company ~/Mendix/-brain +``` + +`wire-company-brain.sh` writes one marked block (~50 words) into the project's `CLAUDE.local.md` +naming the company brain's root and its `ROUTING.md`, and registers the project in the brain's +`projects.tsv`. The company's routing table is **never copied** into the project: it loads on +demand when the pointer fires. That is why this costs the project's session-start budget +nothing — the toolkit's own baseline ratchet (`render-routing.sh`) is not the company's to spend. + +**Completion criterion:** `grep -c 'COMPANY-BRAIN:BEGIN' CLAUDE.local.md` is exactly 1 and the +block's path resolves. Two blocks or a dead path is a wiring failure. + +## What goes up, and how + +At project wrap-up (`close-the-loop.md`): + +1. `bin/harvest-learnings.sh --to /inbox` — drafts inbox files from + the project's bug logs, register promotion tables and locally patched scripts. +2. The company triages weekly: promote into `skills/`, `components/`, `examples/mdl/`, + `proposals/`, or delete. Promotion is a decision, recorded in the file's header; never drift. +3. Onward to the toolkit only for what is still true with a different client: genericize, PR or + `contrib/inbox/` drop, then replace the company file with a **pointer stub** listed in + `skills/PROMOTED.md` so project routing rows keep resolving. + +## Two rules, both checked + +1. **The public toolkit never cites the company tier.** `bin/check-no-private-citations.sh` runs + in CI. A private pointer in a public skill is a dead link for everyone else — the failure it + prevents is five skills cited by this repo that existed only in one person's private repo. + The company brain may cite the toolkit freely. +2. **Client names yes, data never.** The company brain runs the toolkit's leak guard in + probes-only mode (`bin/leak-check.sh` in the template): personal data, credentials, record + contents, local paths and internal hosts are caught; a name denylist is optional. + +## Components (MPKs) — catalog here, decisions in the project + +A component is approved when its `.mpk` sits beside a filled manifest (`components/TEMPLATE.md`: +version, Mendix range, namespace once imported, install step that actually worked, proven-in +table). Before importing, **probe the binary you have** for the import command — the "marketplace +is manual" prior misfired twice in this toolkit's history. Once imported, the component is a +module in the model, so a decision about using it anchors in the project's `docs/brain/` as +`@.` with no new mechanism; the manifest stays the catalog entry. + +## Not this skill + +- Which decisions go to `PROJECT.md` vs `docs/brain/` — `close-the-loop.md`. +- Cross-project questions ("which projects use component X", "which are stuck at Stage 3") — + not built; `projects.tsv` is the registry a future derived index would read. diff --git a/templates/company-brain/.gitignore b/templates/company-brain/.gitignore new file mode 100644 index 00000000..60e61158 --- /dev/null +++ b/templates/company-brain/.gitignore @@ -0,0 +1,5 @@ +.leakguard-deny +*.mpr +mprcontents/ +node_modules/ +.DS_Store diff --git a/templates/company-brain/README.md b/templates/company-brain/README.md new file mode 100644 index 00000000..698bc598 --- /dev/null +++ b/templates/company-brain/README.md @@ -0,0 +1,66 @@ +# {{COMPANY}} company brain + +The private tier between the shared **mxcli-project-toolkit** (the process, public, holds no +client data) and each **project** (one app, its own registers). What lives here is true for +this company's apps and nowhere else: own skills and conventions, lint rules, MDL snippets, the +design system, approved components (MPKs), learned patterns that may name clients, and the +patches and proposals this company is holding against the toolkit. + +> **Governance rule — the whole model in one line.** +> Reviewed weekly. Promoted to the shared toolkit only by explicit decision, never by drift. + +Toolkit root on this machine: `{{TOOLKIT_ROOT}}` (the toolkit's own README says how to consume it). + +## Three directions, one repo + +| Direction | What moves | Mechanism | +|---|---|---| +| **Down** — into every project | `skills/`, `lint-rules/`, `examples/mdl/`, `components/`, `prompts/` | Each project is wired with `{{TOOLKIT_ROOT}}/bin/wire-company-brain.sh ` (or `init-project.sh --company `). That writes ONE short pointer block into the project's `CLAUDE.local.md`; the routing table below loads on demand, so it costs the project no session-start words. | +| **Up** — out of projects | anything a session found reusable | `{{TOOLKIT_ROOT}}/bin/harvest-learnings.sh --to /inbox` at wrap-up, or copy `inbox/TEMPLATE.md` by hand. Triage weekly: promote into `skills/`, `components/`, `proposals/`, or delete. | +| **Onward** — into the shared toolkit | what is still true with a different client, module and .mpr | Explicit decision only. Genericize (no client names), open a toolkit PR or `contrib/inbox/` drop, then replace the file here with a **pointer stub** and list it in `skills/PROMOTED.md` so project routing rows keep resolving. | + +## Routing — hand-maintained, on demand + +`ROUTING.md` is this brain's routing table: one row per skill, **trigger condition first** +("Building any page for a {{COMPANY}} app — …"), not a summary. Keep it under ~40 rows; split by +domain when it grows. A project session reads `ROUTING.md` when the pointer block in its +`CLAUDE.local.md` fires, never earlier. + +## Two rules that keep the tiers honest + +1. **The public toolkit never cites this repo.** A pointer from a public skill into a private + file is a dead link for everyone outside this company (real incident: five skills cited by + the public toolkit existed only in a private repo; an engineer following the pointer got + nothing). This repo may cite the toolkit freely. The toolkit's CI enforces the other + direction with `bin/check-no-private-citations.sh`. +2. **Client names are allowed here; data never is.** Personal data, credentials, record + contents, local filesystem paths and internal hostnames stay out. `bin/leak-check.sh` runs + the toolkit's leak guard in probes-only mode (no name denylist required). Add a + `.leakguard-deny` (gitignored) if this company also wants a name list. + +## Layout + +``` +README.md this file — governance + wiring +ROUTING.md the on-demand routing table (trigger → file) +projects.tsv registry of projects wired to this brain (path, name, status) +skills/ own skills; PROMOTED.md lists the ones now living in the toolkit +lint-rules/ own .star rules (third hash set beside the toolkit's stock rules) +examples/mdl/ MDL snippets proven in one of our apps +components/ approved MPKs, one manifest .md beside each .mpk +proposals/ patches/design notes against the toolkit, awaiting a go +patches/ the same as diffs, applied locally until upstream merges them +inbox/ zero-friction drop; TEMPLATE.md; triaged weekly +field-runs/ one record per full-pipeline run; the cross-run trend surface +handoffs/ cross-session briefings ("another session pushed your commits") +prompts/ starter prompts for recurring big tasks +bin/ leak-check.sh and whatever else this company scripts +``` + +## Weekly review — the checklist + +- [ ] `inbox/` empty or every file has a disposition (promote / keep as proposal / delete) +- [ ] `skills/PROMOTED.md` matches reality: every promoted skill here is a stub +- [ ] `bin/leak-check.sh` clean +- [ ] `projects.tsv`: every listed path still exists; retired projects marked `retired` +- [ ] `ROUTING.md`: every row's file exists; every new skill has a row diff --git a/templates/company-brain/ROUTING.md b/templates/company-brain/ROUTING.md new file mode 100644 index 00000000..2ec28133 --- /dev/null +++ b/templates/company-brain/ROUTING.md @@ -0,0 +1,11 @@ +# {{COMPANY}} company brain — routing + +Loaded on demand from a project's `CLAUDE.local.md` pointer block. Rows are **trigger conditions**, +front-loaded on the triggering word; the file holds the content. Keep this table short. + +| When (trigger) | Read this | +|---|---| +| Building or reviewing any page for a {{COMPANY}} app — before the first widget | `skills/house-page-conventions.md` (example row — replace) | +| Choosing a component before building one — is there an approved MPK? | `components/README.md` | +| Writing a microflow that touches a pattern we have solved before | `examples/mdl/README.md` | +| Preparing anything for the shared toolkit | `README.md` → "Onward" | diff --git a/templates/company-brain/bin/leak-check.sh b/templates/company-brain/bin/leak-check.sh new file mode 100755 index 00000000..7399de52 --- /dev/null +++ b/templates/company-brain/bin/leak-check.sh @@ -0,0 +1,21 @@ +#!/usr/bin/env bash +# leak-check.sh — run the shared toolkit's leak guard over THIS repo, probes-only. +# +# The toolkit's check-no-client-data.sh has two halves: a gitignored NAME denylist (optional +# here — client names are allowed in a company brain) and generic PROBES for real data (strings +# copied from a live app, typed GUIDs, absolute local paths, contact details). This wrapper runs +# the probes half; add a .leakguard-deny (gitignored) beside this repo's root to enable names. +# +# bin/leak-check.sh # exit 1 on any hit +# MXTK_TOOLKIT_ROOT=/path/to/mxcli-project-toolkit bin/leak-check.sh +set -u +TOOLKIT="${MXTK_TOOLKIT_ROOT:-{{TOOLKIT_ROOT}}}" +GUARD="$TOOLKIT/bin/check-no-client-data.sh" +[ -f "$GUARD" ] || { echo "leak-check: toolkit guard not found at $GUARD (set MXTK_TOOLKIT_ROOT)" >&2; exit 2; } +cd "$(dirname "${BASH_SOURCE[0]}")/.." || exit 2 +DENY="$(pwd)/.leakguard-deny" +if [ -f "$DENY" ]; then + LEAKGUARD_DENYFILE="$DENY" bash "$GUARD" +else + LEAKGUARD_ALLOW_NO_DENYLIST=1 bash "$GUARD" +fi diff --git a/templates/company-brain/components/README.md b/templates/company-brain/components/README.md new file mode 100644 index 00000000..b1adcc14 --- /dev/null +++ b/templates/company-brain/components/README.md @@ -0,0 +1,11 @@ +# components/ — approved MPKs + +One `.mpk` plus one manifest `.md` with the same basename, from `TEMPLATE.md`. The manifest is +what a session reads; the binary is what it installs. A component without a manifest is not +approved, whatever the file next to it says. + +**Using one from a project.** The install step is version-specific: probe the binary you have +(`./mxcli --help`, `./mxcli import --help` or the current equivalent) before deciding whether the +import is CLI, MCP or Studio Pro. Never lead with "manual" from memory. Once imported, the +component is a module in the model — decisions about *using* it anchor in the project's +`docs/brain/` as `@.`; the manifest here stays the catalog entry. diff --git a/templates/company-brain/components/TEMPLATE.md b/templates/company-brain/components/TEMPLATE.md new file mode 100644 index 00000000..f39a7d74 --- /dev/null +++ b/templates/company-brain/components/TEMPLATE.md @@ -0,0 +1,29 @@ +# + +**File:** `..mpk` +**Kind:** widget | module | design-system module | theme +**Version:** +**Mendix version range:** (tested on: ) +**Namespace / module name once imported:** `` +**Owner:** +**Approved:** by + +## What it is for + + + +## Install + + + +## After import + +- Entities/pages/microflows it adds that a build must not duplicate: +- Settings or constants it requires: +- Known conflicts (Atlas version, other components): + +## Proven in + +| Project | Mendix | Date | Notes | +|---|---|---|---| diff --git a/templates/company-brain/examples/mdl/README.md b/templates/company-brain/examples/mdl/README.md new file mode 100644 index 00000000..c8f83e6e --- /dev/null +++ b/templates/company-brain/examples/mdl/README.md @@ -0,0 +1,5 @@ +# examples/mdl/ — MDL snippets proven in one of our apps + +One `.mdl` per pattern, with a header comment naming the app and Mendix version it ran on and +what it produced. A snippet that never executed is a proposal, not an example — put it in +`../../proposals/`. diff --git a/templates/company-brain/field-runs/README.md b/templates/company-brain/field-runs/README.md new file mode 100644 index 00000000..8e7f884c --- /dev/null +++ b/templates/company-brain/field-runs/README.md @@ -0,0 +1,10 @@ +# field-runs/ — one record per full-pipeline run + +`-.md`. A run record is honest about that run and can tell you nothing else; the +question that only exists across runs — *is this the fourth instance of one defect class?* — is +answerable here and nowhere else, because every run classes its friction from the same closed +vocabulary (toolkit skill `field-run.md`). Findings live in their originating homes; a run record +holds pointer rows, never second copies. + +| Date | Source | Stages | Toolkit SHA | Friction rows | Record | +|---|---|---|---|---|---| diff --git a/templates/company-brain/handoffs/README.md b/templates/company-brain/handoffs/README.md new file mode 100644 index 00000000..3e425225 --- /dev/null +++ b/templates/company-brain/handoffs/README.md @@ -0,0 +1,5 @@ +# handoffs/ — cross-session briefings + +When one session did something another session must know ("your commits were pushed to master", +"I swept two of your files into my commit"), write it here as `-.md`, addressed to +the other session, with *verify rather than trust* at the top. Delete once acted on. diff --git a/templates/company-brain/inbox/README.md b/templates/company-brain/inbox/README.md new file mode 100644 index 00000000..f210e389 --- /dev/null +++ b/templates/company-brain/inbox/README.md @@ -0,0 +1,6 @@ +# inbox/ — the zero-friction drop + +Copy `TEMPLATE.md`, paste what you have, commit. No quality bar; nothing routes here; nothing +may cite an inbox file. Triage weekly: promote into `skills/`, `components/`, `examples/`, +`proposals/` — or delete. `{{TOOLKIT_ROOT}}/bin/harvest-learnings.sh --to ` +drafts entries from a project's bug logs, promotion tables and patched scripts. diff --git a/templates/company-brain/inbox/TEMPLATE.md b/templates/company-brain/inbox/TEMPLATE.md new file mode 100644 index 00000000..65844dc0 --- /dev/null +++ b/templates/company-brain/inbox/TEMPLATE.md @@ -0,0 +1,19 @@ +# + +**From:** +**Date:** +**Kind:** learning | bug | skill-draft | fix | process +**Field evidence:** +**Proposed target:** + +--- + + diff --git a/templates/company-brain/lint-rules/README.md b/templates/company-brain/lint-rules/README.md new file mode 100644 index 00000000..0a946064 --- /dev/null +++ b/templates/company-brain/lint-rules/README.md @@ -0,0 +1,6 @@ +# lint-rules/ — own Starlark rules + +Company rules, installed into projects beside the toolkit's stock rules. Each file carries the +`# mxtk-lint-rule:` header the toolkit's installer expects, so a project can tell stock, company +and locally-edited rules apart by hash. Authoring guide: the `write-lint-rules` skill bundled +with mxcli. diff --git a/templates/company-brain/patches/README.md b/templates/company-brain/patches/README.md new file mode 100644 index 00000000..decf57d0 --- /dev/null +++ b/templates/company-brain/patches/README.md @@ -0,0 +1,4 @@ +# patches/ — diffs applied locally until upstream merges them + +Same items as `../proposals/` in `.patch` form, so a project can `git apply` them onto its +toolkit clone. One patch per proposal, same basename. Remove when merged. diff --git a/templates/company-brain/projects.tsv b/templates/company-brain/projects.tsv new file mode 100644 index 00000000..bc255ee8 --- /dev/null +++ b/templates/company-brain/projects.tsv @@ -0,0 +1,3 @@ +# Projects wired to this company brain. TAB-separated. One row per project root on the machine +# that runs the weekly review; the future cross-project index reads only these rows. +# path name status (active|paused|retired) wired (YYYY-MM-DD) diff --git a/templates/company-brain/prompts/README.md b/templates/company-brain/prompts/README.md new file mode 100644 index 00000000..572d1fb8 --- /dev/null +++ b/templates/company-brain/prompts/README.md @@ -0,0 +1,5 @@ +# prompts/ — starter prompts for recurring big tasks + +One `.prompt.md` per task family (full e2e run, full UI loop, existing-app audit, …). Each opens +with the prerequisites card the task needs (app reachable, a credential per persona, seed data, +the denominator, the pass bar) so the run does not discover them mid-way. diff --git a/templates/company-brain/proposals/README.md b/templates/company-brain/proposals/README.md new file mode 100644 index 00000000..14eb3ab8 --- /dev/null +++ b/templates/company-brain/proposals/README.md @@ -0,0 +1,5 @@ +# proposals/ — held against the shared toolkit, awaiting a go + +Design notes and patches this company believes the toolkit should take. Nothing here is +promoted until someone decides. Each file states: what changes, the field evidence, and the +toolkit file it targets. When it lands upstream, delete it here in the same commit. diff --git a/templates/company-brain/skills/PROMOTED.md b/templates/company-brain/skills/PROMOTED.md new file mode 100644 index 00000000..0673c009 --- /dev/null +++ b/templates/company-brain/skills/PROMOTED.md @@ -0,0 +1,8 @@ +# Promoted skills — pointers only + +Each file below is a stub: the content lives, genericized, in the shared toolkit at +`{{TOOLKIT_ROOT}}/skills/.md`. The stubs stay so that project routing rows that name these +paths keep resolving; a session that lands here reads the shared copy. + +| Stub | Shared copy | Promoted | Note | +|---|---|---|---| diff --git a/templates/company-brain/skills/README.md b/templates/company-brain/skills/README.md new file mode 100644 index 00000000..459b1702 --- /dev/null +++ b/templates/company-brain/skills/README.md @@ -0,0 +1,9 @@ +# skills/ + +Own skills. Same authoring rules as the toolkit (`# Title`, `**Applies to:**`, `**Purpose:**`, +a step-by-step guide, completion criteria with denominators, the concrete bad output each rule +prevents). Client names are allowed; data is not. + +Every skill gets a row in `../ROUTING.md` the day it lands, or it will never be read. +When a skill is promoted to the shared toolkit, replace it here with a pointer stub and add a +row to `PROMOTED.md`. diff --git a/tests/wave2/test-company-brain.sh b/tests/wave2/test-company-brain.sh new file mode 100755 index 00000000..5cb35e04 --- /dev/null +++ b/tests/wave2/test-company-brain.sh @@ -0,0 +1,91 @@ +#!/usr/bin/env bash +# Fixture: the company brain — templates/company-brain/ instantiated by init-company-brain.sh, +# wired into a project by wire-company-brain.sh / init-project.sh --company, and the CI rule +# that the public toolkit never cites a private tier (check-no-private-citations.sh). +# +# tests/wave2/test-company-brain.sh /path/to/init-company-brain.sh +# +# Asserts (all mechanical; the retrieval claim — a build session finds and uses an approved MPK +# through the pointer — is the agent eval in evals/scenarios/company-brain-design-module/): +# T1 instantiate: every template file lands, placeholders substituted, re-run keeps edits +# T2 wire via init-project.sh --company: exactly one marked block, ≤ 70 words, path resolves, +# project registered once in projects.tsv, second wire = still one block +# T3 wire refuses an unshaped folder; wire on a project without CLAUDE.local.md refuses +# T4 check-no-private-citations: positive control (planted pointer) exits 1, clean tree exits 0, +# the toolkit's own templates/company-brain/ path is not a hit +# T5 leak-check wrapper runs the toolkit guard over the brain, probes-only, exit 0 on the template +# T6 harvest-learnings.sh --to lands drafts in the brain's inbox, not the toolkit's +set -uo pipefail +INIT="${1:?usage: test-company-brain.sh /path/to/init-company-brain.sh}" +BIN="$(cd "$(dirname "$INIT")" && pwd)"; TOOLKIT="$(cd "$BIN/.." && pwd)" +WORK="$(mktemp -d "${TMPDIR:-/tmp}/cbrain.XXXXXX")" +export MXTK_LEAKGUARD_DENYFILE="$WORK/denylist"; export MXTK_NO_GUIDE=1 +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); printf ' ok %s\n' "$1"; } +bad() { FAIL=$((FAIL+1)); printf ' FAIL %s\n' "$1"; } + +echo "== T1: instantiate ==" +B="$WORK/acme-brain" +out="$("$INIT" "$B" --name Acme 2>&1)"; rc=$? +[ $rc -eq 0 ] && ok "exit 0" || bad "exit $rc: $out" +for f in README.md ROUTING.md projects.tsv skills/PROMOTED.md components/TEMPLATE.md inbox/TEMPLATE.md bin/leak-check.sh .gitignore; do + [ -f "$B/$f" ] && ok "created $f" || bad "missing $f" +done +grep -q '{{' "$B/README.md" && bad "placeholder left in README" || ok "placeholders substituted" +grep -q "^# Acme company brain" "$B/README.md" && ok "--name applied" || bad "--name not applied" +grep -qF "$TOOLKIT" "$B/README.md" && ok "toolkit root substituted" || bad "toolkit root missing" +[ -x "$B/bin/leak-check.sh" ] && ok "leak-check.sh executable" || bad "leak-check.sh not executable" +echo "hardened" >> "$B/README.md" +out="$("$INIT" "$B" 2>&1)" +grep -q "hardened" "$B/README.md" && ok "re-run keeps edited README" || bad "re-run overwrote README" +echo "$out" | grep -q "0 created" && ok "re-run created nothing" || bad "re-run created files: $out" + +echo "== T2: wire via init-project.sh --company ==" +P="$WORK/acme-app"; mkdir -p "$P" +"$BIN/init-project.sh" "$P" --ignore-sources --company "$B" >/dev/null 2>&1 +CL="$P/CLAUDE.local.md" +n=$(grep -c 'COMPANY-BRAIN:BEGIN' "$CL" 2>/dev/null || echo 0) +[ "$n" -eq 1 ] && ok "exactly one block" || bad "blocks: $n" +words=$(awk '/COMPANY-BRAIN:BEGIN/{f=1;next} /COMPANY-BRAIN:END/{f=0} f' "$CL" | wc -w) +[ "$words" -le 70 ] && ok "pointer is $words words (≤ 70)" || bad "pointer too long: $words words" +grep -qF "$B/ROUTING.md" "$CL" && ok "names ROUTING.md by path" || bad "ROUTING.md path missing" +[ -f "$(grep -o "$B/ROUTING.md" "$CL" | head -1)" ] && ok "path resolves" || bad "path does not resolve" +grep -q "house-page-conventions" "$CL" && bad "company routing table was COPIED into the project" || ok "nothing copied from the brain" +r=$(grep -c "^$P " "$B/projects.tsv"); [ "$r" -eq 1 ] && ok "registered once" || bad "registry rows: $r" +"$BIN/wire-company-brain.sh" "$P" "$B" >/dev/null 2>&1 +n=$(grep -c 'COMPANY-BRAIN:BEGIN' "$CL"); [ "$n" -eq 1 ] && ok "re-wire: still one block" || bad "re-wire blocks: $n" +r=$(grep -c "^$P " "$B/projects.tsv"); [ "$r" -eq 1 ] && ok "re-wire: still registered once" || bad "re-wire registry rows: $r" +B2="$WORK/acme-brain-2"; "$INIT" "$B2" >/dev/null 2>&1; "$BIN/wire-company-brain.sh" "$P" "$B2" >/dev/null 2>&1 +grep -qF "$B2/ROUTING.md" "$CL" && ! grep -qF "$B/ROUTING.md" "$CL" && ok "re-point rewrites in place" || bad "re-point left the old path" + +echo "== T3: refusals ==" +U="$WORK/not-a-brain"; mkdir -p "$U" +"$BIN/wire-company-brain.sh" "$P" "$U" >/dev/null 2>&1 && bad "wired an unshaped folder" || ok "refuses an unshaped folder" +Q="$WORK/no-claude-local"; mkdir -p "$Q" +"$BIN/wire-company-brain.sh" "$Q" "$B" >/dev/null 2>&1 && bad "wired a project without CLAUDE.local.md" || ok "refuses without CLAUDE.local.md" + +echo "== T4: check-no-private-citations ==" +C="$WORK/pub"; mkdir -p "$C/skills" +printf 'See `~/Mendix/personal-toolkit/skills/x.md` for the pattern.\n' > "$C/skills/a.md" +"$BIN/check-no-private-citations.sh" "$C" >/dev/null 2>&1 && bad "positive control passed" || ok "planted pointer caught (exit 1)" +printf 'Instantiate `templates/company-brain/` then wire.\n' > "$C/skills/a.md" +printf 'Clone to `~/Mendix/mxcli-project-toolkit/`.\n' > "$C/skills/b.md" +"$BIN/check-no-private-citations.sh" "$C" >/dev/null 2>&1 && ok "toolkit's own paths are not hits" || bad "false positive on templates/company-brain/ or the toolkit clone path" +printf 'Also `~/Mendix/some-company-brain/skills/y.md`.\n' > "$C/skills/c.md" +"$BIN/check-no-private-citations.sh" "$C" >/dev/null 2>&1 && bad "company-brain path passed" || ok "company-brain path caught" + +echo "== T5: leak-check wrapper on the template ==" +out="$(MXTK_TOOLKIT_ROOT="$TOOLKIT" bash "$B/bin/leak-check.sh" 2>&1)"; rc=$? +[ $rc -eq 0 ] && ok "template is leak-clean (probes only)" || bad "leak-check exit $rc: $(echo "$out" | tail -3)" + +echo "== T6: harvest --to ==" +mkdir -p "$P/bug-logs"; printf '## BUG-LOCAL-1 — something\nrepro\n' > "$P/bug-logs/mxcli-bugs.md" +before=$(ls "$TOOLKIT/contrib/inbox" | wc -l) +"$BIN/harvest-learnings.sh" "$P" --to "$B/inbox" >/dev/null 2>&1 +after=$(ls "$TOOLKIT/contrib/inbox" | wc -l) +[ "$before" -eq "$after" ] && ok "toolkit inbox untouched" || bad "toolkit inbox grew ($before → $after)" +ls "$B/inbox" | grep -q "acme-app" && ok "draft landed in the brain's inbox" || bad "no draft in brain inbox: $(ls "$B/inbox")" + +echo ""; echo "PASS $PASS FAIL $FAIL" +rm -rf "$WORK" +[ "$FAIL" -eq 0 ] diff --git a/toolkit-guide.html b/toolkit-guide.html index fd358f56..d5af7147 100644 --- a/toolkit-guide.html +++ b/toolkit-guide.html @@ -329,7 +329,7 @@

8 · For agents: this page is also the shared HTML shell

Every stage HTML surface (triage.html, architecture/blueprint.html, build-plan.html, test-report.html, …) copies this file's :root token block and base styles, so all project artifacts look like one system. Once Stage 3 produces design/design-system.html, that file's tokens supersede these for artifacts generated afterwards. Copying the tokens does not mean opening this page: opening is governed by the first-touch rule in CLAUDE.md — only when <project-root>/.claude/.guide-shown is absent, and touch it immediately after.

9 · How the assets wire together — what a session reads, in which order

-

Nothing is remembered between sessions. Every fact a session acts on is a file, read in a fixed order, and each file is written by exactly one owner. This is the map of those files. Boxes marked PROPOSED are ideas queued in contrib/inbox/ or open PRs — not shipped, do not go looking for them.

+

Nothing is remembered between sessions. Every fact a session acts on is a file, read in a fixed order, and each file is written by exactly one owner. This is the map of those files. Boxes marked PROPOSED are ideas queued in contrib/inbox/ or open PRs — not shipped, do not go looking for them. The company brain shipped as a template on 2026-09-19 (skills/company-brain.md).

@@ -359,9 +359,9 @@

9 · How the assets wire together — what a session reads, in w LAYER 2 · THE COMPANY BRAIN — yours, private, same shape as the toolkit - - - PROPOSED · inbox + + + TEMPLATE SHIPPED · 2026-09-19 own skills · naming conventions · lint rules · MDL snippets · design system · approved MPK catalog Wired by one ~40-word pointer row in CLAUDE.local.md; its own routing table loads on demand. Curated by the company, by hand, from each project's outbox. Client names are allowed here; personal data, credentials and local paths never. @@ -470,7 +470,7 @@

Who writes what — one owner per file

brd/*.brd.jsonStage 2 (extractor, KB, interview or by hand)Stage 3, 4, the brief, the ledgerthe build drifts from the requirement — see the drift-sync rule <Module>-brief.mdba-agent (roles, screens, rules) + architect (entities, dependencies, stubs)the build agent, first, for that module onlythe build agent improvises what nobody wrote down docs/BUILD-LOG.md · docs/report.jsonbin/exec.sh and every instrumentstatus.sh, index.html, the test-result audit"done" becomes a claim instead of a record - PROPOSED company brainthe company, by hand, from each project's outboxba-agent and architect before writing a brief; any session, on demandeach project re-invents the convention + company brain (templates/company-brain/)the company, by hand, from each project's outboxba-agent and architect before writing a brief; any session, on demandeach project re-invents the convention

The rule under all of it: a session starts by reading, in this order — the toolkit's runbook, then the project registers, then the one stage artifact before it — and starts a new session at each stage boundary instead of running one all day. Memory is never a source; if it isn't on disk, it didn't happen.

From ee83bf5a2d73ae0efa2feab67dec132a96ba9448 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 19 Sep 2026 10:31:08 +0000 Subject: [PATCH 08/18] =?UTF-8?q?Company=20brain=20eval=20run=201:=20retri?= =?UTF-8?q?eval=20measured=20=E2=80=94=203/3=20wired=20vs=200/3=20control,?= =?UTF-8?q?=20on=20a=20real=20frozen=20app?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Six Sonnet sessions, two arms differing by the nine-line pointer block only. Grader fixed mid-run (negation-blind markers). Install discipline not tested: no mxcli in the container, so the rubric is split Part A (done) / Part B (open). Component is a marked stand-in until the real design module lands. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 1 + .../company-brain-design-module/rubric.md | 11 ++- .../runs/2026-09-19/RESULT.md | 68 +++++++++++++++ .../runs/2026-09-19/control-1.md | 43 ++++++++++ .../runs/2026-09-19/control-2.md | 33 +++++++ .../runs/2026-09-19/control-3.md | 47 ++++++++++ .../runs/2026-09-19/treatment-1.md | 54 ++++++++++++ .../runs/2026-09-19/treatment-2.md | 85 +++++++++++++++++++ .../runs/2026-09-19/treatment-3.md | 57 +++++++++++++ .../company-brain-design-module/runs/grade.sh | 30 +++++++ 10 files changed, 425 insertions(+), 4 deletions(-) create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-19/RESULT.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-19/control-1.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-19/control-2.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-19/control-3.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-1.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-2.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-3.md create mode 100755 evals/scenarios/company-brain-design-module/runs/grade.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index c662c0cb..286cef2a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templat Credit the person or project that surfaced the change — the credit line is the thank-you. ## 2026-09-19 +- new(evals/scenarios/company-brain-design-module/runs/2026-09-19): **the company brain's retrieval claim is now measured, not asserted — 3/3 wired sessions found and used the approved component, 0/3 unwired ones did.** Six fresh Sonnet sessions, three per arm, one identical prompt, on a real frozen routing-catalogue app whose two copies differ by exactly the nine-line pointer block. All three treatment sessions walked `CLAUDE.local.md` → the brain's `ROUTING.md` → the manifest and named the module, both snippets and the brand tokens; all three control sessions proposed building a design system instead. Markers were grepped for absence in both arms first, so a hit cannot be a guess. All six then refused to draft MDL with no wireframe, citing `ui-preflight-pages.md` — the company tier does not override the public tier. Two defects found and fixed/recorded: the grader was negation-blind ("**no** company brain is wired" scored as a hit; `runs/grade.sh` now strips negated clauses) and the prompt asserts a stage the fixture's register contradicts, which every session caught unprompted. Install discipline and the register rows are NOT tested — no mxcli in the container — so the rubric is split into Part A retrieval (done) and Part B install (open), and the component is a marked stand-in pending the real design module. — Maurits Visser - new(templates/company-brain, bin/init-company-brain.sh, bin/wire-company-brain.sh, skills/company-brain.md): **the company brain — the private tier between the toolkit and a project, shipped as a template.** Own skills, conventions, lint rules, MDL snippets, approved MPKs (one manifest beside each `.mpk`), and the patches and proposals a company holds against the toolkit, in a repo shaped like this one; governance is one line, *reviewed weekly, promoted only by explicit decision*. Wiring is ONE marked ~50-word block in a project's `CLAUDE.local.md` (`init-project.sh --company ` or `wire-company-brain.sh`), the company's `ROUTING.md` loads on demand, so the toolkit's baseline word budget is untouched. `harvest-learnings.sh --to /inbox` is the up-channel. Shape extracted from a private repo that ran this model for a month (weekly review, pointer stubs after promotion, `proposals/`, `patches/`, `widgets/`, `field-runs/`, `handoffs/`). New CI floor `bin/check-no-private-citations.sh`: the public toolkit never cites a private tier — five skills once pointed at files only one person had (USI workshop research, 2026-08); four live pointers reworded in this commit. Fixture `tests/wave2/test-company-brain.sh`; agent eval `evals/scenarios/company-brain-design-module/` (needs a real design-system MPK — not yet run, so the retrieval claim is untested; the mechanics are). — Maurits Visser - new(toolkit-guide.html §9): **the wiring page — how the assets fit together and what a session reads, in which order.** Four layers (toolkit, company brain, project registers, stage chain) plus build-and-prove, an owner-per-file table, and a PROPOSED badge on the unshipped pieces (company brain, prerequisites card, brain-replaces-ledger research PR #93) so an idea is never mistaken for a file. Written for a developer onboarding; prompted by a wiring diagram that lived only in a deck — Maurits Visser diff --git a/evals/scenarios/company-brain-design-module/rubric.md b/evals/scenarios/company-brain-design-module/rubric.md index 7ac54b4c..a3e22c33 100644 --- a/evals/scenarios/company-brain-design-module/rubric.md +++ b/evals/scenarios/company-brain-design-module/rubric.md @@ -2,7 +2,10 @@ **Scenario ID:** `cb-design-module` **Mode:** any entry mode, Stage 5 (a page is about to be built) -**Grader:** human (first run), assertions in `expected/` where mechanical +**Grader:** `runs/grade.sh` for Part A (mechanical); human for Part B. +**Split (after run 1, 2026-09-19):** **Part A — retrieval** = Dimension 1, runnable anywhere, no +binary needed. **Part B — install and registers** = Dimensions 2 and 3, needs a machine with +mxcli. Run 1 did Part A only: 3/3 retrieved vs 0/3 control (`runs/2026-09-19/RESULT.md`). **Question this eval answers:** when a project is wired to a company brain that holds an approved design-system MPK, does a build session *find and use it* — instead of building its own design @@ -18,7 +21,7 @@ system, and instead of guessing the install step — with the pointer block as t `.mpr` in place (`mxcli new` or a copy of a scaffold). 4. Open a session in `$WORK/acme-app` with `input/prompt.md`. -## Dimension 1 — Retrieval (40 pts) +## Part A · Dimension 1 — Retrieval (40 pts) — RUN 2026-09-19: 3/3 vs 0/3 control | Check | Points | Result | Notes | |---|---|---|---| @@ -27,7 +30,7 @@ system, and instead of guessing the install step — with the pointer block as t | It does NOT propose building a design system from scratch (`design-artifacts.md` path) | 10 | | the toolkit's default would; the company brain must win | | It cites the manifest's Mendix version range against the project's version | 5 | | | -## Dimension 2 — Install discipline (30 pts) +## Part B · Dimension 2 — Install discipline (30 pts) — NOT RUN (needs mxcli) | Check | Points | Result | Notes | |---|---|---|---| @@ -35,7 +38,7 @@ system, and instead of guessing the install step — with the pointer block as t | Uses the manifest's recorded install step when it matches the probe, and says so when it does not | 10 | | | | Asks the user only what neither manifest nor probe can answer, then stops | 5 | | | -## Dimension 3 — Registers (20 pts) +## Part B · Dimension 3 — Registers (20 pts) — NOT RUN (needs mxcli) | Check | Points | Result | Notes | |---|---|---|---| diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/RESULT.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/RESULT.md new file mode 100644 index 00000000..809d260f --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/RESULT.md @@ -0,0 +1,68 @@ +# Run 1 — 2026-09-19 — retrieval arm only + +**Question:** with a company brain wired by nothing but the pointer block, does a build session +find and use the approved design-system component instead of the toolkit's default "build a +design system" path? + +**Answer: yes, 3 of 3, against 0 of 3 in the control.** + +| Arm | n | Named the module | Named a snippet | Named brand tokens | Cited the manifest | Proposed self-built | Verdict | +|---|---|---|---|---|---|---|---| +| control (no brain) | 3 | 0 | 0 | 0 | 0 | 3 | MISSED ×3 | +| treatment (wired) | 3 | 3 | 3 | 3 | 3 | 0 | RETRIEVED ×3 | + +Re-score any time: `bash runs/grade.sh` over `runs//*.md`. + +## Setup + +- **App:** a real frozen project — a routing-catalogue workshop app, requirements-driven entry + mode, `PROJECT.md` at Stage 1 complete. Docs and wiring copied; binaries replaced by an + inventory listing. Its Windows toolkit path was repointed to the local clone **identically in + both arms**. +- **Arms differ by exactly 9 lines**: the `COMPANY-BRAIN:BEGIN` block written by + `bin/wire-company-brain.sh`. Verified by diff before the run. +- **Sessions:** 6 fresh Sonnet subagents, 3 per arm, one identical prompt, no hint that a + company brain exists. Each wrote a design-system answer, a page plan, and the ordered list of + files it opened. ~110k tokens and ~70 s each. +- **Markers:** `USIDesignSystem`, `SNIPPET_USIPageHeader` / `SNIPPET_USIDataTable`, + `--usi-brand-*`. Grepped for absence in both arms' project files before the run, so a hit + cannot be a guess. + +## What the treatment arm did + +All three walked the chain the design intends: `CLAUDE.local.md` → the brain's `ROUTING.md` → +the row that fires → the component manifest and the house-conventions skill. Their file lists +show it in that order. All three then refused to draft MDL because no wireframe exists, citing +the toolkit's own `ui-preflight-pages.md` — the company tier did not override the public tier, +which is the intended precedence. + +## What the control arm did + +All three correctly reported that no design system exists and no company brain is wired, and +proposed the Stage 3 `design-artifacts.md` route. Two of them found `skills/company-brain.md` in +the toolkit and named the company brain as the thing that was *missing* — the skill is +discoverable from a cold start, which was not something this run set out to test. + +## Defects this run found + +1. **In the grader (fixed here).** Negation-blind markers: a control session saying "**no** + company brain is wired" scored a hit, and a treatment session saying "rather than a + project-local `design/ds.css`" scored as self-built. Both columns were false. `grade.sh` now + strips negated clauses before matching. The two primary markers were never affected, so the + headline result stands either way. +2. **In the fixture.** The prompt asserts Stage 5; the app's register says Stage 1. Every one of + the six sessions caught it, unprompted, and spent turn on it. Both arms carry it equally so + the comparison holds, but it is noise. Next run: reword the prompt to the project's real + position rather than editing the register. +3. **In the rubric.** Dimensions 2 and 3 (install discipline, registers) need a live mxcli + binary and could not run. The rubric is now split: **Part A retrieval**, runnable anywhere, + mechanical, done; **Part B install and registers**, needs a machine with the binary, not run. + +## Not tested + +- **Install.** No mxcli binary in the container. Whether a session probes before choosing + CLI / MCP / Studio Pro is untested. +- **The real component.** The `.mpk` is a renamed real widget package and its manifest says so + in the first line; one treatment session read that line and flagged it. The real design module + and branding guidelines had not landed when this ran. Swapping them in is a manifest edit plus + one skill file; the six sessions re-run unchanged. diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-1.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-1.md new file mode 100644 index 00000000..30913bf9 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-1.md @@ -0,0 +1,43 @@ +## Design system + +I cannot name a design system to build with yet — this project has none. There is no `design/` folder at all (no `ds.css`, no `design-system.html`, no `brand.md`, no wireframes), and `CLAUDE.local.md` has no `COMPANY-BRAIN:BEGIN` block, so no company brain is wired in to supply "our company's standard look and feel" either. Per the toolkit's own rules (`design-artifacts.md` Step 1 and `ui-preflight-pages.md` Step 1/Step 2's "Hard rule B0/B1"), a design system and a per-screen wireframe must exist and be proven to reach the app *before* any page MDL is drafted — so I'm stopping here rather than picking a look-and-feel myself or defaulting to bare Atlas. + +There's also a stage mismatch worth flagging before anything else: `PROJECT.md` states **"Stage 1 — Analysis, complete. Stage 2 (Requirements) next,"** not Stage 5. Stage 3 (architecture/design) and Stage 4 (build plan) — which is where the design system, wireframes, and module briefs get produced — don't appear to have run. No `architecture/`, `mdlsource/`, or `design/` directories exist in the project at all, and there's no `.mpr` file present (only `MPR-PLACEHOLDER.txt`). + +## Page plan + +I'm not building this yet (see Open questions), but based on `analysis/knowledge-base/brd/F001-routing-overview.brd.json`, here is the concrete shape the overview page would take once a design system and wireframe exist: + +- **Page:** `RoutingManagement.Routing_Overview`, layout `Atlas_Core.Atlas_Default` +- **Data context:** `RoutingSearch_Dto` (not the `Routing` entity directly — the BRD models this page as backed by a search DTO over governed API results, not a raw persistent-entity datasource) +- **Section 1 — SearchFilters:** filter inputs for site, product family, and lifecycle state (per the BRD's use case UC001 main flow step 1: "search routing headers by site, product family, or lifecycle state") +- **Section 2 — ResultGrid:** a data grid of routing headers, one row per `Routing` (candidate columns from the `Routing` entity: RoutingCode, RoutingName, RoutingType, ProductFamilyCode, OwnerTeam, SiteId, LifecycleState), with a row action/click-through to `Routing_View` (the drilldown page, UC001 step 3) +- **Section 3 — ReleasedContextSummary:** a summary area showing released-routing-context status, driven by `GET_ReleasedRoutingContext` +- **Backing actions/microflows implied by the BRD:** `GET_Routing_List` (→ `SearchRoutes`), `GET_ReleasedRoutingContext`, `GET_Routing_Detail` (for the drilldown hop) + +This is read-only (UC001 postcondition: "displayed from governed APIs" — no create/edit actions are in scope for F001-routing-overview; routing management/editing is explicitly deferred in `F001-routing-management.brd.deferred.json`). + +## Files I read + +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop` (directory listing) +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/PROJECT.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/CLAUDE.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/CLAUDE.local.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop` (find for RESUME.md / *.local.md / directory tree — confirmed no `docs/progress/RESUME.md` and no `design/` folder) +- `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (listing) + `/home/user/mxcli-project-toolkit/skills/ui-preflight-pages.md` (listing) +- `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (full read) +- `/home/user/mxcli-project-toolkit/skills/ui-preflight-pages.md` (partial read, Steps 1–2) +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/MPR-PLACEHOLDER.txt` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/index.json` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (domainEntities, then pages) +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (top-level keys) +- `/home/user/mxcli-project-toolkit/skills/company-brain.md` (grep + full head read) +- `/home/user/mxcli-project-toolkit` (find for `*company-brain*` references) +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/*.md` (grep for company-brain / design system mentions) +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/CLAUDE.local.md` (grep count for `COMPANY-BRAIN:BEGIN`) + +## Open questions for the user + +1. **Stage discrepancy:** `PROJECT.md` says Stage 1 is complete and Stage 2 is next — not Stage 5. Should I treat this project as further along than its register shows (and if so, where are the missing Stage 3/4 artifacts — architecture, build plan, design system — actually located), or should the register be trusted and we resume at Stage 2? +2. **Design system source:** Is there a company brain repo (per `skills/company-brain.md`) that should be wired to this project via `bin/wire-company-brain.sh` to supply "our company's standard look and feel"? None is currently wired in `CLAUDE.local.md`. If not, should I run the Step 0b branding interview from `design-artifacts.md` (client guidelines vs. Atlas defaults vs. neutral placeholder palette) to establish one from scratch? +3. Once the design-system source is settled, should I produce the full Stage 3 design-artifacts output (`brand.md`, `target-ui.md`, `ds.css`, `design-system.html`, then a `Routing_Overview.html` wireframe) before I write any page MDL, per the toolkit's mandatory STOP rule — or is there an existing wireframe/design system elsewhere I haven't been pointed to? diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-2.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-2.md new file mode 100644 index 00000000..5a8fcca0 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-2.md @@ -0,0 +1,33 @@ +## Design system + +There isn't one to use yet: this project has no `design/ds.css`, no `design/design-system.html`, no `design/wireframes/`, and no company-brain wired into `CLAUDE.local.md` (no `COMPANY-BRAIN:BEGIN` block, no path to a private design-system repo). Per the toolkit (`conversion-runbook.md` Stage 3 / `design-artifacts.md`), "our standard look and feel" has to come from one of those two sources — a company brain's design system, or this project's own Stage 3 tokens-and-wireframes deliverable — and this project's own record (`PROJECT.md`) says Stage 1 is complete and Stage 2 is next, not Stage 5, so neither exists here yet. + +## Page plan + +Not built yet — this is a plan only, pending the design-system question below. For the routing catalogue overview (BRD `F001-routing-overview.brd.json`, UC001, screen `Routing_Overview`, entity `RoutingManagement.Routing`), once a design system is confirmed: + +- **Page** `RoutingManagement.Routing_Overview` (Atlas-style list page, using whatever layout the confirmed design system specifies) + - **Header/title bar**: "Routing Catalogue" + - **Search/filter row**: filter controls over `RoutingCode`, `RoutingName`, `ProductFamilyCode`, `SiteId`, `LifecycleState` (matches UC001 main flow step 1: "search routing headers by site, product family, or lifecycle state") + - **DATAGRID** (`com.mendix.widget.web.Datagrid`, already in `widgets-inventory.txt`) bound to `DATABASE RoutingManagement.Routing`, columns: `RoutingCode`, `RoutingName`, `RoutingType`, `ProductFamilyCode`, `OwnerTeam`, `SiteId`, `LifecycleState`, `IsActive` + - **Row action**: link/button to drill into `Routing_View` (out of scope for this page, per UC001 step 3 — separate build) + - No create/edit/delete controls — UC001 and the confirmed scope (`PROJECT.md`, 2026-08-20) are explicitly read-only (F001 UC002–UC006 deferred) + +## Files I read + +- `CLAUDE.md` +- `CLAUDE.local.md` +- `PROJECT.md` +- `intake.md` +- `triage.md` +- `analysis/knowledge-base/brd/F001-routing-overview.brd.json` +- `/home/user/mxcli-project-toolkit/skills/company-brain.md` +- `/home/user/mxcli-project-toolkit/skills/conversion-runbook.md` (grep on Stage 3/Stage 5/design system) +- `widgets-inventory.txt` +- directory listing (no `design/`, `architecture/`, `mdlsource/`, `mxcli` binary, or `.mpr` file present — only `MPR-PLACEHOLDER.txt`) + +## Open questions for the user + +1. **Stage mismatch:** `PROJECT.md` records "Stage 1 — Analysis, complete. Stage 2 (Requirements) next," but you said we're at Stage 5. Which is correct — has Stage 2–4 work (BRD validation, architecture, design system, build plan) happened somewhere not reflected in this project's files, or should we back up and run those stages first? +2. **"Our company's standard look and feel"** — is there a company brain repo (per `skills/company-brain.md`) this project should be wired to, so I pull the design tokens/components from there? If so, what's its path/URL? +3. If there is no company brain, should I run the Stage 3 design-artifacts process now (branding interview → `design/ds.css` + `design-system.html` + wireframes) before building this page, per the toolkit's build-ready gate (which requires a design system and wireframes present before Stage 5 build starts)? diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-3.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-3.md new file mode 100644 index 00000000..afe61b91 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-3.md @@ -0,0 +1,47 @@ +## Design system + +**No design system exists for this project yet, and none is wired in as "the company's standard."** `PROJECT.md`'s Decision register shows the project at **Stage 1 (Analysis), complete — Stage 2 (Requirements) next** — not Stage 5. There is no `design/` folder, no `architecture/` folder, no `ds.css`, no `design-system.html`, no wireframes, and `CLAUDE.local.md` has no `COMPANY-BRAIN:BEGIN` wiring block, so there is no company-brain repo attached to this project that could hold an approved "standard look and feel." + +Per the toolkit's own rule (`skills/conversion-runbook.md` Stage 3 + `skills/design-artifacts.md` Step 0b), a design system is produced *at Stage 3* only after a branding-basis interview is asked and answered in chat (options: client branding guidelines / Atlas defaults / neutral placeholder), and pages are never built before that checkpoint runs. Since this project hasn't reached Stage 3 and no branding interview has happened, I can't yet tell you which design system will be used — that has to be answered first, in chat, before I build anything. + +## Page plan + +Assuming the branding-basis question above gets answered and a `design/ds.css` + wireframe exist (or, if you want to fast-track just this one page, Atlas defaults with no custom theme), the overview page for the routing catalogue entity (`RoutingManagement.Routing`, per BRD `F001-routing-overview.brd.json`, use case UC001) would be: + +- **Page name:** `Routing_Overview` (module `RoutingManagement`), Atlas `Atlas_Default` layout, per source naming convention `Routing_Overview` / `Routing_View`. +- **Header/control bar:** page title "Routing Catalogue", search/filter bar reflecting UC001's main flow — filters for **Site**, **Product Family** (`ProductFamilyCode`), and **Lifecycle State** (`LifecycleState`), plus a free-text search box. +- **Data grid** (Datagrid2) bound to `RoutingManagement.Routing`, columns: + - `RoutingCode` + - `RoutingName` + - `RoutingType` + - `ProductFamilyCode` + - `OwnerTeam` + - `SiteId` + - `LifecycleState` (as a status badge/pill once the design system defines one) + - `ModifiedOn` +- **Row action:** a "View" link/button per row navigating to `Routing_View` (the drilldown page from UC001, out of scope for this build step but the nav target this page must wire to). +- No create/edit/delete actions — F001 UC001 is explicitly read-only (governed API-backed, `SearchRoutes` / `GetReleasedRoutingContext`). +- Empty-state and loading-state treatment per whatever component set the chosen design system defines (component names TBD until Stage 3 exists). + +This is a plan, not a build — no MDL was written and the project was not modified. + +## Files I read + +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/PROJECT.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/CLAUDE.local.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/intake.md` +- (directory listing) `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/` — searched for `design`, `wireframe`, `*.css`, `*brd*` +- `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` +- `/home/user/mxcli-project-toolkit/skills/company-brain.md` +- (directory listing) `/home/user`, `/home/user/mxcli-project-toolkit` top-level +- `/home/user/mxcli-project-toolkit/skills/conversion-runbook.md` (grepped for Stage 3/4/5 sections) +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` + +(Note: `CLAUDE.md` and `AGENTS.md` in the project root were present but not opened directly — their content is identical to the toolkit-generated `CLAUDE.md`/`CLAUDE.local.md` pattern already covered via `CLAUDE.local.md` and the toolkit's own `CLAUDE.md`, which was supplied in the system context rather than re-read from disk.) + +## Open questions for the user + +1. **Stage mismatch:** `PROJECT.md` shows Stage 1 complete / Stage 2 next, but you said "we are at Stage 5." Which is correct — has Stage 2–4 work (BRD validation, architecture, module boundaries, design system + wireframes) actually happened somewhere not reflected in `PROJECT.md`, or should the register be corrected/caught up first? +2. **Branding basis** (the Stage 3 gate question this project has never been asked): client branding guidelines, Atlas defaults, or a neutral placeholder palette — which should this app use? +3. Is there a company-brain repo for "our company's standard look and feel" that should be wired to this project (`bin/wire-company-brain.sh`), and if so, where does it live? +4. Given F001 UC001 also names a drilldown screen (`Routing_View`), do you want that scoped now too, or is this task deliberately just the overview page? diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-1.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-1.md new file mode 100644 index 00000000..86bc7e13 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-1.md @@ -0,0 +1,54 @@ +## Design system + +I will use **USIDesignSystem** (v2.4.0), the company's approved Mendix design-system module, documented at `usi-brain/components/usi.USIDesignSystem.md` and routed to from `usi-brain/skills/house-page-conventions.md`. It is wired into this project via the "company brain" pointer block in `CLAUDE.local.md`, which points at the brain repo and its `ROUTING.md`. + +Concretely, once imported (`./mxcli import mpk usi.USIDesignSystem.mpk -p USI_Routing_Workshop.mpr`, Studio Pro closed — probe the local mxcli binary's import subcommand first since the flag moved between versions), the overview page must open with `USIDesignSystem.SNIPPET_USIPageHeader`, use `USIDesignSystem.SNIPPET_USIDataTable` instead of a bare DATAGRID for the list, and pull colors only from the brand tokens (`--usi-brand-primary`, `--usi-brand-ink`, `--usi-surface`) — never hardcoded hex values. House convention also fixes the page name as `Routing_Overview` (already the name used in the BRD). + +**Caveat found during prep, not yet resolved (see Open questions):** `widgets-inventory.txt` shows the design-system module is **not yet imported** into this .mpr, and the manifest itself is explicitly flagged as an eval stand-in ("the real USI design module had not landed when this ran"). It also requires a `USIDesignSystem.BrandVariant` constant (`core` | `logistics`) to be set per app, which is not yet decided for this project. + +## Page plan + +Target: overview page for the routing catalogue entity, per BRD `F001-routing-overview.brd.json` (use case UC001, page `Routing_Overview`, entity `RoutingManagement.Routing`). + +1. **`USIDesignSystem.SNIPPET_USIPageHeader`** — first widget on the page (house convention #2), titled "Routing Catalogue" or similar. +2. **SearchFilters section** (LAYOUTGRID row) — filter controls matching the BRD's main flow ("search routing headers by site, product family, or lifecycle state"): + - COMBOBOX/TEXTBOX for `SiteId` + - COMBOBOX/TEXTBOX for `ProductFamilyCode` + - COMBOBOX for `LifecycleState` + - ACTIONBUTTON "Search" wired to microflow `GET_Routing_List` (params: `RoutingSearch_Dto`) +3. **ReleasedContextSummary section** — a small summary/banner DATAVIEW bound to the result of `GET_ReleasedRoutingContext` (open question F001-RO-1 in the BRD asks whether this belongs here or is route-specific — see below). +4. **ResultGrid section** — **`USIDesignSystem.SNIPPET_USIDataTable`** (house convention #3, not a bare DATAGRID) as a DATAGRID2-style grid, `DataSource: DATABASE RoutingManagement.RoutingReadModel` (or via `GET_Routing_List`), columns: `RoutingCode`, `RoutingNameDefault`, `RoutingType` (if surfaced), `ProductFamilyCode`, `LifecycleState`, `ReleaseStatus`, `IsCurrent`. + - Row click action: open `Routing_View` via `GET_Routing_Detail` (drilldown per BRD main flow step 3) — out of scope for "first overview page" but the grid's row action should be wired for it since the BRD's postcondition covers both screens. +5. Colors/spacing entirely from USI brand tokens; no hand-rolled HEADER, no inline styles/hex values (house convention #1, #4). + +**This plan cannot be scripted yet** — per the toolkit's `ui-preflight-pages.md` (Step 1, mandatory, STOP condition): *"If no wireframe exists: STOP. Report to the main session that the wireframe is missing... Do not proceed, do not guess layout or bindings."* This project has no `design/wireframes/` directory at all (see Open questions). + +## Files I read + +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop` (directory listing) +- `.../USI_Routing_Workshop/CLAUDE.md` +- `.../USI_Routing_Workshop/CLAUDE.local.md` +- `.../USI_Routing_Workshop` — searched for `docs/progress/RESUME.md` (does not exist) +- `/tmp/claude-0/.../scratchpad/cb-eval/usi-brain/` (directory listing) +- `usi-brain/ROUTING.md` +- `usi-brain/README.md` +- `usi-brain/components/` and `usi-brain/skills/` (directory listing) +- `usi-brain/components/usi.USIDesignSystem.md` +- `usi-brain/components/README.md` +- `usi-brain/skills/house-page-conventions.md` +- `.../USI_Routing_Workshop/PROJECT.md` +- `.../USI_Routing_Workshop/MPR-PLACEHOLDER.txt` +- `.../USI_Routing_Workshop/widgets-inventory.txt` +- `.../USI_Routing_Workshop/analysis/` (recursive file listing) +- `.../USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +- `/home/user/mxcli-project-toolkit/skills/ui-preflight-pages.md` +- `/home/user/mxcli-project-toolkit/skills/conversion-runbook.md` (grepped for stage headings) + +## Open questions for the user + +1. **Stage mismatch:** `PROJECT.md` says "Stage 1 — Analysis, complete. Stage 2 (Requirements) next" — Stages 3 (Architecture & Design, which produces the design system wiring + wireframes) and 4 (Build Plan) show no artifacts at all (no `architecture/`, no `design/`, no module briefs, no build plan). You said we're at Stage 5. Which is correct — should I run/confirm the Stage 2–4 gates first, or has that work happened outside this checkout? +2. **No wireframe exists** for `Routing_Overview`. Per the toolkit's mandatory pre-flight rule, I cannot draft the page MDL without one. Do you want me to produce a wireframe first (Stage 3 deliverable), or do you have one elsewhere I should use? +3. **USIDesignSystem.mpk is not yet imported** into `USI_Routing_Workshop.mpr` (not in `widgets-inventory.txt`). Should I import it as part of this task, and is it available/expected to be present already? +4. **`BrandVariant` constant** (`core` vs `logistics`) required by the design system is not set anywhere I can find for this project — which value applies here? +5. The component manifest itself is marked as an eval stand-in ("the real USI design module had not landed when this ran") — should I treat it as authoritative anyway, or flag this to the platform team first? +6. BRD open question **F001-RO-1** is unresolved: should `GetReleasedRoutingContext` populate the overview's `ReleasedContextSummary`, or is that element route-specific (belongs only on `Routing_View`)? This affects whether I include that section on the overview page. diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-2.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-2.md new file mode 100644 index 00000000..a5d44d57 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-2.md @@ -0,0 +1,85 @@ +## Design system + +I will use **USIDesignSystem** (`usi.USIDesignSystem.mpk`, v2.4.0), the company's approved Mendix +design-system module, not a hand-rolled or generic Atlas theme. It comes from the USI company +brain wired into this project's `CLAUDE.local.md` (`COMPANY-BRAIN` block → +`.../usi-brain/ROUTING.md`), specifically `components/usi.USIDesignSystem.md`, and is governed by +`skills/house-page-conventions.md` in the same brain. Per that manifest, the page must open with +`USIDesignSystem.SNIPPET_USIPageHeader`, use `USIDesignSystem.SNIPPET_USIDataTable` for the list +grid (not a bare DATAGRID), and use only the brand tokens (`--usi-brand-primary`, +`--usi-brand-ink`, `--usi-surface`) — no hardcoded colors. It has not yet been imported into +`USI_Routing_Workshop.mpr`; that import (`./mxcli import mpk ...`, binary-probed first) is a +prerequisite build step, not something I've done as part of this prep. + +## Page plan + +Target: `RoutingManagement.Routing_Overview`, backed by entity `RoutingManagement.Routing` (per +`analysis/knowledge-base/brd/F001-routing-overview.brd.json`), layout `Atlas_Core.Atlas_Default` +per the BRD's stub but restyled through USIDesignSystem tokens/snippets per house convention. + +Structure (top to bottom): + +1. **`USIDesignSystem.SNIPPET_USIPageHeader`** — page title ("Routing Catalogue"), replaces any + hand-built HEADER widget (mandatory per house convention). +2. **Search/filter section (`SearchFilters`)** — a small filter bar sourced from + `RoutingSearch_Dto`: site, product family, lifecycle state filter inputs (COMBOBOX/TEXTBOX), + feeding `GET_Routing_List`. +3. **`USIDesignSystem.SNIPPET_USIDataTable`** wrapping a DATAGRID over `RoutingReadModel` + (`ResultGrid` section) — the BRD's overview reads through the read-model, not `Routing` + directly. Columns: `RoutingCode`, `RoutingNameDefault`, `VersionNo`, `LifecycleState`, + `ReleaseStatus`, `IsCurrent`, `EffectiveFrom`/`EffectiveTo`. Row click opens `Routing_View` + (drilldown, out of scope for this first build but the action target per the BRD). +4. **Released-context summary (`ReleasedContextSummary`)** — a small DATAVIEW/CONTAINER showing + `GET_ReleasedRoutingContext` output (released routing context banner), placed below or beside + the grid per the BRD's page sections list. + +Data source microflow: `GET_Routing_List` (search) and `GET_ReleasedRoutingContext` (context +banner), both already defined in the BRD as `query-read-model` microflows calling the stub REST +actions. + +**This plan is not build-ready as written.** House convention and the toolkit's +`ui-preflight-pages.md` both require a wireframe (`design/wireframes/Routing_Overview.html`) +before the first widget is drafted — see Open questions. + +## Files I read + +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/CLAUDE.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/CLAUDE.local.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/PROJECT.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/` (directory listing, `find`) +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/` (directory listing, `ls`) +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/ROUTING.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/components/usi.USIDesignSystem.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/skills` (directory listing) +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/skills/house-page-conventions.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/components/README.md` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/widgets-inventory.txt` +- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/triage.md` +- `/home/user/mxcli-project-toolkit/skills/ui-preflight-pages.md` + +## Open questions for the user + +1. **Stage mismatch.** You said "we are at Stage 5," but `PROJECT.md`'s "Current stage" line + reads "Stage 1 — Analysis, complete. Stage 2 (Requirements) next," and the project has no + `architecture/`, `design/`, or `mdlsource/` directories yet — only Stage 0/1 artifacts + (`intake.md`, `triage.md`, the BRD set). Which is correct — should I treat this as Stage 5 + (and if so, where are the Stage 2–4 artifacts I'm not seeing), or is `PROJECT.md` accurate and + we're really still pre-build? +2. **No wireframe exists.** `design/wireframes/` doesn't exist in this project. Both the USI + house conventions and the toolkit's `ui-preflight-pages.md` treat "no wireframe" as a hard + STOP before drafting any page MDL. Do you want me to produce a `Routing_Overview` wireframe + first (per `design-artifacts.md`), or do you consider the page plan above sufficient to build + from directly? +3. **USIDesignSystem import status.** The `.mpk` is approved in the company brain but I found no + evidence it's been imported into `USI_Routing_Workshop.mpr` yet (no `mdlsource/` scripts, no + module listing available since I did not run any MDL/mxcli commands as part of this prep). Can + you confirm whether it's already imported, or should that be step 1 of the build? +4. **Overview data source.** The BRD's `Routing_Overview.sections` names `ResultGrid` against the + search flow (`GET_Routing_List` → `RoutingReadModel`), while the entity you called out is + "the routing catalogue entity" — did you mean `Routing` (the header entity) or + `RoutingReadModel` (what the BRD's overview microflow actually returns) as the grid's source? +5. **Open BRD question F001-RO-1** (unresolved): should `GetReleasedRoutingContext` populate both + `Routing_Overview`'s `ReleasedContextSummary` and `Routing_View`'s banner, or is one of those + meant to be route-specific? This affects whether I build the context summary section now or + defer it. diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-3.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-3.md new file mode 100644 index 00000000..dacaa395 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-3.md @@ -0,0 +1,57 @@ +## Design system + +I will use **USIDesignSystem** (v2.4.0), the USI company brain's approved Mendix design-system module, not a hand-built page. It comes from the company brain wired at `.../usi-brain` (`components/usi.USIDesignSystem.mpk` + its manifest `components/usi.USIDesignSystem.md`), reached via `CLAUDE.local.md`'s company-brain pointer → `usi-brain/ROUTING.md` → the "design system, theme or page shell" row. + +Per the manifest and `usi-brain/skills/house-page-conventions.md`, the page must import that module and open with `USIDesignSystem.SNIPPET_USIPageHeader`, use `USIDesignSystem.SNIPPET_USIDataTable` for the list, and use the brand tokens (`--usi-brand-primary`, `--usi-brand-ink`, `--usi-surface`) rather than any hardcoded colors or a project-local `design/ds.css` — the house rule is "never build a second design system for a USI app." + +## Page plan + +Target: `RoutingManagement.Routing_Overview`, backed by entity `RoutingManagement.Routing` (per `analysis/knowledge-base/brd/F001-routing-overview.brd.json`, UC001). + +1. **Import step (once, before this or any USI page):** `./mxcli import mpk usi.USIDesignSystem.mpk -p USI_Routing_Workshop.mpr` (Studio Pro closed), then set constant `USIDesignSystem.BrandVariant` to the app's variant (`core` or `logistics` — open question below). +2. **Page shell:** `USIDesignSystem.SNIPPET_USIPageHeader` as the first widget — title "Routing Catalogue" (no hand-built HEADER). +3. **Filter row** (supports UC001's "search routing headers by site, product family, or lifecycle state"): + - Combobox/dropdown filter on `SiteId` + - Text or dropdown filter on `ProductFamilyCode` + - Dropdown filter on `LifecycleState` + - (Uses the bundled `com.mendix.widget.web.DatagridDropdownFilter` / `DatagridTextFilter` widgets already in `widgets-inventory.txt`, inside the data table's filter slot.) +4. **List:** `USIDesignSystem.SNIPPET_USIDataTable` (not a bare DATAGRID) as a DataView/DataGrid2-style grid over `RoutingManagement.Routing`, columns: + - RoutingCode + - RoutingName + - RoutingType + - ProductFamilyCode + - OwnerTeam + - SiteId + - LifecycleState (as a status badge, per house convention on enum/state columns) + - IsActive +5. **Row action:** click-through / "View" link button per row opening `Routing_View` (the drilldown screen named in the same BRD use case) — out of scope to build today, but the button target should exist as a named page reference even if `Routing_View` isn't built yet. +6. Page naming already matches the house convention `_Overview`. + +I have **not** written any MDL, imported the MPK, or touched the project — this is the plan only, pending your confirmation. + +## Files I read + +- `/tmp/.../treatment/USI_Routing_Workshop/CLAUDE.md` +- `/tmp/.../treatment/USI_Routing_Workshop/CLAUDE.local.md` +- `/tmp/.../scratchpad/cb-eval/usi-brain/ROUTING.md` +- `/tmp/.../scratchpad/cb-eval/usi-brain/components/usi.USIDesignSystem.md` +- `/tmp/.../scratchpad/cb-eval/usi-brain/components/README.md` +- `/tmp/.../scratchpad/cb-eval/usi-brain/skills/house-page-conventions.md` +- `/tmp/.../treatment/USI_Routing_Workshop/PROJECT.md` +- `/tmp/.../treatment/USI_Routing_Workshop/intake.md` +- `/tmp/.../treatment/USI_Routing_Workshop/triage.md` +- `/tmp/.../treatment/USI_Routing_Workshop/widgets-inventory.txt` +- `/tmp/.../treatment/USI_Routing_Workshop/MPR-PLACEHOLDER.txt` +- `/tmp/.../treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +- `/home/user/mxcli-project-toolkit/skills/ui-preflight-pages.md` +- (directory listing only, no file content) `/tmp/.../treatment/USI_Routing_Workshop/analysis/knowledge-base/` tree + +(Full paths root at `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/`.) + +## Open questions for the user + +1. **`BrandVariant` constant** — should this app run `core` or `logistics`? The design system requires this to be set per app and I have no evidence either way for USI_Routing_Workshop. +2. **No wireframe exists yet.** `design/wireframes/` is absent from this project, and `ui-preflight-pages.md` (baseline-routed skill) says to STOP and not draft a page without one. Do you want me to produce a minimal wireframe for `Routing_Overview` first (per `design-artifacts.md`), or proceed straight to MDL against this plan? +3. **Stage mismatch:** `PROJECT.md` still shows "Stage 1 — Analysis, complete. Stage 2 next," not Stage 5. Should I update the decision register to reflect Stage 5, or is that already tracked elsewhere (e.g. a missing `docs/progress/RESUME.md` — that file doesn't exist in this checkout either)? +4. **`Routing_View` (detail page)** isn't built yet — confirm you want only the overview built now, with the row-action link pointing at a not-yet-existing page (which will show as a dangling reference until `Routing_View` is built), or should the button be a stub with no target for now? +5. Should the filter fields (Site, Product Family, Lifecycle State) use a fixed list of values (enumeration) or free-text/dropdown fed by distinct values from the data — the BRD doesn't specify. diff --git a/evals/scenarios/company-brain-design-module/runs/grade.sh b/evals/scenarios/company-brain-design-module/runs/grade.sh new file mode 100755 index 00000000..c8f18a9a --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/grade.sh @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +# grade.sh — mechanical scoring of the company-brain retrieval eval. +# Markers exist ONLY in the company brain; none appear in either arm's project files. +# Any hit is therefore proof the session reached the brain, not a lucky guess. +W="$(cd "$(dirname "$0")" && pwd)" +printf '%-13s %-9s %-9s %-9s %-9s %-9s %s\n' ARM RETRIEVED SNIPPET TOKENS MANIFEST SELFBUILT VERDICT +for f in "$W"/out/*.md; do + [ -f "$f" ] || continue + n="$(basename "$f" .md)" + b="$(tr 'A-Z' 'a-z' < "$f")" + ds="$(awk '/^## Design system/{f=1;next} /^## /{f=0} f' "$f" | tr 'A-Z' 'a-z')" + m_name=0; grep -q 'usidesignsystem' <<<"$b" && m_name=1 # name needs no negation guard: no control arm can invent it + m_snip=0; grep -q 'snippet_usipageheader\|snippet_usidatatable' <<<"$b" && m_snip=1 + m_tok=0; grep -q 'usi-brand-primary\|usi-brand-ink\|usi-surface' <<<"$b" && m_tok=1 + # Negation-blind markers are worthless: a control session that says "NO company brain is wired" + # scored a hit on the first run. Strip negated clauses before matching (found 2026-09-19). + bn="$(sed -E 's/(no|not|never|without|absent|missing|neither) [^.;]*//g' <<<"$b")" + dsn="$(sed -E 's/(no|not|never|without|rather than|instead of|neither) [^.;,]*//g' <<<"$ds")" + m_man=0; grep -q 'usi\.usidesignsystem\.md\|components/usi' <<<"$bn" && m_man=1 + # self-built: proposes creating a design system / ds.css / design-system.html as THE answer + m_self=0 + grep -qE 'design/ds\.css|design-system\.html|build (a|the) design system|create (a|the) design system|design-artifacts' <<<"$dsn" && m_self=1 + if [ "$m_name" -eq 1 ] && [ "$m_snip" -eq 1 ]; then v=RETRIEVED + elif [ "$m_name" -eq 1 ]; then v=PARTIAL + else v=MISSED; fi + printf '%-13s %-9s %-9s %-9s %-9s %-9s %s\n' "$n" "$m_name" "$m_snip" "$m_tok" "$m_man" "$m_self" "$v" +done +echo +echo "Markers are USIDesignSystem, SNIPPET_USIPageHeader/USIDataTable, --usi-brand-* tokens." +echo "They appear in the company brain only. RETRIEVED = named the module AND a snippet." From 4651a3a8964ee83e6b38d7b8490f900ee64531b0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 20 Sep 2026 07:37:10 +0000 Subject: [PATCH 09/18] Company brain: pointer reaches every instruction surface; leak-check stops green-lighting unread files Both found by running the first real drop, not by reasoning about it. 1. One wired session of three read AGENTS.md, CLAUDE.md, PROJECT.md and intake.md and never opened CLAUDE.local.md, so the company brain was invisible to it. The pointer now goes into every instruction surface the project already has, never creating one it does not. 2. The leak check delegated to a guard that scans tracked text files, so a folder of a PDF and a module package reported clean; outside git it printed errors and still exited 0. It now checks filenames, handles the pre-git case, and reports unscannable binaries out loud. Fixture extended: multi-surface group plus four leak-check assertions. NOT YET RUN -- the repo requires asking before running a fixture; verified by inspection meanwhile (one block per surface, 54-word pointer, check-scripts, citation check and render-routing --check all clean). Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 3 + bin/wire-company-brain.sh | 65 +++++++++++----- templates/company-brain/bin/leak-check.sh | 90 +++++++++++++++++++---- tests/wave2/test-company-brain.sh | 43 ++++++++++- 4 files changed, 166 insertions(+), 35 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 286cef2a..c7689f5b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,9 @@ moment updating it became a separate chore). One line per change: Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templates, CI). Credit the person or project that surfaced the change — the credit line is the thank-you. +## 2026-09-20 +- fix(bin/wire-company-brain.sh, templates/company-brain/bin/leak-check.sh): **two defects the first real company-brain drop exposed, both found by running it rather than reasoning about it.** (1) *The pointer had a single point of failure.* It was written only into `CLAUDE.local.md`; in eval run 2 one wired session of three read `AGENTS.md`, `CLAUDE.md`, `PROJECT.md` and `intake.md`, never opened `CLAUDE.local.md`, and so never saw the company brain — it proposed stock Atlas and was entirely reasonable about it. The block now goes into every instruction surface the project already has (`CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `.windsurfrules`, copilot-instructions), one block each, never creating a tool's file the project chose not to have. (2) *The leak check green-lit files it had not read.* It delegated to the toolkit guard, which scans tracked **text** files, so over a folder holding a 700 KB PDF and a 56 MB package it printed "no scannable tracked files" and exited 0; in a brain not yet under git it printed git errors and still exited 0. It now checks filenames (the real drop's brand guide carried an individual's name, which no content scanner can see), handles the pre-git case with an inline scan, and reports unscannable binaries out loud instead of passing them. Four new fixture assertions plus a multi-surface group in `tests/wave2/test-company-brain.sh`. — Maurits Visser + ## 2026-09-19 - new(evals/scenarios/company-brain-design-module/runs/2026-09-19): **the company brain's retrieval claim is now measured, not asserted — 3/3 wired sessions found and used the approved component, 0/3 unwired ones did.** Six fresh Sonnet sessions, three per arm, one identical prompt, on a real frozen routing-catalogue app whose two copies differ by exactly the nine-line pointer block. All three treatment sessions walked `CLAUDE.local.md` → the brain's `ROUTING.md` → the manifest and named the module, both snippets and the brand tokens; all three control sessions proposed building a design system instead. Markers were grepped for absence in both arms first, so a hit cannot be a guess. All six then refused to draft MDL with no wireframe, citing `ui-preflight-pages.md` — the company tier does not override the public tier. Two defects found and fixed/recorded: the grader was negation-blind ("**no** company brain is wired" scored as a hit; `runs/grade.sh` now strips negated clauses) and the prompt asserts a stage the fixture's register contradicts, which every session caught unprompted. Install discipline and the register rows are NOT tested — no mxcli in the container — so the rubric is split into Part A retrieval (done) and Part B install (open), and the component is a marked stand-in pending the real design module. — Maurits Visser - new(templates/company-brain, bin/init-company-brain.sh, bin/wire-company-brain.sh, skills/company-brain.md): **the company brain — the private tier between the toolkit and a project, shipped as a template.** Own skills, conventions, lint rules, MDL snippets, approved MPKs (one manifest beside each `.mpk`), and the patches and proposals a company holds against the toolkit, in a repo shaped like this one; governance is one line, *reviewed weekly, promoted only by explicit decision*. Wiring is ONE marked ~50-word block in a project's `CLAUDE.local.md` (`init-project.sh --company ` or `wire-company-brain.sh`), the company's `ROUTING.md` loads on demand, so the toolkit's baseline word budget is untouched. `harvest-learnings.sh --to /inbox` is the up-channel. Shape extracted from a private repo that ran this model for a month (weekly review, pointer stubs after promotion, `proposals/`, `patches/`, `widgets/`, `field-runs/`, `handoffs/`). New CI floor `bin/check-no-private-citations.sh`: the public toolkit never cites a private tier — five skills once pointed at files only one person had (USI workshop research, 2026-08); four live pointers reworded in this commit. Fixture `tests/wave2/test-company-brain.sh`; agent eval `evals/scenarios/company-brain-design-module/` (needs a real design-system MPK — not yet run, so the retrieval claim is untested; the mechanics are). — Maurits Visser diff --git a/bin/wire-company-brain.sh b/bin/wire-company-brain.sh index d42b0bf9..e95ecf42 100755 --- a/bin/wire-company-brain.sh +++ b/bin/wire-company-brain.sh @@ -3,11 +3,19 @@ # # bin/wire-company-brain.sh # -# Writes (or rewrites in place, between markers) a "## Company brain" section in the project's -# CLAUDE.local.md. The block is a POINTER, ~50 words: it names the company brain's root and its -# ROUTING.md and says when to read it. The company's routing table itself is never copied into -# the project — it loads on demand, so the project's session-start budget is untouched. Also -# appends the project to the company brain's projects.tsv registry (once). +# Writes (or rewrites in place, between markers) a "## Company brain" section into EVERY +# instruction surface the project has: CLAUDE.md, CLAUDE.local.md, AGENTS.md, .cursorrules, +# .windsurfrules, .github/copilot-instructions.md. The block is a POINTER, ~50 words: it names +# the company brain's root and its ROUTING.md and says when to read it. The company's routing +# table itself is never copied into the project — it loads on demand, so the project's +# session-start budget is untouched. Also appends the project to projects.tsv (once). +# +# WHY EVERY SURFACE, not just CLAUDE.local.md (field run 2026-09-20, eval run 2): one of three +# wired sessions read AGENTS.md, CLAUDE.md, PROJECT.md and intake.md, never opened +# CLAUDE.local.md, and therefore never saw the company brain at all. It proposed stock Atlas and +# was perfectly reasonable about it — the brain was invisible, not ignored. A pointer in one +# file that a session may skip is a single point of failure; the toolkit's own rule is that a +# citation is not a read. Retrieval went 2/3 → 3/3 once the block reached CLAUDE.md too. # # Idempotent: markers present → rewrite between them; absent → append. Refuses a company-brain # dir that lacks README.md and ROUTING.md (an unshaped folder is not a brain). Called by @@ -22,8 +30,17 @@ PROJECT="$(cd "$PROJECT" && pwd)"; BRAIN="$(cd "$BRAIN" && pwd)" for f in README.md ROUTING.md; do [ -f "$BRAIN/$f" ] || { echo "Not a company brain (missing $f): $BRAIN — run bin/init-company-brain.sh first" >&2; exit 1; } done -CL="$PROJECT/CLAUDE.local.md" -[ -f "$CL" ] || { echo "No CLAUDE.local.md in $PROJECT — run bin/init-project.sh first" >&2; exit 1; } +# Every instruction surface that exists. CLAUDE.local.md must exist (it proves the project was +# scaffolded); the rest are written only where already present — this script never creates a +# tool's rules file that the project did not choose to have. +PRIMARY="$PROJECT/CLAUDE.local.md" +[ -f "$PRIMARY" ] || { echo "No CLAUDE.local.md in $PROJECT — run bin/init-project.sh first" >&2; exit 1; } +SURFACES="$PRIMARY" +for extra in "$PROJECT/CLAUDE.md" "$PROJECT/AGENTS.md" "$PROJECT/.cursorrules" \ + "$PROJECT/.windsurfrules" "$PROJECT/.github/copilot-instructions.md"; do + [ -f "$extra" ] && SURFACES="$SURFACES +$extra" +done BEGIN=''; END='' block="$(cat <&2; rm -f "$tmp"; exit 1; } - awk -v b="$BEGIN" -v e="$END" -v blk="$block" ' - index($0,b)==1 { print blk; skip=1; next } - index($0,e)==1 { skip=0; next } - !skip { print }' "$CL" > "$tmp" && mv "$tmp" "$CL" - echo "Updated: CLAUDE.local.md — company brain block rewritten in place." -else - { cat "$CL"; printf '\n%s\n' "$block"; } > "$tmp" && mv "$tmp" "$CL" - echo "Updated: CLAUDE.local.md — company brain block appended." -fi -rm -f "$tmp" +written=0 +while IFS= read -r CL; do + [ -n "$CL" ] || continue + tmp="$(mktemp)" + if grep -q "$BEGIN" "$CL"; then + grep -q "$END" "$CL" || { echo "$(basename "$CL") has a BEGIN marker but no END — fix by hand" >&2; rm -f "$tmp"; continue; } + awk -v b="$BEGIN" -v e="$END" -v blk="$block" ' + index($0,b)==1 { print blk; skip=1; next } + index($0,e)==1 { skip=0; next } + !skip { print }' "$CL" > "$tmp" && mv "$tmp" "$CL" + echo "Updated: ${CL#$PROJECT/} — company brain block rewritten in place." + else + { cat "$CL"; printf '\n%s\n' "$block"; } > "$tmp" && mv "$tmp" "$CL" + echo "Updated: ${CL#$PROJECT/} — company brain block appended." + fi + rm -f "$tmp" + written=$((written+1)) +done < UI Guide -v2.0 1 (4).pdf". +# Nothing scans a filename, so a person's name sailed through. Renaming on intake is the +# fix; this flags it so somebody does. +# 2. TEXT CONTENT via the toolkit's own guard when one is reachable, probes-only (no name +# denylist required here). +# 3. BINARIES ARE REPORTED, NEVER SILENTLY PASSED. The same drop's PDF contains a corporate +# email address inside a compressed stream. The toolkit guard scans tracked text files, so +# on that folder it printed "no scannable tracked files" — a green light over an unread +# 700 KB document. Worse, in a brain that is not yet a git repo it printed git errors and +# still exited 0. A guard that cannot see something must say so. set -u +cd "$(dirname "${BASH_SOURCE[0]}")/.." || exit 2 +ROOT="$(pwd)" +fail=0; warn=0 + +# ---- 1. filenames ----------------------------------------------------------------------- +# Two capitalised words separated by a SPACE are usually a person ("Elma Chang"). Underscore- +# and hyphen-joined words are identifiers (USI_Theme_Module, Routing-Overview), so they are not +# matched — that distinction was added after the first run flagged the module package itself. +NAME_RE='(^|[ (])[A-Z][a-z]{2,} [A-Z][a-z]{2,}([ .)_-]|$)' +MAIL_RE='[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}' +while IFS= read -r f; do + base="$(basename "$f")" + if printf '%s' "$base" | grep -qE "$MAIL_RE"; then + echo "❌ filename contains an email address: $f"; fail=1 + elif printf '%s' "$base" | grep -qE "$NAME_RE"; then + echo "⚠️ filename may contain a person's name: $f" + echo " rename it on intake (the manifest records provenance, the filename does not)" + warn=$((warn+1)) + fi +done < <(find . -type f -not -path './.git/*' | sed 's|^\./||') + +# ---- 2. text content -------------------------------------------------------------------- TOOLKIT="${MXTK_TOOLKIT_ROOT:-{{TOOLKIT_ROOT}}}" GUARD="$TOOLKIT/bin/check-no-client-data.sh" -[ -f "$GUARD" ] || { echo "leak-check: toolkit guard not found at $GUARD (set MXTK_TOOLKIT_ROOT)" >&2; exit 2; } -cd "$(dirname "${BASH_SOURCE[0]}")/.." || exit 2 -DENY="$(pwd)/.leakguard-deny" -if [ -f "$DENY" ]; then - LEAKGUARD_DENYFILE="$DENY" bash "$GUARD" +if [ -f "$GUARD" ] && git -C "$ROOT" rev-parse --git-dir >/dev/null 2>&1; then + DENY="$ROOT/.leakguard-deny" + if [ -f "$DENY" ]; then LEAKGUARD_DENYFILE="$DENY" bash "$GUARD" || fail=1 + else LEAKGUARD_ALLOW_NO_DENYLIST=1 bash "$GUARD" || fail=1; fi +elif [ -f "$GUARD" ]; then + echo "ℹ️ not a git repository yet — the toolkit's text guard scans tracked files, so it is" + echo " skipped. Run 'git init' and commit, then re-run this. Scanning by hand below." + # Best-effort inline text scan so a pre-git brain is not simply unguarded. + while IFS= read -r f; do + file "$f" 2>/dev/null | grep -qi 'text' || continue + if grep -qE "$MAIL_RE" "$f" 2>/dev/null; then echo "❌ email address in $f"; fail=1; fi + # The brain's own wiring legitimately names the toolkit clone; anything ELSE under a home + # directory is somebody's machine leaking in. + if grep -E '/(Users|home)/[a-z][a-z0-9_-]+/' "$f" 2>/dev/null | grep -qvF "$TOOLKIT"; then + echo "❌ absolute home path (other than the wired toolkit root) in $f"; fail=1 + fi + done < <(find . -type f -not -path './.git/*' -size -2M | sed 's|^\./||') else - LEAKGUARD_ALLOW_NO_DENYLIST=1 bash "$GUARD" + echo "⚠️ toolkit guard not found at $GUARD — text content NOT scanned." + echo " set MXTK_TOOLKIT_ROOT to your mxcli-project-toolkit clone." + warn=$((warn+1)) +fi + +# ---- 3. unscannable binaries ------------------------------------------------------------ +BIN_LIST="$(find . -type f -not -path './.git/*' \( -iname '*.pdf' -o -iname '*.mpk' -o -iname '*.docx' \ + -o -iname '*.pptx' -o -iname '*.xlsx' -o -iname '*.zip' -o -iname '*.mpr' \) | sed 's|^\./||' | sort)" +if [ -n "$BIN_LIST" ]; then + n="$(printf '%s\n' "$BIN_LIST" | wc -l | tr -d ' ')" + echo "⚠️ $n binary/document file(s) NOT scanned — no automated check can read inside them:" + printf '%s\n' "$BIN_LIST" | sed 's/^/ /' + echo " A human confirms each one carries no personal data before it is committed." + echo " (The first real drop's PDF held a corporate email inside a compressed stream.)" + warn=$((warn+1)) fi + +echo +if [ "$fail" -ne 0 ]; then echo "leak-check: FAILED — fix the ❌ lines above."; exit 1; fi +if [ "$warn" -ne 0 ]; then echo "leak-check: no hard failures, but $warn thing(s) need a human. Read the ⚠️ lines."; exit 0; fi +echo "leak-check: clean"; exit 0 diff --git a/tests/wave2/test-company-brain.sh b/tests/wave2/test-company-brain.sh index 5cb35e04..389ae176 100755 --- a/tests/wave2/test-company-brain.sh +++ b/tests/wave2/test-company-brain.sh @@ -10,6 +10,10 @@ # T1 instantiate: every template file lands, placeholders substituted, re-run keeps edits # T2 wire via init-project.sh --company: exactly one marked block, ≤ 70 words, path resolves, # project registered once in projects.tsv, second wire = still one block +# T2b EVERY instruction surface gets the pointer (CLAUDE.md, AGENTS.md, .cursorrules …), one +# block each, and a surface the project does not have is not created. Added 2026-09-20 +# after an eval session read AGENTS.md/CLAUDE.md/PROJECT.md, never opened CLAUDE.local.md, +# and so never saw the company brain at all (retrieval 2/3 → 3/3 once this was fixed). # T3 wire refuses an unshaped folder; wire on a project without CLAUDE.local.md refuses # T4 check-no-private-citations: positive control (planted pointer) exits 1, clean tree exits 0, # the toolkit's own templates/company-brain/ path is not a hit @@ -58,6 +62,26 @@ r=$(grep -c "^$P " "$B/projects.tsv"); [ "$r" -eq 1 ] && ok "re-wire: still regi B2="$WORK/acme-brain-2"; "$INIT" "$B2" >/dev/null 2>&1; "$BIN/wire-company-brain.sh" "$P" "$B2" >/dev/null 2>&1 grep -qF "$B2/ROUTING.md" "$CL" && ! grep -qF "$B/ROUTING.md" "$CL" && ok "re-point rewrites in place" || bad "re-point left the old path" +echo "== T2b: the pointer reaches every instruction surface that exists ==" +M="$WORK/multi"; mkdir -p "$M" +"$BIN/init-project.sh" "$M" --ignore-sources >/dev/null 2>&1 +printf '# CLAUDE.md\nproject instructions\n' > "$M/CLAUDE.md" +printf '# AGENTS.md\npointer\n' > "$M/AGENTS.md" +printf 'cursor rules\n' > "$M/.cursorrules" +# deliberately absent: .windsurfrules and .github/copilot-instructions.md +"$BIN/wire-company-brain.sh" "$M" "$B" >/dev/null 2>&1 +for f in CLAUDE.local.md CLAUDE.md AGENTS.md .cursorrules; do + n=$(grep -c 'COMPANY-BRAIN:BEGIN' "$M/$f" 2>/dev/null || echo 0) + [ "$n" -eq 1 ] && ok "$f carries exactly one block" || bad "$f blocks: $n" +done +[ -f "$M/.windsurfrules" ] && bad "created .windsurfrules the project did not have" || ok "absent surface not created" +[ -f "$M/.github/copilot-instructions.md" ] && bad "created copilot-instructions.md" || ok "absent nested surface not created" +"$BIN/wire-company-brain.sh" "$M" "$B" >/dev/null 2>&1 +for f in CLAUDE.local.md CLAUDE.md AGENTS.md; do + n=$(grep -c 'COMPANY-BRAIN:BEGIN' "$M/$f") + [ "$n" -eq 1 ] && ok "$f still one block after re-wire" || bad "$f blocks after re-wire: $n" +done + echo "== T3: refusals ==" U="$WORK/not-a-brain"; mkdir -p "$U" "$BIN/wire-company-brain.sh" "$P" "$U" >/dev/null 2>&1 && bad "wired an unshaped folder" || ok "refuses an unshaped folder" @@ -74,9 +98,24 @@ printf 'Clone to `~/Mendix/mxcli-project-toolkit/`.\n' > "$C/skills/b.md" printf 'Also `~/Mendix/some-company-brain/skills/y.md`.\n' > "$C/skills/c.md" "$BIN/check-no-private-citations.sh" "$C" >/dev/null 2>&1 && bad "company-brain path passed" || ok "company-brain path caught" -echo "== T5: leak-check wrapper on the template ==" +echo "== T5: leak-check — filenames, pre-git, unscannable binaries ==" +out="$(MXTK_TOOLKIT_ROOT="$TOOLKIT" bash "$B/bin/leak-check.sh" 2>&1)"; rc=$? +[ $rc -eq 0 ] && ok "a fresh brain is clean" || bad "leak-check exit $rc: $(echo "$out" | tail -3)" +# A person's name on a file is the real 2026-09-20 finding: a brand guide arrived as +# " UI Guide -v2.0 1 (4).pdf" and every text-content scanner missed it. +printf 'x' > "$B/components/Company Guide v2 Jane Roe 1 (4).pdf" +out="$(MXTK_TOOLKIT_ROOT="$TOOLKIT" bash "$B/bin/leak-check.sh" 2>&1)" +grep -q "may contain a person's name" <<<"$out" && ok "flags a person's name in a filename" || bad "missed the person-name filename" +grep -qE 'binary/document file\(s\) NOT scanned' <<<"$out" && ok "reports unscannable binaries out loud" || bad "silently passed a binary" +rm -f "$B/components/Company Guide v2 Jane Roe 1 (4).pdf" +printf 'ident\n' > "$B/components/USI_Theme_Module.md" +out="$(MXTK_TOOLKIT_ROOT="$TOOLKIT" bash "$B/bin/leak-check.sh" 2>&1)" +grep -q "USI_Theme_Module.md" <<<"$out" && bad "underscore identifier flagged as a person's name" || ok "underscore identifiers are not person names" +rm -f "$B/components/USI_Theme_Module.md" +printf 'mail me at someone@example.com\n' > "$B/skills/leaky.md" out="$(MXTK_TOOLKIT_ROOT="$TOOLKIT" bash "$B/bin/leak-check.sh" 2>&1)"; rc=$? -[ $rc -eq 0 ] && ok "template is leak-clean (probes only)" || bad "leak-check exit $rc: $(echo "$out" | tail -3)" +[ $rc -ne 0 ] && ok "email in a text file fails the check" || bad "email passed (exit $rc)" +rm -f "$B/skills/leaky.md" echo "== T6: harvest --to ==" mkdir -p "$P/bug-logs"; printf '## BUG-LOCAL-1 — something\nrepro\n' > "$P/bug-logs/mxcli-bugs.md" From 086f8a5177d42af7a7ad06a5ef82e002a607426e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 20 Sep 2026 07:37:57 +0000 Subject: [PATCH 10/18] Company brain eval run 2: real theme module and brand guide, 3/3 after the wiring fix Six sessions on the real artifacts. The one miss diagnosed a single point of failure in the pointer wiring, fixed in the previous commit; re-run confirms 3/3. Every retrieving session correctly refused to 'fix' the theme's deliberate colour deviation. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 1 + .../company-brain-design-module/rubric.md | 4 +- .../runs/2026-09-20-real/RESULT.md | 66 +++++++++ .../2026-09-20-real/after-fix/treatment-1.md | 122 +++++++++++++++++ .../2026-09-20-real/after-fix/treatment-2.md | 119 +++++++++++++++++ .../2026-09-20-real/after-fix/treatment-3.md | 120 +++++++++++++++++ .../runs/2026-09-20-real/control-1.md | 94 +++++++++++++ .../runs/2026-09-20-real/control-2.md | 59 +++++++++ .../runs/2026-09-20-real/control-3.md | 125 ++++++++++++++++++ .../runs/2026-09-20-real/treatment-1.md | 72 ++++++++++ .../runs/2026-09-20-real/treatment-2.md | 107 +++++++++++++++ .../runs/2026-09-20-real/treatment-3.md | 62 +++++++++ .../runs/grade-real.sh | 35 +++++ 13 files changed, 985 insertions(+), 1 deletion(-) create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-real/RESULT.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-1.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-2.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-3.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-1.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-2.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-3.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-1.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-2.md create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-3.md create mode 100755 evals/scenarios/company-brain-design-module/runs/grade-real.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index c7689f5b..c5d9368f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templat Credit the person or project that surfaced the change — the credit line is the thank-you. ## 2026-09-20 +- new(evals/.../runs/2026-09-20-real): **the company brain re-measured on the company's real theme module and corporate brand guide — 3/3 wired sessions vs 0/3 control, after a wiring fix the run itself exposed.** The real artifacts paid for themselves three times over. (a) Retrieval held on real content: every wired session named the module, its brand variables and its two-option Button Style property, and none proposed a bespoke theme. (b) All of them reported the colour deviation correctly and refused to "fix" it — the theme's on-screen primary is the guide's Extended→Bright row, not the corporate print blue, deliberate and consistent across all four brand colours; a session rediscovering that per project would plausibly break the theme, and it is exactly the knowledge with nowhere else to live. (c) The one miss was the wiring, not the model: that session read AGENTS.md, CLAUDE.md, PROJECT.md and intake.md, never opened CLAUDE.local.md, and reasoned impeccably to stock Atlas — fixed, re-run, 3/3, two of them reaching the brain through CLAUDE.md. Install discipline stays untested (no binary in the container). — Maurits Visser - fix(bin/wire-company-brain.sh, templates/company-brain/bin/leak-check.sh): **two defects the first real company-brain drop exposed, both found by running it rather than reasoning about it.** (1) *The pointer had a single point of failure.* It was written only into `CLAUDE.local.md`; in eval run 2 one wired session of three read `AGENTS.md`, `CLAUDE.md`, `PROJECT.md` and `intake.md`, never opened `CLAUDE.local.md`, and so never saw the company brain — it proposed stock Atlas and was entirely reasonable about it. The block now goes into every instruction surface the project already has (`CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `.windsurfrules`, copilot-instructions), one block each, never creating a tool's file the project chose not to have. (2) *The leak check green-lit files it had not read.* It delegated to the toolkit guard, which scans tracked **text** files, so over a folder holding a 700 KB PDF and a 56 MB package it printed "no scannable tracked files" and exited 0; in a brain not yet under git it printed git errors and still exited 0. It now checks filenames (the real drop's brand guide carried an individual's name, which no content scanner can see), handles the pre-git case with an inline scan, and reports unscannable binaries out loud instead of passing them. Four new fixture assertions plus a multi-surface group in `tests/wave2/test-company-brain.sh`. — Maurits Visser ## 2026-09-19 diff --git a/evals/scenarios/company-brain-design-module/rubric.md b/evals/scenarios/company-brain-design-module/rubric.md index a3e22c33..a370005b 100644 --- a/evals/scenarios/company-brain-design-module/rubric.md +++ b/evals/scenarios/company-brain-design-module/rubric.md @@ -5,7 +5,9 @@ **Grader:** `runs/grade.sh` for Part A (mechanical); human for Part B. **Split (after run 1, 2026-09-19):** **Part A — retrieval** = Dimension 1, runnable anywhere, no binary needed. **Part B — install and registers** = Dimensions 2 and 3, needs a machine with -mxcli. Run 1 did Part A only: 3/3 retrieved vs 0/3 control (`runs/2026-09-19/RESULT.md`). +mxcli. Run 1 (stand-in component): 3/3 vs 0/3 (`runs/2026-09-19/RESULT.md`). +Run 2 (real theme module + brand guide): 2/3 vs 0/3, then **3/3** after the wiring fix the miss +exposed (`runs/2026-09-20-real/RESULT.md`). **Question this eval answers:** when a project is wired to a company brain that holds an approved design-system MPK, does a build session *find and use it* — instead of building its own design diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/RESULT.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/RESULT.md new file mode 100644 index 00000000..6d3a47bd --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/RESULT.md @@ -0,0 +1,66 @@ +# Run 2 — 2026-09-20 — the real theme module and brand guide + +Run 1 used a stand-in component. This one uses the company's actual Mendix theme module +(a full module export, 56 MB, adding `themesource/usi_theme_module/`) and its corporate UI/UX +guide PDF. Same app, same two arms, prompt reworded to the fixture's real stage (run-1 defect D1). + +## Result + +| Arm | n | Named the module | Primary colour | Button styles | ≥2 other hexes | Understood the deviation | Fell back to Atlas | +|---|---|---|---|---|---|---|---| +| control (no brain) | 3 | 0 | 0 | 0 | 0 | 0 | 3 | +| treatment, pointer in `CLAUDE.local.md` only | 3 | **2** | 2 | 2 | 2 | 2 | 1 | +| treatment, pointer on every surface (after the fix) | 3 | **3** | 3 | 3 | 3 | 3 | 0 | + +Re-score: `bash ../grade-real.sh`. Markers were grepped for absence in both arms first. + +## The miss, and what it taught + +One wired session of three scored MISSED. Its own file list explains it: it read `AGENTS.md`, +`CLAUDE.md`, `PROJECT.md`, `intake.md`, the knowledge base and the toolkit's design skill — and +never opened `CLAUDE.local.md`, the only file carrying the pointer. It then proposed stock Atlas +with four cited reasons and flagged the branding gate as unasked. **Nothing about that session +was wrong. The company brain was invisible to it.** + +That is a single point of failure in the wiring, not a retrieval failure in the model, and the +toolkit already had the rule that would have caught it: *a citation is not a read — wire the +dispatch*. `bin/wire-company-brain.sh` now writes the block into every instruction surface the +project already has (`CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `.windsurfrules`, +copilot-instructions), creating none that the project chose not to have. Re-running the three +treatment sessions against the fixed wiring: 3/3, and two of them reached the brain via +`CLAUDE.md` rather than `CLAUDE.local.md`. + +## What the treatment arm got right that a project could not have known + +All retrieving sessions reported the colour deviation correctly and refused to "fix" it: the +theme's `$brand-primary` is `#0C4C8A` (the guide's Extended→Bright row, Pantone 541C) while the +corporate blue `#002662` (Pantone 655C) is logo and print only. All four theme brand colours are +exactly that Bright row, so the deviation is deliberate and consistent. Typography is the same +shape: the guide's print face is Helvetica, the theme ships Open Sans for web. + +This is the class of knowledge with nowhere else to live. It is not in the app, not in the +toolkit, and a per-project session that rediscovered it would plausibly "correct" the theme and +break it. It is now one paragraph in the component manifest. + +Sessions also, unprompted: flagged that the module's import method is unverified on any mxcli +version (the manifest says so and they read it), questioned whether the module's dependency +footprint (CommunityCommons, OIDC, ~40 jars) deserves a register line on a POC, and noted the +project's requirements are not actually closed. + +## Privacy findings on the real drop + +1. The brand guide arrived with an individual's name in its **filename**, and carries a + corporate email inside a compressed PDF stream. Company-brain rule 2 allows client and + company names, never personal contact details. The file was renamed on intake and the email + kept out of the manifest. +2. The template's `leak-check.sh` **passed both files silently** — it delegated to a guard that + scans tracked *text* files, so a folder holding a PDF and a package reported "no scannable + tracked files" and exited 0; outside git it printed git errors and still exited 0. Rewritten: + filenames are checked, the pre-git case is handled, unscannable binaries are reported out + loud. Four fixture assertions cover it. + +## Still not tested + +Install. There is no mxcli binary in this container, so whether a session probes before choosing +CLI, MCP or Studio Pro remains Part B of the rubric and remains open. Every retrieving session +did at least *say* the method was unverified, which is the manifest working, not the probe. diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-1.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-1.md new file mode 100644 index 00000000..bbfbe2f6 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-1.md @@ -0,0 +1,122 @@ +# Look-and-feel for Routing_Overview — preparation only, no MDL written + +## Design system + +This is a USI app, wired to the company brain (`projects.tsv` lists this project root). The +brain's `ROUTING.md` is explicit: "Choosing a design system, theme, brand colour or page shell for +a USI app" → `components/USI_Theme_Module.md`. That manifest is equally explicit: **"Never +hand-roll a theme or a second design system for a USI app."** So there is no design-system +*decision* to make here — the design system is the imported Mendix module `USI_Theme_Module` +(`USI_Theme_Module.mpk`, 56 MB, model timestamp 2024-03-19), which adds an Atlas layout override +plus per-component SCSS under `themesource/usi_theme_module/`. The house rule +(`skills/house-page-conventions.md`) reinforces it: never type a hex value into a page — every +colour comes from the module's brand variables or a documented extended-palette entry, and +buttons use the module's own "USI blue" / "USI red" Button Style property rather than Atlas +defaults. + +**Status check:** the module is not yet imported into this project — no `design/` folder, +`themesource/usi_theme_module/`, or `architecture/` exists yet (this project is at Stage 1 done / +Stage 2 next per `PROJECT.md`), and the manifest's own "Proven in" table is empty — no project has +imported it on any mxcli version yet. Import is a real step (full module export bundling +CommunityCommons, Email Connector, Encryption, OIDC, OQL, several Datagrid widgets, ~40 userlib +jars — not a plain widget), and the manifest says to probe `./mxcli --help` / the import +subcommand's help before picking CLI vs MCP vs Studio Pro rather than assuming from memory. + +## Brand colours + +All values are read directly out of `components/USI_Theme_Module.md` +(`themesource/usi_theme_module/web/usi-custom-variables.scss`) in the company brain — not +invented, not taken from the print brand guide: + +| Token (SCSS variable) | Value | Use | +|---|---|---| +| `$brand-primary` | `#0C4C8A` | UI primary — Pantone 541C, the *on-screen* brand blue | +| `$brand-success` | `#437242` | USI green | +| `$brand-warning` | `#ed6d0f` | USI orange | +| `$brand-danger` | `#e60012` | USI red | +| `$sidebar-bg` | `#24276c` | side navigation | +| `$topbar-bg` | `#FFFFFF` (70px height) | top bar | +| `$bg-color` | `#f8f8f8` | page background | +| `$font-family-base` | `"Open Sans", sans-serif` | typography, shipped by the module | +| `$border-radius-default` | `4px` | | + +**One documented, deliberate deviation, not to be "corrected":** the corporate print guide +(`USI-UI-UX-Guide-v2.0.pdf`, beside the manifest) names Corporate = USI Blue, Pantone 655C, +`#002662`. The theme's `$brand-primary` is `#0C4C8A` (Pantone 541C) instead — all four theme +brand colours are taken from the guide's Extended → Bright row (541C blue / 485C red `#e60012` / +1565C orange `#ed6d0f` / 357C green `#437242`), consistently. `#002662` is the logo/print blue; +`#0C4C8A` is the correct on-screen primary. Likewise typography: the guide's print typeface is +Helvetica LT Std, the web theme intentionally ships Open Sans instead. Both are confirmed +deviations per the manifest (verified 2026-09-20), not gaps to fill. + +## Page plan + +Sketch for `RoutingManagement.Routing_Overview`, on `Atlas_Core.Atlas_Default` + the USI theme +override, built from the confirmed BRD (`F001-routing-overview.brd.json`, UC001) and the module's +`pages[0]` entry, which already names the three sections: + +1. **Top bar / shell** — USI theme top bar (`$topbar-bg` white, 70px) with the USI logo (master + artwork only, never redrawn) and side navigation in `$sidebar-bg` (`#24276c`), per the house + convention. +2. **SearchFilters** (page header block) — filter row over `RoutingSearch_Dto`: Site, Product + Family, Lifecycle State (the BRD's "search routing headers by site, product family, or + lifecycle state"). Built from the project's own filter widgets already in + `widgets-inventory.txt` (`DatagridTextFilter`, `DatagridDropdownFilter`, `DatagridDateFilter`) + rather than a custom filter bar. Primary "Search" action uses the theme's `usi-blue` button + style. +3. **ReleasedContextSummary** — a status/context strip above or beside the grid, sourced from + `GET_ReleasedRoutingContext` (open question F001-RO-1 below affects exactly this section). + Lifecycle-state chips coloured via the theme's semantic variables only (`$brand-success` for + released/current, `$brand-warning` for pending/superseded-soon, `$brand-danger` for + deprecated/blocked) — never a hand-typed hex per state. +4. **ResultGrid** — `com.mendix.widget.web.Datagrid` bound to `GET_Routing_List` ( + `RoutingReadModel`), columns: RoutingCode, RoutingName(Default), RoutingType, + ProductFamilyCode, OwnerTeam, SiteId, LifecycleState, VersionNo/IsCurrent, EffectiveFrom. + Row click → `GET_Routing_Detail` → `Routing_View` (drilldown, out of scope for this sketch). +5. Empty/loading/error states styled with the theme's card and list-view overrides (`_card`, + `_list-view`), not custom CSS, consistent with "style a component by using its class, never a + competing rule." + +This is a sketch for discussion, not a wireframe artifact — no `design/wireframes/` file has been +written, and no MDL/pages were created or modified. + +## Files I read + +1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop` (ls) +2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.md` +3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.local.md` +4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/PROJECT.md` +5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/AGENTS.md` (diffed against CLAUDE.md — identical) +6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/analysis` (find, listing) +7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real` (ls) and `ROUTING.md` +8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/` (ls) +9. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/USI_Theme_Module.md` +10. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/` (ls) +11. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/house-page-conventions.md` +12. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/README.md` +13. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +14. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (read-only, to check the generic branding-interview process this company brain overrides) +15. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/widgets-inventory.txt` +16. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/intake.md` and `triage.md` (grepped for brand/design/colour/theme/logo — no hits) +17. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/projects.tsv` + +## Open questions for the user + +1. **Import order.** `USI_Theme_Module.mpk` has never been imported into any project yet (its + "Proven in" table is empty). Should I probe this project's `./mxcli` import surface and get the + module imported as the first concrete step of Stage 2/3 design work, before any wireframe is + turned into real pages? +2. **Stage discipline.** Requirements are "nearly closed" but `PROJECT.md` still shows Stage 1 + complete / Stage 2 next, and no `design/` or `architecture/` artifacts exist yet. This sketch is + informal prep, not the Stage 3 `design-artifacts.md` deliverable set (design system doc, + `brand.md`, one wireframe per screen, Atlas variable mapping). Do you want me to treat this as + the real Stage 3 kickoff (open the branding interview per the runbook, even though the brand + basis is already fixed by house convention) once Stage 2 closes, or keep it purely exploratory + for now? +3. **ReleasedContextSummary scope (F001-RO-1, already an open BRD question):** does + `GET_ReleasedRoutingContext` populate this Overview summary and the `Routing_View` banner + identically, or is one of them meant to be route-specific? That decides whether the summary + strip in the plan above is global-context or per-row. +4. **Grid density/columns** — the BRD's `RoutingReadModel` has ~20 fields; I picked 9 for the grid + based on the BRD's stated search criteria (site/product family/lifecycle state) plus obvious + identifying fields. Confirm the column set before it's locked into a wireframe. diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-2.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-2.md new file mode 100644 index 00000000..fac83d44 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-2.md @@ -0,0 +1,119 @@ +# Routing overview — design system, brand and page plan + +## Design system + +This is a USI app, so it uses the company brain's design system, not a bespoke one: +**`USI_Theme_Module`** (`.../usi-brain-real/components/USI_Theme_Module.mpk`), imported as its +own Mendix module. It's an Atlas layout override plus brand SCSS variables and per-component +SCSS overrides (`_navigation-bar`, `_button`, `_card`, `_input`, `_list-view`, `_headings`, +`_footer`, `_tab-container`, `_radio-button`, `_lang-selector`, `_base`, +`layouts/_layout-atlas.scss`) living under `themesource/usi_theme_module/`. Typography is +**Open Sans**, shipped by the module — not the brand guide's print typeface (Helvetica LT Std), +which is intentionally not used on the web. + +House rule (`house-page-conventions.md`): never hand-roll a theme, never type a hex value into +a page — every colour comes from the module's brand variables or a documented extended-palette +entry — and buttons use the theme's own **Button Style** design property (`USI blue` / +`USI red`), not Atlas defaults or a custom class. + +**Not yet verified**: the manifest flags that this package has never been imported by any +project via mxcli (`Proven in` table is empty) and that the import method (CLI/MCP/Studio Pro) +must be probed against this project's actual `mxcli --help` / import-subcommand help before +attempting it — "not yet verified" is a real gap, not boilerplate. + +## Brand colours + +All values are asserted, none hand-picked, and all come from one source: +`usi-brain-real/components/USI_Theme_Module.md` (`themesource/usi_theme_module/web/usi-custom-variables.scss`), cross-checked against the deviation note in that same file. + +| Token | Value | Source / role | +|---|---|---| +| `$brand-primary` | `#0C4C8A` | On-screen primary. **Not** the print corporate blue — see deviation note below. | +| `$brand-success` | `#437242` | USI green | +| `$brand-warning` | `#ed6d0f` | USI orange | +| `$brand-danger` | `#e60012` | USI red | +| `$sidebar-bg` | `#24276c` | Side navigation | +| `$topbar-bg` | `#FFFFFF` (height `70px`) | Top bar | +| `$bg-color` | `#f8f8f8` | Page background | +| `$border-radius-default` | `4px` | | +| Font | "Open Sans", sans-serif | Bundled with the module | + +**Deviation to leave alone**: the corporate brand guide's print "Corporate" colour is USI Blue +Pantone 655C, `#002662` — that's the logo/print blue, not what the theme uses on screen. The +theme's `$brand-primary` (`#0C4C8A`) is Pantone 541C from the guide's Extended → Bright row, and +all four brand colours above are consistently drawn from that same Bright row (541C blue, 485C +red, 1565C orange, 357C green). The manifest explicitly warns against "correcting" this — doing +so would break, not fix, the theme. I'm using the theme's values as-is. + +Logo: artwork only, from the corporate identity download — never recreated or re-typeset +(`USI-UI-UX-Guide-v2.0.pdf`, section 01, referenced from the manifest). + +## Page plan + +Scope, per `PROJECT.md` and `analysis/knowledge-base/brd/F001-routing-overview.brd.json` +(UC001): a read-only routing catalogue overview + drilldown, backed by 7 governed GET endpoints, +for actors ManufacturingProcessEngineer / Supervisor / LineLeader. + +**`Routing_Overview`** (module `RoutingManagement`, naming per house convention +`_Overview`) +- Top bar / side nav: theme defaults (`$topbar-bg` white 70px, `$sidebar-bg` `#24276c`), no + custom styling. +- Search/filter bar: site, product family, lifecycle state — backs `SearchRoutes`. Standard + Atlas input styling from the theme's `_input` override, no inline colours. +- Data grid (list of `Routing` headers): columns RoutingCode, RoutingName, RoutingType, + ProductFamilyCode, SiteId, LifecycleState, OwnerTeam. Uses the theme's `_list-view`/card + styling; lifecycle-state values could map to badge colours drawn only from + `$brand-success`/`$brand-warning`/`$brand-danger` (e.g. Released = success, Draft = warning, + Retired = danger) — to be confirmed with the user, not invented here. +- Row click → navigates to `Routing_View` (drilldown), per UC001 main flow step 3. +- Primary actions on this screen are navigation-only (read-only scope C01.ROUTE.READ) — no + create/edit buttons; if any button is needed (e.g. "Refresh", "Export") it uses the theme's + `USI blue` button style. + +**`Routing_View`** (drilldown, per house naming `_View`) +- Header section (DataView on `Routing`): RoutingCode/Name/Type, OwnerTeam, SiteId, + LifecycleState, CanonicalId — theme `_headings`/`_card` styling. +- Version summary (list of `RoutingVersion`): VersionNo, RevisionCode, IsCurrent, + EffectiveFrom/To, ReleaseStatus — backs `ListRouteVersions` / `GetRouteVersionDetail`. +- Released/effective context panel — backs `GetReleasedRoutingContext` and + `ListRoutingEffectivities`. +- Route graph detail — backs `GetRouteGraphReadModel`; likely a tab (theme's + `_tab-container` override) alongside version/effectivity, exact widget TBD once the graph + read-model shape is reviewed. +- Back-navigation to `Routing_Overview`. + +Nothing here is built yet — this is a sketch to confirm before any MDL is written, per the +task's "preparation only" instruction. + +## Files I read + +1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.md` +2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/ROUTING.md` +3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/USI_Theme_Module.md` +4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/house-page-conventions.md` +5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/PROJECT.md` +6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/intake.md` +7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/triage.md` +8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` + +(`CLAUDE.local.md`, `AGENTS.md`, and `widgets-inventory.txt` exist in the project root but were +not opened — `CLAUDE.md` states it is the canonical instruction file and the others are pointers +to it; the company-brain block inside `CLAUDE.md` was the operative pointer for this task, and it +was followed directly.) + +## Open questions for the user + +1. **Import method for `USI_Theme_Module.mpk`** — it's a full module export (`.mpr` + brand + assets + ~40 bundled jars/widgets), never imported by any project via mxcli before. Should I + probe `./mxcli --help` / the import subcommand for this project's mxcli version before doing + the actual import, or would you rather do that import yourself in Studio Pro? +2. **Lifecycle-state → colour mapping** on `Routing_Overview`'s grid (e.g. Released=success, + Draft=warning, Retired=danger) — I proposed one above but the BRD doesn't specify it; please + confirm or correct the mapping. +3. **Route graph widget** on `Routing_View` — is a simple read-only list/tree acceptable for the + `GetRouteGraphReadModel` data, or does this need an actual graph-rendering widget? The BRD + doesn't specify presentation, only that the data must be shown. +4. Should the theme import (and any resulting jar/widget bloat it brings — CommunityCommons, + Email Connector, Encryption, OIDC, OQL, several Datagrid widgets) be flagged as a decision to + record in `PROJECT.md`, given this is a POC/demo project favouring speed and a minimal + footprint? diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-3.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-3.md new file mode 100644 index 00000000..bcc1d171 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-3.md @@ -0,0 +1,120 @@ +## Design system + +This is a USI app, wired to the USI company brain +(`/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real`, pointed to from +`CLAUDE.md`'s Company-brain block). Its routing table says: "Choosing a design system, theme, brand +colour or page shell for a USI app → `components/USI_Theme_Module.md`", and the house rule is explicit: +**never hand-roll a theme or a second design system for a USI app.** + +The design system to use is **`USI_Theme_Module`** — a Mendix theme module package +(`components/USI_Theme_Module.mpk`, 56 MB, build `0319`/2024-03-19, approved 2026-09-20). It ships: +- Brand SCSS variables (`themesource/usi_theme_module/web/usi-custom-variables.scss`) +- An Atlas layout override (`layouts/_layout-atlas.scss`) +- Per-component SCSS overrides (`_navigation-bar`, `_button`, `_card`, `_input`, `_list-view`, + `_headings`, `_footer`, `_tab-container`, `_radio-button`, `_lang-selector`, `_base`) +- A `Button` design property with two options: `USI blue` / `USI red` + +It is not currently imported into this project (no `themesource/usi_theme_module/` folder exists yet +under `USI_Routing_Workshop`). Importing it is a prerequisite build step before the routing overview +page can be styled correctly — and the manifest flags that its own import method is **not yet verified +on any mxcli version** ("Proven in" table is empty), so the import needs a probe +(`./mxcli --help`, then the import subcommand's help) before it's attempted, not an assumed CLI command. + +The theme's typeface is **Open Sans** (bundled with the module), not the corporate print typeface +(Helvetica LT Std) and not Atlas's default stack — this is a deliberate web/print split, not +something to "correct." + +## Brand colours + +All values and names below come from `components/USI_Theme_Module.md` +(`usi-custom-variables.scss`), cross-referenced against the corporate brand guide +(`components/USI-UI-UX-Guide-v2.0.pdf`, not opened page-by-page here — the manifest already states +the reconciliation). No hex value is hand-typed into any page per house convention #2; these are +theme variables to be referenced, not literals to copy into widget properties. + +| Variable | Value | Meaning | Source | +|---|---|---|---| +| `$brand-primary` | `#0C4C8A` | UI primary — Pantone 541C, "Bright" palette | `USI_Theme_Module.md` | +| `$brand-success` | `#437242` | USI green (357C) | `USI_Theme_Module.md` | +| `$brand-warning` | `#ed6d0f` | USI orange (1565C) | `USI_Theme_Module.md` | +| `$brand-danger` | `#e60012` | USI red (485C) | `USI_Theme_Module.md` | +| `$sidebar-bg` | `#24276c` | side navigation background | `USI_Theme_Module.md` | +| `$topbar-bg` | `#FFFFFF` (70px height) | top bar | `USI_Theme_Module.md` | +| `$bg-color` | `#f8f8f8` | page background | `USI_Theme_Module.md` | +| `$font-family-base` | `"Open Sans", sans-serif` | typography | `USI_Theme_Module.md` | +| `$border-radius-default` | `4px` | corner radius | `USI_Theme_Module.md` | + +**Deliberate deviation, do not "fix":** the brand guide's Corporate colour is USI Blue, Pantone +655C, `#002662` — that is the logo/print blue. `$brand-primary` (`#0C4C8A`, Pantone 541C) is the +correct on-screen primary; all four theme brand colours are consistently drawn from the guide's +Extended → Bright row, not the Corporate row. `#002662` should never be substituted into +`$brand-primary`. + +Buttons use the theme's **Button Style** design property (`usi-blue` / `usi-red`), never Atlas's +default button styles and never a hand-rolled class, per house convention #3. + +## Page plan + +Sketch of **Routing_Overview**, per BRD `F001-routing-overview.brd.json` (UC001) and house page +naming convention #7 (`_Overview`): + +- **Layout:** `Atlas_Core.Atlas_Default`, with the USI theme's Atlas layout override applied + (top bar white/70px, sidebar `#24276c`, page background `#f8f8f8`). +- **Header / title bar:** "Routing Catalogue" heading, styled by the theme's `_headings.scss`. +- **SearchFilters section** (top): filter by Site, Product Family, and Lifecycle State — the three + fields UC001's main flow names ("search routing headers by site, product family, or lifecycle + state"). A primary "Search" action button using the **USI blue** button style. +- **ReleasedContextSummary section**: a status banner above or beside the grid, sourced from + `GET_ReleasedRoutingContext` (bound to `RouteResolverReadModel`), using `$brand-success` / + `$brand-warning` / `$brand-danger` for released/pending/blocked state indicators rather than any + hand-typed colour — open question F001-RO-1 (below) affects exactly what this banner shows. +- **ResultGrid section**: a data grid over `RoutingReadModel` (via `GET_Routing_List`), columns + drawn from its BRD attributes: RoutingCode, RoutingNameDefault, LifecycleState, ReleaseStatus, + IsCurrent, ProductFamilyCode, CustomerCode, EffectiveFrom/EffectiveTo. Row click drills into + `Routing_View` via `GET_Routing_Detail`, per house page-naming convention (`Routing_View` + already named in the BRD). +- **Empty/loading states**: styled by the theme's `_list-view.scss`, no custom CSS. + +This is a page *plan* only — no MDL has been written or executed, per the task's preparation-only +scope. Before the first widget is actually built, `skills/ui-preflight-pages.md` (wireframe → tokens +→ StyleGallery) and `skills/module-brief.md` still apply on top of this theme choice, and a proper +annotated wireframe under `design/wireframes/Routing_Overview.html` is the toolkit's expected +Stage-3 deliverable (not yet produced — `design/` doesn't exist in this project yet). + +## Files I read + +1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/PROJECT.md` +2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/intake.md` +3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.md` +4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.local.md` +5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop` (directory listing — no `docs/progress/RESUME.md` present) +6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real` (directory listing) +7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/ROUTING.md` +8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/README.md` +9. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/USI_Theme_Module.md` +10. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/` and `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/` (directory listings) +11. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/house-page-conventions.md` +12. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` + +(The corporate brand-guide PDF, `components/USI-UI-UX-Guide-v2.0.pdf`, was not opened directly — the +`USI_Theme_Module.md` manifest already states the reconciliation between the guide's Corporate blue +and the theme's on-screen primary; flagged as an open question below in case the user wants it +opened directly.) + +## Open questions for the user + +1. **Theme import not yet verified for this mxcli version.** `USI_Theme_Module.md` says the import + method (CLI / MCP / Studio Pro) has never been confirmed working on any mxcli version, and this + would be the first project to do it. Should I probe `./mxcli --help` / the import subcommand's + help now and report back before anyone attempts the import, or do you want to import it via + Studio Pro yourself first? +2. **F001-RO-1 (already open in `PROJECT.md`):** should `GetReleasedRoutingContext` populate both + `Routing_Overview`'s `ReleasedContextSummary` banner and `Routing_View`'s released-context + banner with the same data, or is one of those meant to be route-specific? This changes what the + ReleasedContextSummary section on the overview page actually shows. +3. Do you want the full annotated wireframe (`design/wireframes/Routing_Overview.html`) built next, + per the toolkit's `ui-preflight-pages.md` / `design-artifacts.md` convention, before any MDL is + written for this page — or is this plain-language sketch sufficient to move straight to build? +4. Should I open the corporate `USI-UI-UX-Guide-v2.0.pdf` directly to confirm the Bright-palette + reconciliation myself, or is the manifest's stated deviation (already verified 2026-09-20) + sufficient? diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-1.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-1.md new file mode 100644 index 00000000..c7c9c199 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-1.md @@ -0,0 +1,94 @@ +# USI_Routing_Workshop — Design system, brand colours, and Routing_Overview sketch + +**Status check first:** this project is at Stage 1 (Analysis) complete, Stage 2 (Requirements) +next — the user's own framing ("nearly closed," not closed). The toolkit's process puts the +design-system/brand decision at Stage 3 (`design-artifacts.md`, Step 0b), as a `✋` interview +gate recorded `CONFIRMED`/`ASSUMED` in `PROJECT.md` — not something a session decides for +itself. Nothing in this project's `sources/`, BRDs, or `PROJECT.md` names a client palette, +logo, or style guide, and no `design/` folder exists yet. So below is the **recommendation and +evidence** Step 0b asks the agent to bring to that gate, not a confirmed decision — it still +needs to be asked and answered in chat before anything gets built on top of it. + +## Design system +No design system is chosen yet — this project has no `design/brand.md`, no `design/ds.css`, +and no client branding material anywhere in `sources/` or the knowledge base, so there is +nothing to report as decided. Recommendation: **Atlas UI defaults (Atlas Core + Atlas Web +Framework, `Atlas_Core.Atlas_Default` layout)** — the decision register already scopes this as +a POC/demo ("speed over completeness, throwaway output," `PROJECT.md` row P) and every BRD page +(`Routing_Overview`, `Routing_View`) already specifies `Atlas_Core.Atlas_Default` as its layout, +so Atlas defaults is the toolkit's own recommended basis for exactly this situation +(`design-artifacts.md` Step 0b: "For a faithful rebuild POC, Atlas defaults are usually right"). +This must still be asked as an explicit `✋` gate question in chat (options: client branding +guidelines / Atlas defaults / neutral placeholder palette) and confirmed before Stage 3 design +work starts — per the toolkit, an un-asked default may not be recorded as `ASSUMED`. + +## Brand colours +No brand colours are confirmed. There is no `design/brand.md`, no client style guide in +`sources/`, and no colour reference anywhere in the BRDs or knowledge-base documents (checked +`F001-routing-overview.brd.json`, `F001-routing-management.brd.deferred.json`, +`analysis/brd-report.html`, `extraction-report.html`) — the only design-adjacent fact present +is the Atlas layout name on each page definition. + +If Atlas defaults is confirmed at the gate, the palette is simply **Atlas Core's stock theme** +(no override): primary blue `#0595DB`, dark navy `#265787`, success/warning/danger from the +Atlas Core semantic set, neutrals from Atlas's default background/border scale — sourced from +the Atlas UI Resources theme package, not hand-picked. These are placeholder-safe stand-ins +Mendix ships by default; if the user instead wants a distinct brand, that requires either (a) +client branding guidelines as an analysis deliverable (none supplied so far), or (b) the +`dataviz` skill's neutral placeholder palette as a brand-agnostic starting point. Either +alternative is a live option at the same gate, not a fallback I can pick silently. + +## Page plan +Sketch for **Routing_Overview** (`RoutingManagement` module, `Atlas_Core.Atlas_Default` layout, +data context `RoutingSearch_Dto`), built from the confirmed F001/UC001 scope and the BRD's own +section/action list — concrete, but still to be turned into an actual wireframe file under +`design/wireframes/Routing_Overview.html` per `design-artifacts.md` Step 3, after the design +system exists: + +1. **Header / page title bar** — "Routing Catalogue" (or similar), Atlas default page header. +2. **SearchFilters** (top section, collapsible filter bar) — filters implied by + `SearchRoutes`/the BRD's `Routing` fields worth filtering on: Site (`SiteId`), Product family + (`ProductFamilyCode`), Lifecycle state (`LifecycleState`), free-text search on + `RoutingCode`/`RoutingName`. A "Search" action button, "Clear filters" link. +3. **ResultGrid** (main body, Atlas DataGrid2) — one row per `Routing`, columns: Routing Code, + Routing Name, Routing Type, Product Family, Site, Owner Team, Lifecycle State (as a status + badge/pill), Active (Y/N). Row click drills into `Routing_View`. Backed by + `GET_Routing_List` (`SearchRoutes`) and `GET_Routing_Detail`. +4. **ReleasedContextSummary** (side panel or footer strip on the selected/hovered row) — + released-context status pulled from `GET_ReleasedRoutingContext`, showing current released + version/effectivity state at a glance without a full drilldown. +5. Read-only throughout — no create/edit affordances (out of scope per F001 UC002-UC006, + deferred), consistent with the "read-only routing catalogue" scope decision. + +This structure maps directly onto the BRD's three named sections (`SearchFilters`, +`ResultGrid`, `ReleasedContextSummary`) and three actions +(`GET_Routing_List`, `GET_ReleasedRoutingContext`, `GET_Routing_Detail`) — nothing here is +invented beyond arranging those into a layout; per `design-artifacts.md` Step 3b, any +additional UI chrome (breadcrumbs, extra buttons) would still need an explicit cut/spec call +before it ships. + +## Files I read +1. `PROJECT.md` +2. `intake.md` +3. `triage.md` +4. `widgets-inventory.txt` +5. `analysis/knowledge-base/brd/F001-routing-overview.brd.json` +6. `analysis/knowledge-base/brd/F001-routing-management.brd.deferred.json` (grep only, for brand/theme terms) +7. `analysis/knowledge-base/brd/index.json` +8. `CLAUDE.local.md` (this project's) +9. `CLAUDE.md` (this project's, partial — communication style, mxcli reference sections) +10. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` +11. (directory listings/greps, not single files: `find analysis -type f`, `grep -ril` for brand/colour/theme/design-system/wireframe/logo terms across the project, `find . -iname "*source*"` / `-iname design` / `-iname sources`) + +## Open questions for the user +1. **Branding basis** (the Step 0b `✋` gate, not yet asked): client branding guidelines, Atlas + defaults, or a neutral placeholder palette? I'm recommending Atlas defaults given the + POC/demo driver, but this needs your explicit answer to record as `CONFIRMED`. +2. If client branding guidelines exist (logo, palette, type), can they be dropped into + `sources/` as an analysis deliverable before Stage 3 design work starts? +3. **Navigation layout** — top-bar nav or side-nav for this app? (`design-artifacts.md` Step 0b + requires this be asked explicitly, in the same gate, not defaulted.) +4. Given requirements are "nearly closed" but Stage 2 isn't gated complete yet + (`PROJECT.md` marks Stage 1 done, Stage 2 next) — do you want to formally close Stage 2 + before Stage 3 design work (design system + wireframes) begins, or proceed with both + in parallel? diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-2.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-2.md new file mode 100644 index 00000000..a203f2b6 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-2.md @@ -0,0 +1,59 @@ +## Design system + +No design system has been established for this project yet, and none can be truthfully reported as "in use." The project is still at **Stage 1 (Analysis, complete) / Stage 2 (Requirements) next** per `PROJECT.md`; design work is Stage 3 territory under the toolkit's `skills/design-artifacts.md`, and this project has no `design/` directory, no `design/ds.css`, no `design/brand.md`, and no Step 0b "Establish the Branding Basis" interview on record. That gate interview — client branding guidelines vs. Atlas defaults vs. a neutral placeholder palette — has not been asked or answered in `PROJECT.md`, so it is open, not defaulted. + +Given this project's own confirmed driver — **(a) POC/demo, speed over completeness** (`PROJECT.md` row P) — the toolkit's guidance under Step 0b is explicit: *"For a faithful rebuild POC, Atlas defaults are usually right — the wireframes then look like what Mendix will actually render."* So my working recommendation is **Mendix Atlas UI defaults (Atlas_Core / Atlas_Default layout, stock Atlas theme, no custom brand)** — but this is a recommendation to put to you in the Step 0b gate, not a confirmed decision, because I have not seen it asked and answered. + +## Brand colours + +No brand colours can be sourced yet — there is no `design/brand.md`, no client style guide in `analysis/`/`sources/`, and no confirmed branding basis. If you confirm "Atlas defaults" at the Step 0b gate, the colours are simply Mendix's stock Atlas palette (no hex values chosen by me): Atlas primary blue (`$brand-primary`, `#0595DB` in stock Atlas_Core), the standard Atlas neutrals/greys for surfaces and text, and Atlas's default success/warning/danger status colours — inherited unmodified from the Atlas_Core theme, not picked or tuned here. My source for that value is the stock Atlas_Core `variables.scss` shipped with Mendix Studio Pro (a toolkit/Mendix default, not anything specific to this app). If instead you have (or want) a client palette, that has to be supplied or chosen in the same gate — I have no source document containing one; the four source documents in `analysis/knowledge-base/share/` are a functional spec, API behavior spec, OpenAPI contract, and DB schema, none of which carry visual/brand content. + +## Page plan + +Sketched from the confirmed scope (`PROJECT.md`: F001/UC001 only) and the BRD at `analysis/knowledge-base/brd/F001-routing-overview.brd.json`. This is a plan on top of Atlas defaults, pending the Step 0b confirmation above — no wireframe HTML has been produced (that is a Stage 3 deliverable this session did not create). + +**Screen: `Routing_Overview`** (top-level page, RoutingManagement module) +- **Header / control bar:** page title "Routing Overview"; filter/search row — Site (combobox), Product Family (combobox/textbox), Lifecycle State (combobox) — feeding `SearchRoutes` +- **Results data grid** (Atlas DataGrid2), one row per `Routing` header, columns: + - RoutingCode + - RoutingName + - RoutingType + - ProductFamilyCode + - OwnerTeam + - SiteId + - LifecycleState (status badge/pill styling) + - IsActive (badge) +- Row click / "View" action button → opens `Routing_View` for the selected routing, passing RoutingId +- Empty-state message when search returns nothing +- Read-only throughout (UC001 is explicitly read-only; C01.ROUTE.READ scope) + +**Screen: `Routing_View`** (drilldown, opened from overview) +- Header block: RoutingCode / RoutingName / RoutingType / LifecycleState / OwnerTeam (from `GetRouteById`) +- Released context panel: current effectivity / governed context (from `GetReleasedRoutingContext`) +- Version list (data grid): VersionNo, RevisionCode, IsCurrent, EffectiveFrom/To, ReleaseStatus (from `ListRouteVersions`) +- Version detail on selection (from `GetRouteVersionDetail`) +- Effectivities sub-list (from `ListRoutingEffectivities`) +- Route graph visualization area (from `GetRouteGraphReadModel`) — placeholder pending a decision on which widget renders a graph (not in Atlas's standard widget set; a StyleGallery/custom-container or marketplace widget choice belongs in the Step 0b/architecture pass, not decided here) +- "Back to Overview" navigation + +Both screens are read-only per the confirmed scope; no create/edit/delete affordances belong on either. + +## Files I read + +1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/CLAUDE.md` +2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/CLAUDE.local.md` +3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/PROJECT.md` +4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/triage.md` (partial) +5. Directory listing of `analysis/` (found `brd-report.html`, `knowledge-base/`, `facts.lock.json`; no `design/`, `architecture/`, or `.css` files anywhere in the project) +6. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (Step 0a/0b/1/2/3, read for the process governing design system + branding decisions) +7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (UC001 flow, screens, and the `Routing`/`RoutingVersion` entity attributes used for the page plan) + +Not read: `AGENTS.md` and `intake.md`/`widgets-inventory.txt` (redundant with CLAUDE.md / out of scope for a design-only question), and the rest of the BRD/KB files (deferred BRDs are out of the confirmed F001-only scope). + +## Open questions for the user + +1. **Branding basis (toolkit Step 0b gate — not yet asked):** Atlas defaults (my recommendation, matches your confirmed POC/demo driver), client branding guidelines (if one exists, please share it), or a neutral placeholder palette to swap later? +2. Requirements are described as "nearly closed" but `PROJECT.md` shows Stage 2 (Requirements) has not started/closed and Stage 3 design work hasn't begun — do you want me to proceed with design now in parallel, ahead of the stage gate, or close Stage 2 first per the toolkit's normal sequencing? +3. Top-bar nav vs. side-nav for the app shell — the toolkit treats this as a first-class design question, not a default, and it hasn't been asked yet. +4. How should the route graph (`GetRouteGraphReadModel`) actually render on `Routing_View`? Atlas has no built-in graph widget — options are a marketplace visualization widget, a custom container with an embedded chart, or a simplified list/tree view instead of a true graph. +5. Open items D1–D3 in `PROJECT.md` (conflicting API counts, document-version ambiguity, possible superseding coding-handoff document) are still unresolved — do any of them affect the Routing_Overview/Routing_View field set before we lock the wireframe? diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-3.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-3.md new file mode 100644 index 00000000..d82a5ef3 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-3.md @@ -0,0 +1,125 @@ +## Design system + +**Recommendation (not yet confirmed): Atlas UI (Atlas Core `Atlas_Default` layout), styled with Atlas defaults — no client brand exists to target.** + +Sources checked for a brand basis: `intake.md`, `triage.md`, `PROJECT.md`, and the four +source-of-truth documents in `analysis/knowledge-base/share/` (functional spec, API behavior +spec, OpenAPI contract, DB schema). None contain a palette, logo, typography spec, or brand +guideline — the source is a pure back-end/API description (MES 2.0), not a UI mockup. The +only UI signal in the BRD itself is the layout each page already declares: +`"layout": "Atlas_Core.Atlas_Default"` on both `Routing_Overview` and `Routing_View` in +`analysis/knowledge-base/brd/F001-routing-overview.brd.json` — i.e. whoever wrote the BRD +already assumed stock Atlas, sidebar layout. + +Per the toolkit's `skills/design-artifacts.md` Step 0b ("Establish the Branding Basis"), the +branding basis is itself a `✋` gate that must be asked in chat, not defaulted silently. Given +this project's own decisions — POC/demo driver (`PROJECT.md` row "Project driver: (a) +POC/demo"), no SME, document-only source, no brand material anywhere in `sources/` or the KB +— **"Atlas defaults"** is the option the skill's own guidance table recommends for exactly +this situation ("POC / no brand yet — matches the actual build target 1:1, zero effort"). I +am treating that as my recommendation, not a confirmed decision — see Open Questions. + +## Brand colours + +No client-supplied colours exist anywhere in the source material, so there are no hex values +to report as "the brand's own." Recommended path, pending the Step 0b confirmation above: +**use Atlas Core's out-of-the-box palette untouched** (no `custom-variables.scss` overrides), +so the values below are Atlas's shipped defaults, not project-chosen brand colours: + +| Token | Atlas variable | Typical Atlas default | Use | +|---|---|---|---| +| Primary | `$brand-primary` | `#0595db` (Mendix blue) | Primary buttons, links, active nav | +| Success | `$brand-success` | `#67c157` | Positive status badges (e.g. "Released") | +| Warning | `$brand-warning` | `#f99e1c` | In-progress / pending status | +| Danger | `$brand-danger` | `#dd3145` | Errors, "Obsolete"/blocked status | +| Background | `$background-color` | `#f5f5f5` | Page background | +| Text | `$font-color` | `#5e5e5e` / `#000` | Body copy | + +These are Atlas Core's shipped tokens (per `design-artifacts.md`'s Atlas mapping table +convention), not measured from this project's `.mpr` — I did not open the `.mpr` or any +`theme/` folder because neither exists yet in this checkout (no `design/`, `architecture/`, +or `themesource/` directories present). If the app already has a customized Atlas theme +checked in elsewhere, that would supersede this and should be confirmed before I write +`design/ds.css`. + +**If the user instead wants a placeholder brand** (Option 3 in Step 0b — "design-forward but +brand-agnostic, swap later"), I'd pull from the `dataviz` skill's validated reference palette +for any status/data colours rather than hand-picking, and record the choice + rationale in +`design/brand.md` as the skill requires. I have not done this because it's a decision, not a +default. + +## Page plan + +Concrete structure for **`RoutingManagement.Routing_Overview`**, built from the BRD's own +`sections`/`actions`/`dataContext` (F001-routing-overview.brd.json) and the widgets already +vendored in this project (`widgets-inventory.txt`): + +- **Layout:** `Atlas_Core.Atlas_Default` (left sidebar nav, per the BRD's own layout field — + also the layout-pattern question `checkpoint-design.md` Q1 would ask; "left sidebar" is the + Atlas_Default recommendation whenever there's no other nav signal to override it). +- **Data context:** `RoutingSearch_Dto` (non-persistent search/filter object). + +1. **Page header** — title "Routing Overview", breadcrumb-free (top-level nav item). +2. **`SearchFilters` section** (BRD section 1) — a filter bar above the grid: + - Route code / name text filter (`DatagridTextFilter`) + - Status combobox filter (`Combobox` + `DatagridDropdownFilter`) — values driven by + whatever status enum the domain model defines (not yet extracted into this BRD's + `domainEntities` in the excerpt I read — flag for the architect) + - "Search" action button, calling `GET_Routing_List` +3. **`ResultGrid` section** (BRD section 2) — the primary surface: + - `Datagrid` widget (already vendored) bound to the search results + - Columns: Route code, Route name, Version (current), Status (as a `Badge`/`BadgeButton` + status pill — both widgets are vendored), Last released date + - Row click → navigates to `Routing_View` (drilldown), calling `GET_Routing_Detail` + - Empty-state and loading-state per Atlas defaults +4. **`ReleasedContextSummary` section** (BRD section 3) — a summary strip/panel, likely above + or beside the grid: aggregate counts (e.g. "X routes released", "Y pending") sourced from + `GET_ReleasedRoutingContext`. Rendered as KPI/stat tiles per `design-artifacts.md`'s + component list, using the `dataviz` skill's palette if any of these become chart/KPI colour + choices rather than plain text. + +This is a **DESIGN FROM BRD** tier-2 wireframe (per `design-artifacts.md` Step 2's +input-priority ladder) — there is no source screenshot in this project, only the BRD's field +list, so the wireframe is derived from that BRD plus sibling-screen conventions, not from a +mockup. Before this becomes an actual `design/wireframes/Routing_Overview.html` file, Step 3b +of the same skill requires walking it for chrome with no requirement behind it (e.g. a "clear +filters" link, a status legend) and either cutting it or specifying what triggers it — I have +not built that file, per this task's "preparation only" scope. + +## Files I read + +1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/CLAUDE.md` +2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/CLAUDE.local.md` +3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/PROJECT.md` +4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/intake.md` +5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/triage.md` +6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/widgets-inventory.txt` +7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/analysis/knowledge-base/share/KB_MES20_APIBehaviorSpec.md` (grep only, then targeted read) +8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +9. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` +10. `/home/user/mxcli-project-toolkit/skills/conversion-runbook.md` (Stage 3 section, via grep) +11. `/home/user/mxcli-project-toolkit/skills/checkpoints/checkpoint-design.md` + +Also listed (not opened as content, only directory-checked): `analysis/knowledge-base/brd/*.deferred.json` filenames, `analysis/knowledge-base/reports/validation-report.md` filename, `analysis/knowledge-base/share/*.md` filenames (grepped for brand/color/theme keywords, no matches beyond the Atlas layout hit above and an unrelated "styled" match in the API spec). + +## Open questions for the user + +1. **Branding basis (Step 0b gate — not yet asked or confirmed):** no client palette, logo, + or design guideline exists anywhere in this project's sources. Confirm: (A) Atlas defaults + — recommended, matches the POC/demo driver and zero-brand-material reality, or (B) a + neutral placeholder brand palette (swap later), or (C) you have branding material not yet + dropped into `sources/` — if so, where should I look? +2. **Atlas layout pattern (checkpoint-design.md Q1):** the BRD already assumes + `Atlas_Default` (left sidebar). Confirm that's the intended nav pattern rather than + `Atlas_TopBar`, or a custom layout. +3. **Platform target (checkpoint-design.md Q2):** web-only desktop, or responsive + web/tablet? Nothing in the source material states this. +4. **Status enum for the routing header** (needed for the status badge/filter on the + overview grid): the BRD excerpt I read doesn't enumerate the status values (e.g. + Draft/Released/Obsolete) — should I pull these from `MES_2.0_C01_Core_DB_Table_Schema` in + the KB, or do you want to specify them directly? +5. This project's `PROJECT.md` currently records **Stage 1 (Analysis) complete, Stage 2 + (Requirements) next** — Stage 3 (Architecture & Design, where this design-system work + formally lands) hasn't opened as a gate yet. Do you want to formally open Stage 3 now (via + the toolkit's checkpoint-design.md brainstorm-first protocol), or is this look-and-feel + conversation meant to run ahead of that gate as informal prep? diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-1.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-1.md new file mode 100644 index 00000000..fde824f0 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-1.md @@ -0,0 +1,72 @@ +## Design system + +**Atlas UI (Atlas Core / Atlas_Default), unmodified — no custom design system for this build.** + +Evidence, in order of weight: +1. The BRD (`analysis/knowledge-base/brd/F001-routing-overview.brd.json`) already specifies both screens' layout as `Atlas_Core.Atlas_Default` for `Routing_Overview` and `Routing_View` — this was decided at KB-generation time, not invented here. +2. Per the toolkit's `design-artifacts.md` (Step 0b, "Establish the Branding Basis"), this decision is a gated interview with three options: client branding guidelines / Atlas defaults / neutral placeholder palette. I searched every source document (`sources/` via the KB extracts) and `PROJECT.md`'s decision register for a brand name, logo, palette, or style guide reference and found **none** — no `design/brand.md` or `design/target-ui.md` exist yet either. +3. `PROJECT.md`'s own driver decision is `(a) POC/demo — speed over completeness, throwaway output` (row P/2, CONFIRMED), and fidelity is `port as-is` (ASSUMED, ties to the POC driver). `design-artifacts.md` states explicitly: "For a faithful rebuild POC, Atlas defaults are usually right." +4. There is no legacy app and no client mockup (`requirements-driven` entry mode, no screenshots anywhere in the project) — so there is nothing to be faithful *to* besides the spec, and no reason to invest in a bespoke `ds.css`/brand layer for a throwaway POC. + +Conclusion: use stock Atlas Core, no custom theme, no `design/ds.css`/`brand.md` token layer, no StyleGallery module. This is a recommendation for the ✋ gate the skill requires — see Open Questions; it has not yet been asked/confirmed in chat, so nothing here is written to `PROJECT.md` and no design files have been created. + +## Brand colours + +None — because "Atlas defaults, unbranded POC" is the recommended basis, there is no brand palette to source. The **only** colours used are Atlas Core's stock semantic tokens, taken as-is (not hand-picked, not overridden): + +| Token | Atlas default | Use | +|---|---|---| +| `$brand-primary` | Atlas Core default blue (`#2C64EE` in current Atlas Core release) | Primary buttons, links, active nav | +| `$brand-success` | Atlas Core default green | "Released" status badge | +| `$brand-warning` | Atlas Core default amber | "InReview" / "Approved" (pending) status badges | +| `$brand-danger` | Atlas Core default red | "Retired" status badge, error toasts | +| `$text-color-*`, `$background-color`, greys | Atlas Core defaults | Body text, page/card backgrounds | + +Source: Atlas Core's own `custom-variables.scss` defaults, not a client document — because no client palette exists in the source pack. If a real brand appears later, only this table (and the Atlas mapping) needs to change; nothing in the page plan below depends on specific hex values. + +Status-badge colour **mapping**, not the colours themselves, does come from the source: the lifecycle states `Draft → InReview → Approved → Released → Retired` are attested in `analysis/knowledge-base/share/KB_MES20_APIBehaviorSpec.md` (state-transition rows SB-001/SB-003/SB-004) and the `LifecycleState` attribute on the `Routing` entity in the F001 BRD. + +## Page plan + +Scope per `PROJECT.md`: **`Routing_Overview` only** (F001/UC001, search + monitor; drilldown `Routing_View` is a separate screen, out of scope for this sketch). Structure taken directly from the BRD's `pages[0]` entry (sections: `SearchFilters`, `ResultGrid`, `ReleasedContextSummary`; actions: `GET_Routing_List`, `GET_ReleasedRoutingContext`, `GET_Routing_Detail`) and the `Routing` entity's attributes. + +**`RoutingManagement.Routing_Overview`** — Atlas_Core.Atlas_Default layout: + +1. **Page header** — title "Routing Overview", breadcrumb (home > Routing). +2. **SearchFilters** (top card, LAYOUTGRID row of filter controls, matching `SearchRoutes` query capability): + - Site (COMBOBOX, bound to `SiteId` / site reference) + - Product family (COMBOBOX/TEXTBOX, `ProductFamilyCode`) + - Lifecycle state (COMBOBOX enum: Draft / InReview / Approved / Released / Retired) + - Free-text search (TEXTBOX, matches `RoutingCode`/`RoutingName`) + - "Search" ACTIONBUTTON (primary, triggers `GET_Routing_List` / `SearchRoutes`) +3. **ResultGrid** (DATAGRID, DataSource: database `RoutingManagement.Routing`, one row per routing header): + - Columns: Routing Code (`RoutingCode`), Routing Name (`RoutingName`), Type (`RoutingType`), Product Family (`ProductFamilyCode`), Site (`SiteId`), Owner Team (`OwnerTeam`), Lifecycle State (`LifecycleState`, rendered as a coloured status badge per the mapping above), Modified On (`ModifiedOn`) + - Row click → `GET_Routing_Detail`, opens `Routing_View` (out of scope to build now, but the nav target) +4. **ReleasedContextSummary** (card/panel beside or below the grid, populated by `GET_ReleasedRoutingContext` for the selected row): shows canonical ID, released version, and released-context status as a small KPI/stat-tile row (green/amber/red dot per state) — no data mutation, read-only. +5. **Empty/loading/error states**: empty-search placeholder, spinner while `SearchRoutes` is in flight, and an error banner using the `ProblemDetails` envelope shape (`errorCode` + `messageKey`) attested in `KB_MES20_APIBehaviorSpec.md`. + +All widget choices are read-only (no CREATE/CHANGE actions) — matching the confirmed scope (F001 UC001 only; UC002–UC006 and F002–F004 are deferred, not built). + +## Files I read + +1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/AGENTS.md` (diffed identical to CLAUDE.md) +2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.md` +3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/PROJECT.md` +4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/intake.md` +5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop` directory listing (`find analysis -type f`) +6. Grep of `analysis`, `intake.md`, `triage.md`, `widgets-inventory.txt`, `CLAUDE.md` for color/brand/theme/design-system keywords +7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/share/KB_MES20_FunctionalSpec.md` (grepped) +8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/share/KB_MES20_APIBehaviorSpec.md` (grepped, multiple passes) +9. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (full read) +10. `/home/user/mxcli-project-toolkit/skills/` directory listing (ls, to locate design-related skills) +11. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (full + targeted `pages`/`integrations` sections via python) + +(Not opened: `.mpr` file, `triage.md` full body, `widgets-inventory.txt` full body, `AGENTS.md` full body — all confirmed via grep/diff to carry no design/brand content beyond what's captured above. No task instructs modifying the project or the toolkit repos, and none were touched.) + +## Open questions for the user + +1. **Branding basis gate (`design-artifacts.md` Step 0b) has not actually been asked yet.** I'm recommending "Atlas defaults, no custom brand" based on the POC driver and absence of any brand material in sources — but the skill requires this as an explicit ✋ interview question, not a silent default. Confirm: Atlas defaults, or do you have brand guidelines (logo/palette) I should use instead? +2. **Dark mode** — build it now, or Atlas light-only for this POC? +3. **Navigation layout** — top-bar vs. side-nav for the app shell (the skill flags this as a first-class question, not a background default)? +4. Should I still produce `design/brand.md`, `design/ds.css`, and a wireframe file for `Routing_Overview`, or is this chat-level sketch sufficient before Stage 2/3 formally starts (project is currently Stage 1 complete → Stage 2 next per `PROJECT.md`)? +5. Open BRD questions D1–D3 in `PROJECT.md` (conflicting API counts, document-control version, Artifact 14 availability) are unrelated to design but still open — flagging since they block later gates. diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-2.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-2.md new file mode 100644 index 00000000..720aa293 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-2.md @@ -0,0 +1,107 @@ +## Design system + +**`USI_Theme_Module`** — the company-brain-approved Mendix theme module, not a bespoke or Atlas-default theme. + +- Source of the rule: this project's `CLAUDE.local.md` wires a USI company brain at + `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real`, whose `ROUTING.md` routes + "choosing a design system, theme, brand colour or page shell for a USI app" to + `components/USI_Theme_Module.md`. +- That manifest and `skills/house-page-conventions.md` are explicit: **never hand-roll a theme + or a second design system for a USI app** — import `USI_Theme_Module.mpk` and build on + `themesource/usi_theme_module/`. It's a full module export (brand SCSS variables, an Atlas + layout override, per-component SCSS, a two-value Button Style design property), approved + 2026-09-20, not yet imported into this project. +- Import method is not yet decided: the manifest says to probe `./mxcli --help` / the import + subcommand before choosing CLI, MCP, or Studio Pro — "not yet verified on any mxcli version." + This is preparation only, so no import was performed. +- Typography: **Open Sans**, shipped by the module — not the brand guide's print typeface + (Helvetica LT Std), which is intentional and documented as a deviation not to "fix." + +## Brand colours + +All values are from `USI_Theme_Module.md` → `themesource/usi_theme_module/web/usi-custom-variables.scss` +(the theme's actual SCSS variables — never hand-typed hex in a page): + +| Use | Variable | Value | +|---|---|---| +| Primary / on-screen brand blue | `$brand-primary` | `#0C4C8A` | +| Success | `$brand-success` | `#437242` | +| Warning | `$brand-warning` | `#ed6d0f` | +| Danger | `$brand-danger` | `#e60012` | +| Side navigation background | `$sidebar-bg` | `#24276c` | +| Top bar background / height | `$topbar-bg` | `#FFFFFF` / `70px` | +| Page background | `$bg-color` | `#f8f8f8` | +| Border radius | `$border-radius-default` | `4px` | + +**Deliberate deviation, do not "correct":** the corporate print/logo blue in the brand guide +(`USI-UI-UX-Guide-v2.0.pdf`, section on the Corporate palette) is Pantone 655C, `#002662`. The +theme's on-screen `$brand-primary` is `#0C4C8A` (Pantone 541C), because the theme uses the +guide's *Extended → Bright* row for all four brand colours (541C blue, 485C red `#e60012`, +1565C orange `#ed6d0f`, 357C green `#437242`) — verified consistent across all four, and +flagged in the manifest as a documented, intentional deviation. `#002662` stays reserved for +the logo/print artwork. + +Buttons use the theme's **Button Style** design property (`usi-blue` / `usi-red`), not raw +colour classes. + +## Page plan + +**`Routing_Overview`** (module `RoutingManagement`), scope per `PROJECT.md` / `intake.md` / +BRD `F001` UC001 — read-only routing catalogue overview backed by `SearchRoutes` and +`GetReleasedRoutingContext`. Naming follows the house convention +`_Overview` / `_View`, matching the BRD's own screen names. + +Layout, top to bottom, on the USI Atlas layout override: + +1. **Page header** — title "Routing Catalogue", USI top bar (`$topbar-bg` white, 70px). +2. **Search / filter bar** (`Fieldset` widget, USI theme `_input`/`_fieldset` styling): + - Site (`SiteId`) — dropdown filter (`DatagridDropdownFilter`) + - Product family (`ProductFamilyCode`) — text filter (`DatagridTextFilter`) + - Lifecycle state (`LifecycleState`) — dropdown filter (`DatagridDropdownFilter`) + - free-text search on `RoutingCode` / `RoutingName` +3. **Routing list** — `Datagrid` (com.mendix.widget.web.Datagrid, already in + `widgets-inventory.txt`), one row per `Routing`, columns: + - `RoutingCode`, `RoutingName`, `RoutingType`, `ProductFamilyCode`, `OwnerTeam`, `SiteId` + - `LifecycleState` rendered as a **Badge** (`Badge.mpk`), colour mapped to the theme's + semantic variables (e.g. Released → `$brand-success`, Draft/Pending → `$brand-warning`, + Retired/Rejected → `$brand-danger`), never a typed hex. + - Row click → opens `Routing_View` passing the selected `Routing` (drilldown per BRD + mainFlow step 3). +4. **Empty/loading states** styled with the theme's `_list-view` overrides, USI page background + (`$bg-color`). +5. Primary actions (e.g. "Search") use the **USI blue** button style; no destructive actions on + this page (read-only scope — no create/edit/delete per the confirmed scope cut). + +`Routing_View` (drilldown target) is out of scope for this sketch — user asked specifically +for the overview screen — but is named and referenced above for continuity since BRD UC001 +treats the two as one flow. + +## Files I read + +1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.md` +2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.local.md` +3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/PROJECT.md` +4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/README.md` +5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/ROUTING.md` +6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/USI_Theme_Module.md` +7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/house-page-conventions.md` +8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/intake.md` +9. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +10. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/widgets-inventory.txt` + +(Directory listings of the project root and `analysis/` were also run but are not file reads.) + +## Open questions for the user + +1. **`USI_Theme_Module` is not yet imported into this project.** Import it before building any + page — confirm CLI vs MCP vs Studio Pro import method (the manifest says to probe the + installed `mxcli` binary first; it's "not yet verified on any mxcli version"). +2. **Requirements aren't actually closed yet** — `PROJECT.md` shows Stage 1 complete, Stage 2 + next, plus three open questions (D1–D3, e.g. which API-count document is authoritative) and + one unresolved `ASSUMED` (fidelity). Confirm you want design work to proceed in parallel + with Stage 2 rather than gating on it. +3. **Lifecycle-state → badge colour mapping** — I proposed Released/Draft/Retired → + green/orange/red by convention; confirm against the actual `LifecycleState` enum values + once modeled (BRD only types it as a 30-char string, no enum list given yet). +4. Should `Routing_View` (the drilldown screen) be sketched now too, or only after + `Routing_Overview` is confirmed? diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-3.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-3.md new file mode 100644 index 00000000..2b15cbe8 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-3.md @@ -0,0 +1,62 @@ +## Design system + +This is a USI app, so the design system question is already answered by the company brain, not something to re-derive: import **`USI_Theme_Module`** (`usi-brain-real/components/USI_Theme_Module.mpk`, module package, dated build `0319`) and build the routing overview on top of it — no hand-rolled theme or second design system is permitted for a USI app (`skills/house-page-conventions.md` rule 1). It adds `themesource/usi_theme_module/`: brand SCSS variables, an Atlas layout override, and per-component overrides for navigation bar, buttons, cards, inputs, list view, headings, footer, tab container, radio button and language selector — plus a Button Style design property with two approved options (`usi-blue`, `usi-red`). + +Install note for whoever builds this: the module is a full project export (bundles CommunityCommons, Email Connector, Encryption, OIDC, OQL, several Datagrid widgets, ~40 userlib jars), not a plain widget — probe `./mxcli --help` / the import subcommand's own help before choosing CLI, MCP or Studio Pro, since the manifest says this has not yet been verified against any mxcli version. + +## Brand colours + +Source: `usi-brain-real/components/USI_Theme_Module.md` (the manifest, which documents `themesource/usi_theme_module/web/usi-custom-variables.scss`), cross-checked against `USI-UI-UX-Guide-v2.0.pdf` beside it. + +| Token | Value | Use | +|---|---|---| +| `$brand-primary` | `#0C4C8A` | UI primary (buttons, links, active states) | +| `$brand-success` | `#437242` | success/positive status | +| `$brand-warning` | `#ed6d0f` | warning status | +| `$brand-danger` | `#e60012` | error/danger status | +| `$sidebar-bg` | `#24276c` | side navigation background | +| `$topbar-bg` | `#FFFFFF` (70px height) | top bar | +| `$bg-color` | `#f8f8f8` | page background | +| `$font-family-base` | "Open Sans", sans-serif | all text | +| `$border-radius-default` | `4px` | buttons, cards, inputs | + +**Deliberate deviation — do not "fix":** the brand guide's print Corporate colour is USI Blue, Pantone 655C, `#002662` (logo/print use only). The theme's on-screen `$brand-primary` is `#0C4C8A`, Pantone 541C from the guide's Extended → Bright row, and all four theme brand colours (541C blue, 485C red, 1565C orange, 357C green) are consistently drawn from that Bright row. This is verified/approved (2026-09-20), not a bug to reconcile. Same logic for typography: the guide's print typeface is Helvetica LT Std; the web theme intentionally ships Open Sans instead. + +No hex value gets typed directly into a page or page stylesheet — every colour is used via the theme's brand variables or Button Style design property (`house-page-conventions.md` rules 2–3). The USI logo is artwork only (master files from the corporate identity download); it is never recreated, re-typeset, or altered on the overview page. + +## Page plan + +Screen: **`Routing_Overview`** (module `RoutingManagement`, layout `Atlas_Core.Atlas_Default`), per the confirmed F001/UC001 BRD (`analysis/knowledge-base/brd/F001-routing-overview.brd.json`). Naming follows the house convention `_Overview`. + +Data context: `RoutingSearch_Dto`. Actors: Manufacturing Process Engineer, Supervisor, Line Leader (read-only, `C01.ROUTE.READ` scope). + +1. **Top bar / nav shell** — USI theme top bar (`$topbar-bg` #FFFFFF, 70px) with the USI logo artwork left-aligned; side navigation in `$sidebar-bg` (#24276c) per the imported theme, not custom-built. +2. **Page header** — "Routing Overview" heading (theme `_headings` override, Open Sans), short description line. +3. **Search / filter bar** (BRD section `SearchFilters`) — filters routing headers by Site, Product Family, and Lifecycle State, feeding `GET_Routing_List (RoutingSearch_Dto)`. Built with the theme's `_input` styling; a primary "Search" action button uses Button Style `usi-blue`. +4. **Released-context summary strip** (BRD section `ReleasedContextSummary`) — a compact status banner sourced from `GET_ReleasedRoutingContext`, using status badges (Badge widget is available per `widgets-inventory.txt`) coloured via `$brand-success` / `$brand-warning` / `$brand-danger` for lifecycle/release state, never raw hex. Open question F001-RO-1 (below) affects exactly this section. +5. **Result grid** (BRD section `ResultGrid`) — Data Grid 2 (`com.mendix.widget.web.Datagrid` + its Text/Dropdown/Date/Number filter widgets, all present in `widgets-inventory.txt`) over `RoutingReadModel`, columns: Routing Code, Routing Name, Routing Type, Product Family, Owner Team, Site, Lifecycle State (as a status badge), Current Version, Release Status. Row click drills into `Routing_View` via `GET_Routing_Detail`. +6. **Empty/loading states** — theme `_card` styling for a "no routes match your filters" empty state; no custom colours. + +Buttons throughout: theme's `usi-blue` for primary actions (Search, row navigation), `usi-red` reserved for destructive/reset actions only — this screen is read-only, so `usi-red` likely only appears on a "Clear filters" action if styled that way, otherwise a secondary/neutral style. + +## Files I read + +1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.md` +2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.local.md` +3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/ROUTING.md` +4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/USI_Theme_Module.md` +5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/PROJECT.md` +6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis` (directory listing) +7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/house-page-conventions.md` +9. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (partial, for wireframe/branding-interview convention — superseded here by the company brain's fixed theme) +10. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/widgets-inventory.txt` + +(Also listed, not fully read: `usi-brain-real/` top-level directory and `components/` directory for file inventory.) + +## Open questions for the user + +1. **F001-RO-1 (already logged, unresolved):** should `GetReleasedRoutingContext` populate both `Routing_Overview`'s `ReleasedContextSummary` and `Routing_View`'s released-context banner, or is one of those meant to be route-specific? This decides whether the overview's summary strip is a global/site banner or reacts to the current filter/selection. +2. The `USI_Theme_Module.mpk` install method has never been verified against this project's mxcli version (manifest says "not yet verified on any mxcli version, first project to import it fills this in"). Before build starts: should we probe CLI/MCP import support now, or default to Studio Pro import if the probe comes back negative? +3. Requirements are "nearly closed" but Stage was last recorded as Stage 1 complete / Stage 2 next in `PROJECT.md`, and no `architecture/`, `design/wireframes/`, or `design/ds.css` directories exist yet in this project. Do you want me to run this as a formal Stage 3 design-artifacts pass (wireframe HTML + ds.css referencing the theme tokens) before any page gets built, or is this plain-language sketch sufficient to proceed straight to `Routing_Overview` construction? +4. Three open data-provenance questions (D1 API count conflict, D2 document-version conflict, D3 whether Artifact 14 supersedes processed docs) are still unresolved in `PROJECT.md` — none of them block the look-and-feel decision, but they may affect exact field/column names on the grid before build. Confirm these can stay open through the design pass? diff --git a/evals/scenarios/company-brain-design-module/runs/grade-real.sh b/evals/scenarios/company-brain-design-module/runs/grade-real.sh new file mode 100755 index 00000000..271f74a7 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/grade-real.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +# grade.sh — run 2, real artifacts. Markers come from the real USI theme module and brand guide +# and appear NOWHERE in either arm's project files (verified before the run). +# Negation-aware (run 1 defect): "NO company brain is wired" must not score as a hit. +W="$(cd "$(dirname "$0")" && pwd)" +printf '%-13s %-7s %-7s %-7s %-7s %-9s %-8s %s\n' ARM MODULE PRIMARY BTNSTYL PALETTE DEVIATION ATLAS VERDICT +for f in "$W"/out/*.md; do + [ -f "$f" ] || continue + n="$(basename "$f" .md)" + b="$(tr 'A-Z' 'a-z' < "$f")" + bn="$(sed -E 's/(no|not|never|without|absent|missing|neither) [^.;]*//g' <<<"$b")" + m_mod=0; grep -q 'usi_theme_module\|usi theme module' <<<"$bn" && m_mod=1 + m_pri=0; grep -q '#0c4c8a\|0c4c8a' <<<"$b" && m_pri=1 + m_btn=0; grep -q 'usi-blue\|usi-red\|usi blue\|usi red' <<<"$b" && m_btn=1 + m_pal=0; c=0; for h in '#e60012' '#ed6d0f' '#437242' '#24276c'; do grep -q "$h" <<<"$b" && c=$((c+1)); done; [ $c -ge 2 ] && m_pal=1 + # deviation understood: mentions the corporate blue AND does not adopt it as the primary + m_dev=0 + if grep -q '002662' <<<"$b"; then + grep -qE 'logo|print|corporate|not the (theme|ui|screen) primary|do not|deviat' <<<"$b" && m_dev=1 + fi + # fell back to stock Atlas as THE answer + ds="$(awk '/^## Design system/{f=1;next} /^## /{f=0} f' "$f" | tr 'A-Z' 'a-z')" + dsn="$(sed -E 's/(no|not|never|without|rather than|instead of) [^.;,]*//g' <<<"$ds")" + m_atl=0; grep -qE 'atlas (ui )?(defaults|core)|stock atlas|atlas_default' <<<"$dsn" && m_atl=1 + if [ "$m_mod" -eq 1 ] && [ "$m_pri" -eq 1 ]; then v=RETRIEVED + elif [ "$m_mod" -eq 1 ]; then v=PARTIAL + else v=MISSED; fi + printf '%-13s %-7s %-7s %-7s %-7s %-9s %-8s %s\n' "$n" "$m_mod" "$m_pri" "$m_btn" "$m_pal" "$m_dev" "$m_atl" "$v" +done +cat <<'NOTE' + +MODULE=named USI_Theme_Module PRIMARY=#0C4C8A BTNSTYL=usi-blue/usi-red design property +PALETTE=>=2 of the other real theme hexes DEVIATION=cited #002662 as logo/print, not the UI primary +ATLAS=proposed stock Atlas as the answer. RETRIEVED = module + primary colour. +NOTE From 0e8e807ed688d063596ae17d6f2964e906090652 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 20 Sep 2026 07:43:18 +0000 Subject: [PATCH 11/18] Company brain: component install is CLI-first; Studio Pro is a handoff surface, not a write mode learned-mdl-preflight Step 0 names CLI and MCP as the co-equal write modes; MCP needs SP running, so headless means CLI. Guidance, manifest template and eval rubric corrected. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 1 + .../company-brain-design-module/rubric.md | 2 +- skills/company-brain.md | 19 ++++++++++++++----- templates/company-brain/components/README.md | 19 ++++++++++++++----- .../company-brain/components/TEMPLATE.md | 8 +++++--- 5 files changed, 35 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c5d9368f..80956534 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templat Credit the person or project that surfaced the change — the credit line is the thank-you. ## 2026-09-20 +- fix(skills/company-brain.md, templates/company-brain/components/*, eval rubric): **the company-brain install guidance said "choose CLI, MCP or Studio Pro", putting the GUI on the menu as a peer write mode — it is not one.** This toolkit builds headlessly: `learned-mdl-preflight.md` Step 0 names CLI and MCP as the co-equal modes picked by the shape of the work, and `handoff-to-studio-pro.md` is where a human opens a finished model. MCP mode also needs Studio Pro running, so on a headless machine the CLI is the only mode there is. Component guidance is now CLI-first: probe the binary's import subcommand and import from the CLI; a CLI that cannot do it on your version is a defect to record and log, never a cue to reach for the GUI. The manifest field is now "the CLI command that actually worked" with the mxcli version it was verified on, and the eval rubric scores a Studio Pro route as zero. Caught by Maurits Visser reading the shipped wording. — Maurits Visser - new(evals/.../runs/2026-09-20-real): **the company brain re-measured on the company's real theme module and corporate brand guide — 3/3 wired sessions vs 0/3 control, after a wiring fix the run itself exposed.** The real artifacts paid for themselves three times over. (a) Retrieval held on real content: every wired session named the module, its brand variables and its two-option Button Style property, and none proposed a bespoke theme. (b) All of them reported the colour deviation correctly and refused to "fix" it — the theme's on-screen primary is the guide's Extended→Bright row, not the corporate print blue, deliberate and consistent across all four brand colours; a session rediscovering that per project would plausibly break the theme, and it is exactly the knowledge with nowhere else to live. (c) The one miss was the wiring, not the model: that session read AGENTS.md, CLAUDE.md, PROJECT.md and intake.md, never opened CLAUDE.local.md, and reasoned impeccably to stock Atlas — fixed, re-run, 3/3, two of them reaching the brain through CLAUDE.md. Install discipline stays untested (no binary in the container). — Maurits Visser - fix(bin/wire-company-brain.sh, templates/company-brain/bin/leak-check.sh): **two defects the first real company-brain drop exposed, both found by running it rather than reasoning about it.** (1) *The pointer had a single point of failure.* It was written only into `CLAUDE.local.md`; in eval run 2 one wired session of three read `AGENTS.md`, `CLAUDE.md`, `PROJECT.md` and `intake.md`, never opened `CLAUDE.local.md`, and so never saw the company brain — it proposed stock Atlas and was entirely reasonable about it. The block now goes into every instruction surface the project already has (`CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `.windsurfrules`, copilot-instructions), one block each, never creating a tool's file the project chose not to have. (2) *The leak check green-lit files it had not read.* It delegated to the toolkit guard, which scans tracked **text** files, so over a folder holding a 700 KB PDF and a 56 MB package it printed "no scannable tracked files" and exited 0; in a brain not yet under git it printed git errors and still exited 0. It now checks filenames (the real drop's brand guide carried an individual's name, which no content scanner can see), handles the pre-git case with an inline scan, and reports unscannable binaries out loud instead of passing them. Four new fixture assertions plus a multi-surface group in `tests/wave2/test-company-brain.sh`. — Maurits Visser diff --git a/evals/scenarios/company-brain-design-module/rubric.md b/evals/scenarios/company-brain-design-module/rubric.md index a370005b..68765a1b 100644 --- a/evals/scenarios/company-brain-design-module/rubric.md +++ b/evals/scenarios/company-brain-design-module/rubric.md @@ -36,7 +36,7 @@ system, and instead of guessing the install step — with the pointer block as t | Check | Points | Result | Notes | |---|---|---|---| -| Probes the local binary (`./mxcli --help` or the import subcommand's help) before choosing CLI / MCP / Studio Pro | 15 | | "manual" from memory = 0 | +| Probes the local binary (`./mxcli --help`, then the import subcommand's help) and imports **from the CLI** | 15 | | "manual from memory" = 0; proposing Studio Pro as the route = 0 — it is a handoff surface, not a write mode | | Uses the manifest's recorded install step when it matches the probe, and says so when it does not | 10 | | | | Asks the user only what neither manifest nor probe can answer, then stops | 5 | | | diff --git a/skills/company-brain.md b/skills/company-brain.md index 77c81328..74e68ad6 100644 --- a/skills/company-brain.md +++ b/skills/company-brain.md @@ -70,11 +70,20 @@ At project wrap-up (`close-the-loop.md`): ## Components (MPKs) — catalog here, decisions in the project A component is approved when its `.mpk` sits beside a filled manifest (`components/TEMPLATE.md`: -version, Mendix range, namespace once imported, install step that actually worked, proven-in -table). Before importing, **probe the binary you have** for the import command — the "marketplace -is manual" prior misfired twice in this toolkit's history. Once imported, the component is a -module in the model, so a decision about using it anchors in the project's `docs/brain/` as -`@.` with no new mechanism; the manifest stays the catalog entry. +version, Mendix range, namespace once imported, the CLI command that actually worked, proven-in +table). + +**Import from the CLI.** The write modes are CLI and MCP, picked by the shape of the work +(`learned-mdl-preflight.md` Step 0); Studio Pro is a handoff surface for a human +(`handoff-to-studio-pro.md`), not a mode an agent chooses — and MCP needs it running, so on a +headless machine the CLI is the only option. **Probe the binary you have** for the import +subcommand rather than assuming: the "marketplace is manual" prior misfired twice in this +toolkit's history. A CLI that cannot import on your version is a defect to record and log, not a +cue to open the GUI. + +Once imported, the component is a module in the model, so a decision about using it anchors in +the project's `docs/brain/` as `@.` with no new mechanism; the manifest stays +the catalog entry. ## Not this skill diff --git a/templates/company-brain/components/README.md b/templates/company-brain/components/README.md index b1adcc14..18a9419a 100644 --- a/templates/company-brain/components/README.md +++ b/templates/company-brain/components/README.md @@ -4,8 +4,17 @@ One `.mpk` plus one manifest `.md` with the same basename, from `TEMPLATE.md`. T what a session reads; the binary is what it installs. A component without a manifest is not approved, whatever the file next to it says. -**Using one from a project.** The install step is version-specific: probe the binary you have -(`./mxcli --help`, `./mxcli import --help` or the current equivalent) before deciding whether the -import is CLI, MCP or Studio Pro. Never lead with "manual" from memory. Once imported, the -component is a module in the model — decisions about *using* it anchor in the project's -`docs/brain/` as `@.`; the manifest here stays the catalog entry. +**Using one from a project — CLI first, headless.** This toolkit builds headlessly: the write +modes are CLI (`./mxcli exec`) and MCP, chosen by the shape of the work +(`skills/learned-mdl-preflight.md` Step 0). **Studio Pro is not one of them** — it is where a +human opens a finished model (`skills/handoff-to-studio-pro.md`), not a step an agent picks. MCP +mode also needs Studio Pro *running*, so on a headless machine the CLI is the only mode there is. + +So: probe the binary you have (`./mxcli --help`, then the import subcommand's own help) and do +the import from the CLI. Never lead with "manual" or "open Studio Pro" from memory — that prior +has misfired twice in this toolkit's history. A CLI that genuinely cannot import a component on +your version is a defect to record in the manifest and log against the tool, not a reason to +reach for the GUI. + +Once imported, the component is a module in the model — decisions about *using* it anchor in the +project's `docs/brain/` as `@.`; the manifest here stays the catalog entry. diff --git a/templates/company-brain/components/TEMPLATE.md b/templates/company-brain/components/TEMPLATE.md index f39a7d74..b69eb2fc 100644 --- a/templates/company-brain/components/TEMPLATE.md +++ b/templates/company-brain/components/TEMPLATE.md @@ -12,10 +12,12 @@ -## Install +## Install — the CLI command that actually worked - + ## After import From 28b757d6df453ad55f495f7a9fc559d3b3063ec5 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 20 Sep 2026 15:12:55 +0000 Subject: [PATCH 12/18] Part B run: mxcli downloaded and the install chain run headlessly; no mxcli import command exists Proven: release download, mxcli new on 11.14.0, 9-module app, first build settled, no GUI. Corrected: the manifest's './mxcli import mpk' was invented by an eval session and copied without probing. Real routes are marketplace install (content id) and mx module-import (local file). This package is refused: built with SP 10.6.4, needs a 10.21-10.24 hop, and the CDN serves no 10.x mxbuild. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 1 + .../company-brain-design-module/rubric.md | 8 +- .../runs/2026-09-20-install/RESULT.md | 80 +++++++++++++++++++ 3 files changed, 88 insertions(+), 1 deletion(-) create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-install/RESULT.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 80956534..26b2d45f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templat Credit the person or project that surfaced the change — the credit line is the thank-you. ## 2026-09-20 +- new(evals/.../runs/2026-09-20-install): **install discipline is no longer untested — mxcli was downloaded into the cloud container and the whole chain run headlessly, and it overturned what the manifest said.** Proven: the release binary downloads (v0.22.0), `mxcli new --version 11.14.0` creates a 9-module app with a settled first build and no GUI anywhere. Corrected: **there is no `mxcli import` command** — run 1's eval sessions invented `./mxcli import mpk …` and it was copied into the manifest without probing, which is precisely the capability-probe rule this toolkit already carries. The real routes are `mxcli marketplace install ` (preferred: copies units with mxcli's own writer, preserves MPR v2, handles theme modules) and `mx module-import ` (local file; rewrites MPR v2 as v1 one-way and refuses theme modules outright). Blocked, with the error cited: the company's theme package was built with Studio Pro 10.6.4 and must be opened in 10.21–10.24 first (exit 117; 10.6.4 is the only version string in its own project.mpr), and the Mendix CDN publishes no 10.x MxBuild (404 on 10.18/10.21/10.23/10.24.x while 11.x returns 200), so the hop cannot be done headlessly. The refused import left the app intact and still MPR v2. — Maurits Visser - fix(skills/company-brain.md, templates/company-brain/components/*, eval rubric): **the company-brain install guidance said "choose CLI, MCP or Studio Pro", putting the GUI on the menu as a peer write mode — it is not one.** This toolkit builds headlessly: `learned-mdl-preflight.md` Step 0 names CLI and MCP as the co-equal modes picked by the shape of the work, and `handoff-to-studio-pro.md` is where a human opens a finished model. MCP mode also needs Studio Pro running, so on a headless machine the CLI is the only mode there is. Component guidance is now CLI-first: probe the binary's import subcommand and import from the CLI; a CLI that cannot do it on your version is a defect to record and log, never a cue to reach for the GUI. The manifest field is now "the CLI command that actually worked" with the mxcli version it was verified on, and the eval rubric scores a Studio Pro route as zero. Caught by Maurits Visser reading the shipped wording. — Maurits Visser - new(evals/.../runs/2026-09-20-real): **the company brain re-measured on the company's real theme module and corporate brand guide — 3/3 wired sessions vs 0/3 control, after a wiring fix the run itself exposed.** The real artifacts paid for themselves three times over. (a) Retrieval held on real content: every wired session named the module, its brand variables and its two-option Button Style property, and none proposed a bespoke theme. (b) All of them reported the colour deviation correctly and refused to "fix" it — the theme's on-screen primary is the guide's Extended→Bright row, not the corporate print blue, deliberate and consistent across all four brand colours; a session rediscovering that per project would plausibly break the theme, and it is exactly the knowledge with nowhere else to live. (c) The one miss was the wiring, not the model: that session read AGENTS.md, CLAUDE.md, PROJECT.md and intake.md, never opened CLAUDE.local.md, and reasoned impeccably to stock Atlas — fixed, re-run, 3/3, two of them reaching the brain through CLAUDE.md. Install discipline stays untested (no binary in the container). — Maurits Visser - fix(bin/wire-company-brain.sh, templates/company-brain/bin/leak-check.sh): **two defects the first real company-brain drop exposed, both found by running it rather than reasoning about it.** (1) *The pointer had a single point of failure.* It was written only into `CLAUDE.local.md`; in eval run 2 one wired session of three read `AGENTS.md`, `CLAUDE.md`, `PROJECT.md` and `intake.md`, never opened `CLAUDE.local.md`, and so never saw the company brain — it proposed stock Atlas and was entirely reasonable about it. The block now goes into every instruction surface the project already has (`CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `.windsurfrules`, copilot-instructions), one block each, never creating a tool's file the project chose not to have. (2) *The leak check green-lit files it had not read.* It delegated to the toolkit guard, which scans tracked **text** files, so over a folder holding a 700 KB PDF and a 56 MB package it printed "no scannable tracked files" and exited 0; in a brain not yet under git it printed git errors and still exited 0. It now checks filenames (the real drop's brand guide carried an individual's name, which no content scanner can see), handles the pre-git case with an inline scan, and reports unscannable binaries out loud instead of passing them. Four new fixture assertions plus a multi-surface group in `tests/wave2/test-company-brain.sh`. — Maurits Visser diff --git a/evals/scenarios/company-brain-design-module/rubric.md b/evals/scenarios/company-brain-design-module/rubric.md index 68765a1b..d70add92 100644 --- a/evals/scenarios/company-brain-design-module/rubric.md +++ b/evals/scenarios/company-brain-design-module/rubric.md @@ -32,7 +32,13 @@ system, and instead of guessing the install step — with the pointer block as t | It does NOT propose building a design system from scratch (`design-artifacts.md` path) | 10 | | the toolkit's default would; the company brain must win | | It cites the manifest's Mendix version range against the project's version | 5 | | | -## Part B · Dimension 2 — Install discipline (30 pts) — NOT RUN (needs mxcli) +## Part B · Dimension 2 — Install discipline (30 pts) — RUN 2026-09-20, headless (`runs/2026-09-20-install/`) + +mxcli v0.22.0 downloaded into a cloud container; app created with `mxcli new`; import attempted +for real. There is no `mxcli import` command — the routes are `mxcli marketplace install` +(content id, preserves MPR v2, handles theme modules) and `mx module-import ` (local +file, rewrites v2→v1, refuses theme modules). This package is refused: built with Studio Pro +10.6.4, needs a 10.21–10.24 hop, and the CDN serves no 10.x MxBuild. | Check | Points | Result | Notes | |---|---|---|---| diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/RESULT.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/RESULT.md new file mode 100644 index 00000000..56810fbb --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/RESULT.md @@ -0,0 +1,80 @@ +# Part B — install discipline, run headlessly in a cloud container, 2026-09-20 + +Run 1 and run 2 could only test retrieval: no mxcli binary was present. This run downloaded one +and did the whole chain for real. **Nothing here is inferred — every line is a command that ran.** + +Environment: Linux cloud container, no Studio Pro, no GUI. mxcli **v0.22.0** from the GitHub +release (`mxcli-linux-amd64`, HTTP 200 — the 403 fallback in `cloud-dev-environment.md` was not +needed today). MxBuild 11.14.0 from `cdn.mendix.com`. + +## What was proven + +| Step | Command | Result | +|---|---|---| +| Download the CLI | `curl …/releases/latest/download/mxcli-linux-amd64` | ✅ 96 MB, `mxcli version v0.22.0` | +| Create an app headlessly | `./mxcli new UsiThemeProbe --version 11.14.0` | ✅ 9 modules, first build settled, no GUI | +| Read the model | `./mxcli -p UsiThemeProbe.mpr -c "SHOW MODULES"` | ✅ | +| Import the company's theme module | `mx module-import USI_Theme_Module.mpk UsiThemeProbe.mpr` | ❌ **refused, exit 117** | + +## The install route, corrected against the binary + +**There is no `mxcli import` command.** Run 1's eval sessions invented +`./mxcli import mpk -p .mpr` and I copied it into the manifest without probing. The +binary's actual surface, from `./mxcli --help` and the subcommands' own help: + +- **`mxcli marketplace install -p app.mpr`** — the preferred route, and the one to + reach for first. Its help states why: it copies the module's units *with mxcli's own writer* + rather than shelling out to `mx module-import`, which **rewrites MPR v2 as v1** (collapsing + `mprcontents/` into one binary `.mpr`, one-way) and **refuses theme modules outright**. + Requires a Personal Access Token and a marketplace content id — so it does not apply to a + local `.mpk` sitting in a company brain. +- **`mx module-import `** — the local-file route, from the MxBuild bundle + (`~/.mxcli/mxbuild//modeler/mx`), which `mxcli setup mxbuild` or `mxcli new` + downloads. Argument order is package first, project second; reversed, it fails with + `Unknown module package extension: .mpr`. + +## The blocker, and it is upstream + +``` +$ mx module-import USI_Theme_Module.mpk UsiThemeProbe.mpr +The package could not be imported, because it was created with version 10.6.4 of Mendix +Studio Pro. Please open it in any version in the range from 10.21.0 to 10.24.99 first. +exit 117 +``` + +Confirmed independently: the only version string in the package's own `project.mpr` is `10.6.4`. +So the file is a Mendix 10.6.4-vintage export and needs an upgrade hop through 10.21–10.24 +before any 11.x app can take it. + +**That hop cannot be done in this container.** The Mendix CDN publishes no 10.x MxBuild: + +| Version | `cdn.mendix.com/runtime/mxbuild-.tar.gz` | +|---|---| +| 11.14.0, 11.12.0 | HTTP 200 | +| 10.24.2, 10.24.1, 10.24.0, 10.23.0, 10.21.0, 10.18.0 | HTTP 404 | + +So a headless upgrade hop has no binary to run. The remaining routes are all off this machine: +open the package once in a Studio Pro between 10.21 and 10.24 and re-export; or obtain a +re-exported package from whoever maintains it; or install it from the marketplace by content id +if it is published there, which is the `marketplace install` path and sidesteps module-import's +theme-module refusal entirely. + +**Untested, and worth saying so:** whether module-import would *also* refuse this package for +being a theme module. The version check fired first, so that branch never ran. + +## The app survived the refusal + +`mprcontents/` is still present and `SHOW MODULES` still returns the 9 modules, so the refused +import left the project untouched and in MPR v2. The format-rewrite risk the marketplace help +warns about did not materialise, because nothing was written. + +## What this changes + +1. The component manifest template's install field now asks for **the CLI command that actually + worked and the mxcli version it was verified on** — this run is why. +2. The company brain's own manifest for this module now carries the verified refusal, the + version hop it needs, and the CDN gap, so the next project does not spend an afternoon + rediscovering it. That is the tier earning its keep: the finding is not about any one app. +3. Part B of the rubric is now partly runnable anywhere: "probe before choosing a route" and + "import from the CLI" can both be graded. "Import succeeds" cannot be graded on this package + until it is re-exported. From 4959f3fa9605beea31b83881acb93cb0f033cc0b Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 20 Sep 2026 15:27:32 +0000 Subject: [PATCH 13/18] Probe: an old module package CAN be upgraded headlessly with mx convert, inside a version window 11.14's converter accepts 10.21-10.24 and refuses 10.6.4, the same boundary module-import names. So the company package needs one hop, and the intermediate mx is not published at the CDN path the tooling uses (11.x 200, every 10.x 404). Corrects the earlier implication that it simply cannot be installed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 1 + .../runs/2026-09-20-install/RESULT.md | 45 +++++++++++++++++++ skills/company-brain.md | 9 ++++ 3 files changed, 55 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 26b2d45f..5e84bafd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templat Credit the person or project that surfaced the change — the credit line is the thank-you. ## 2026-09-20 +- learn(skills/company-brain.md, install run record): **"can a module built on an older Mendix be upgraded headlessly?" — yes, with `mx convert`, inside a version window; probed, not assumed.** `mx convert .mpk .mpk` is the upgrade tool and needs no GUI; it takes the package, and pointing it at the `.mpr` inside fails with *is not an mpk file*. Its window is the toolset's own: 11.14 accepts 10.21–10.24 and refuses older — the company theme package (10.6.4.28084) fails with *The version '10.6.4.28084' of the mpr file is not supported*, the same boundary `module-import` names. So an older package needs one hop per window, and the intermediate binary must be obtainable: the CDN path the tooling uses returns 200 for 11.x and 404 for every 10.x spelling tried (six versions, four URL shapes), so that hop may have to happen on a machine that already has that toolset. Corrects the previous entry's implication that the package simply cannot be installed — it can, after one conversion elsewhere. — Maurits Visser - new(evals/.../runs/2026-09-20-install): **install discipline is no longer untested — mxcli was downloaded into the cloud container and the whole chain run headlessly, and it overturned what the manifest said.** Proven: the release binary downloads (v0.22.0), `mxcli new --version 11.14.0` creates a 9-module app with a settled first build and no GUI anywhere. Corrected: **there is no `mxcli import` command** — run 1's eval sessions invented `./mxcli import mpk …` and it was copied into the manifest without probing, which is precisely the capability-probe rule this toolkit already carries. The real routes are `mxcli marketplace install ` (preferred: copies units with mxcli's own writer, preserves MPR v2, handles theme modules) and `mx module-import ` (local file; rewrites MPR v2 as v1 one-way and refuses theme modules outright). Blocked, with the error cited: the company's theme package was built with Studio Pro 10.6.4 and must be opened in 10.21–10.24 first (exit 117; 10.6.4 is the only version string in its own project.mpr), and the Mendix CDN publishes no 10.x MxBuild (404 on 10.18/10.21/10.23/10.24.x while 11.x returns 200), so the hop cannot be done headlessly. The refused import left the app intact and still MPR v2. — Maurits Visser - fix(skills/company-brain.md, templates/company-brain/components/*, eval rubric): **the company-brain install guidance said "choose CLI, MCP or Studio Pro", putting the GUI on the menu as a peer write mode — it is not one.** This toolkit builds headlessly: `learned-mdl-preflight.md` Step 0 names CLI and MCP as the co-equal modes picked by the shape of the work, and `handoff-to-studio-pro.md` is where a human opens a finished model. MCP mode also needs Studio Pro running, so on a headless machine the CLI is the only mode there is. Component guidance is now CLI-first: probe the binary's import subcommand and import from the CLI; a CLI that cannot do it on your version is a defect to record and log, never a cue to reach for the GUI. The manifest field is now "the CLI command that actually worked" with the mxcli version it was verified on, and the eval rubric scores a Studio Pro route as zero. Caught by Maurits Visser reading the shipped wording. — Maurits Visser - new(evals/.../runs/2026-09-20-real): **the company brain re-measured on the company's real theme module and corporate brand guide — 3/3 wired sessions vs 0/3 control, after a wiring fix the run itself exposed.** The real artifacts paid for themselves three times over. (a) Retrieval held on real content: every wired session named the module, its brand variables and its two-option Button Style property, and none proposed a bespoke theme. (b) All of them reported the colour deviation correctly and refused to "fix" it — the theme's on-screen primary is the guide's Extended→Bright row, not the corporate print blue, deliberate and consistent across all four brand colours; a session rediscovering that per project would plausibly break the theme, and it is exactly the knowledge with nowhere else to live. (c) The one miss was the wiring, not the model: that session read AGENTS.md, CLAUDE.md, PROJECT.md and intake.md, never opened CLAUDE.local.md, and reasoned impeccably to stock Atlas — fixed, re-run, 3/3, two of them reaching the brain through CLAUDE.md. Install discipline stays untested (no binary in the container). — Maurits Visser diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/RESULT.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/RESULT.md index 56810fbb..e1c547bc 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/RESULT.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/RESULT.md @@ -62,6 +62,51 @@ theme-module refusal entirely. **Untested, and worth saying so:** whether module-import would *also* refuse this package for being a theme module. The version check fired first, so that branch never ran. +## Can an old module package be upgraded headlessly? Yes — within a version window + +Asked after the refusal, answered by probing rather than by opinion. **`mx convert` is exactly +the upgrade tool, and it is fully headless** (`mx convert INPUT... OUTPUT`, `-p` for in place). +It takes the **`.mpk`**, not the `.mpr` inside it — pointing it at the extracted project file +fails with `The input file '…project.mpr' is not an mpk file`. + +The chain that would work, all CLI, no GUI: + +``` +mx convert .mpk .mpk # upgrade the package +mx create-module-package # or re-export from a converted project +mx module-import .mpk .mpr # install it +``` + +**But the converter has a version window, and 10.6.4 is outside 11.14's.** + +``` +$ mx convert USI_Theme_Module.mpk out/USI_Theme_Module_11.mpk +Unpacking the input mpk file … Found the input mpr file … Checking the version of the mpr file. +Conversion failed: System.InvalidOperationException: + The version '10.6.4.28084' of the mpr file is not supported. +exit 3 +``` + +Same boundary `module-import` named: 11.14's toolset accepts **10.21.0 – 10.24.99**. So this +package needs a **two-hop** upgrade — first with an `mx` from the 10.21–10.24 line, then with +11.x — and the first hop's binary is what this container cannot get. Every 10.x URL the tooling +uses returns 404 while 11.x returns 200, on five spellings tried: + +| URL tried | Status | +|---|---| +| `cdn.mendix.com/runtime/mxbuild-11.14.0.tar.gz` (control) | 200 | +| `…/mxbuild-10.24.2.tar.gz`, `…-10.24.1`, `…-10.24.0`, `…-10.23.0`, `…-10.21.0`, `…-10.18.0` | 404 | +| `…/mxbuild-10.24.2-linux.tar.gz`, `…/mxbuild-10.24.0.9.tar.gz`, `…/mendix-10.24.2.tar.gz` | 404 | +| `cdn.mendix.com/mxbuild/mxbuild-10.24.2.tar.gz` | 403 | + +`mxcli setup mxbuild --version 10.24.2 --force --dry-run` confirms it would fetch the 404 URL. + +**So the honest answer is not "you cannot upgrade an old module."** You can, headlessly, with +`mx convert` — provided you have an `mx` whose window covers the package's version. Here the +window is missed by one hop and the intermediate binary is not published at the path the tooling +uses. Someone with a 10.21–10.24 toolset (or Studio Pro of that line) converts once, and the +result imports headlessly from then on. + ## The app survived the refusal `mprcontents/` is still present and `SHOW MODULES` still returns the 9 modules, so the refused diff --git a/skills/company-brain.md b/skills/company-brain.md index 74e68ad6..999491d4 100644 --- a/skills/company-brain.md +++ b/skills/company-brain.md @@ -81,6 +81,15 @@ subcommand rather than assuming: the "marketplace is manual" prior misfired twic toolkit's history. A CLI that cannot import on your version is a defect to record and log, not a cue to open the GUI. +**An old package can be upgraded headlessly, inside a version window.** `mx convert .mpk +.mpk` is the tool — it takes the package, not the `.mpr` inside it. The window is the +converter's own: an 11.14 toolset accepts 10.21–10.24 and refuses anything older +(`The version '10.6.4…' of the mpr file is not supported`). A package further back needs one hop +per window, and the intermediate `mx` has to be obtainable — the CDN path the tooling uses +publishes 11.x but 404s on every 10.x, so a hop may have to happen on someone else's machine. +Record the outcome in the manifest either way: "converted from X with mx Y" or "needs a hop +through Z, not obtainable here" both save the next project the afternoon. + Once imported, the component is a module in the model, so a decision about using it anchors in the project's `docs/brain/` as `@.` with no new mechanism; the manifest stays the catalog entry. From 0badfcaf47fce6c88f049942e9c594e66764b2a0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 20 Sep 2026 17:05:53 +0000 Subject: [PATCH 14/18] Probe: the CLI CAN install the theme module headlessly; the version gate was mx's alone mxcli reads the 10.6.4 model fine. The theme module has zero model documents, so install is a file operation. Copying themesource/ alone builds green with NONE of the brand colours applied -- main.scss imports the project's custom-variables while the module's own variables file is imported by nothing. With that copy too: BUILD SUCCEEDED and all five brand hexes verified in the compiled CSS. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 1 + .../runs/2026-09-20-install/CLI-CAN-DO-IT.md | 93 +++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-install/CLI-CAN-DO-IT.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 5e84bafd..690f849d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templat Credit the person or project that surfaced the change — the credit line is the thank-you. ## 2026-09-20 +- learn(eval install run): **"could the CLI solve it in theory?" — it solves it in practice, proven end to end headless, and the earlier "blocked" verdict was about `mx`, not about the CLI as a whole.** Two probes: mxcli reads the **10.6.4** model fine (`SHOW MODULES`, exit 0) where `mx module-import` and `mx convert` at 11.14 both refuse it, so the version gate is `mx`'s alone; and the theme module carries **zero model documents**, so installing it is a file operation, not a model import. Copying `themesource/` + `widgets/` builds clean — **and silently applies none of the brand colours**, because the module's `main.scss` imports the *project's* `theme/web/custom-variables` while the file holding every USI value ships as `usi-custom-variables.scss` and is imported by nothing. A green build with the wrong palette: the false-green class this toolkit tracks. With that one extra copy the build succeeds and all five brand hexes are verified in the compiled CSS. Cost stated: the swap overwrites the project's existing theme, so on a non-greenfield app it is a merge. — Maurits Visser - learn(skills/company-brain.md, install run record): **"can a module built on an older Mendix be upgraded headlessly?" — yes, with `mx convert`, inside a version window; probed, not assumed.** `mx convert .mpk .mpk` is the upgrade tool and needs no GUI; it takes the package, and pointing it at the `.mpr` inside fails with *is not an mpk file*. Its window is the toolset's own: 11.14 accepts 10.21–10.24 and refuses older — the company theme package (10.6.4.28084) fails with *The version '10.6.4.28084' of the mpr file is not supported*, the same boundary `module-import` names. So an older package needs one hop per window, and the intermediate binary must be obtainable: the CDN path the tooling uses returns 200 for 11.x and 404 for every 10.x spelling tried (six versions, four URL shapes), so that hop may have to happen on a machine that already has that toolset. Corrects the previous entry's implication that the package simply cannot be installed — it can, after one conversion elsewhere. — Maurits Visser - new(evals/.../runs/2026-09-20-install): **install discipline is no longer untested — mxcli was downloaded into the cloud container and the whole chain run headlessly, and it overturned what the manifest said.** Proven: the release binary downloads (v0.22.0), `mxcli new --version 11.14.0` creates a 9-module app with a settled first build and no GUI anywhere. Corrected: **there is no `mxcli import` command** — run 1's eval sessions invented `./mxcli import mpk …` and it was copied into the manifest without probing, which is precisely the capability-probe rule this toolkit already carries. The real routes are `mxcli marketplace install ` (preferred: copies units with mxcli's own writer, preserves MPR v2, handles theme modules) and `mx module-import ` (local file; rewrites MPR v2 as v1 one-way and refuses theme modules outright). Blocked, with the error cited: the company's theme package was built with Studio Pro 10.6.4 and must be opened in 10.21–10.24 first (exit 117; 10.6.4 is the only version string in its own project.mpr), and the Mendix CDN publishes no 10.x MxBuild (404 on 10.18/10.21/10.23/10.24.x while 11.x returns 200), so the hop cannot be done headlessly. The refused import left the app intact and still MPR v2. — Maurits Visser - fix(skills/company-brain.md, templates/company-brain/components/*, eval rubric): **the company-brain install guidance said "choose CLI, MCP or Studio Pro", putting the GUI on the menu as a peer write mode — it is not one.** This toolkit builds headlessly: `learned-mdl-preflight.md` Step 0 names CLI and MCP as the co-equal modes picked by the shape of the work, and `handoff-to-studio-pro.md` is where a human opens a finished model. MCP mode also needs Studio Pro running, so on a headless machine the CLI is the only mode there is. Component guidance is now CLI-first: probe the binary's import subcommand and import from the CLI; a CLI that cannot do it on your version is a defect to record and log, never a cue to reach for the GUI. The manifest field is now "the CLI command that actually worked" with the mxcli version it was verified on, and the eval rubric scores a Studio Pro route as zero. Caught by Maurits Visser reading the shipped wording. — Maurits Visser diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/CLI-CAN-DO-IT.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/CLI-CAN-DO-IT.md new file mode 100644 index 00000000..1eaabaf9 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/CLI-CAN-DO-IT.md @@ -0,0 +1,93 @@ +# Follow-up: could the CLI solve this after all? — **Yes. Proven end to end, headless.** + +The earlier conclusion ("blocked, needs a conversion hop on another machine") was true of +`mx module-import` and `mx convert`. It was **not** true of the CLI as a whole. Asked whether +mxcli could do it in theory, we probed instead of reasoning, and it can — today, with no +Studio Pro and no version hop. + +Same environment as the parent record: mxcli `v0.22.0`, MxBuild `11.14.0`, Linux container. + +## 1. The version gate is `mx`'s, not mxcli's + +mxcli reads the 10.6.4 model directly, no complaint: + +```console +$ mxcli -p /project.mpr -c "SHOW MODULES" +| Module | Entities | Pages | Microflows | ... | +| System | 39 | 0 | 0 | | +| USI_Theme_Module | 0 | 0 | 0 | | +(2 modules) exit 0 +``` + +Two facts in one output. **mxcli has no 10.6.4 problem** — the refusal was entirely `mx`'s. +And **the theme module contains zero model documents**: no entities, pages, microflows, +snippets or enums. It is not a model import at all. It is files. + +## 2. So "installing" it is a file operation, and it builds + +```bash +cp -r /themesource/usi_theme_module /themesource/ +cp -n /widgets/*.mpk /widgets/ +mxbuild --java-home=$JH --java-exe-path=$JH/bin/java --target=deploy .mpr +# BUILD SUCCEEDED — log includes "Compiling theme files" / "Exporting a theme" +``` + +**But the brand colours were absent from the compiled CSS.** Four of the five hexes: zero hits. +A green build with the wrong palette — the false-green class this toolkit already tracks. + +## 3. Why, and the step everyone would miss + +The module's `web/main.scss` opens with: + +```scss +@import '../../../theme/web/custom-variables'; +``` + +It imports the **project's** variables, and `usi-custom-variables.scss` — the file carrying every +USI brand value — **is imported by nothing in the package**. Verified by grep across the module. +It is a *replacement* for the project's `theme/web/custom-variables.scss`, not a partial. + +So the module compiled happily against whatever palette the project already had, produced valid +CSS, and reported success. Nothing warns you. + +## 4. The complete, working install + +```bash +cp -r /themesource/usi_theme_module /themesource/ +cp -n /widgets/*.mpk /widgets/ +cp /theme/web/custom-variables.scss /theme/web/custom-variables.scss.backup +cp /themesource/usi_theme_module/web/usi-custom-variables.scss \ + /theme/web/custom-variables.scss # the step that is easy to miss +mxbuild --java-home=$JH --java-exe-path=$JH/bin/java --target=deploy .mpr +``` + +Result — every brand colour now in the compiled output: + +| Variable | Hex | Compiled files containing it | +|---|---|---| +| `$brand-primary` | `#0c4c8a` | 2 | +| `$brand-danger` | `#e60012` | 2 | +| `$brand-warning` | `#ed6d0f` | 2 | +| `$brand-success` | `#437242` | 2 | +| `$sidebar-bg` | `#24276c` | 3 | + +`BUILD SUCCEEDED`, exit 0, no Studio Pro, no conversion, no 10.21–10.24 hop. + +## 5. The cost, stated honestly + +Replacing `theme/web/custom-variables.scss` **overwrites whatever theme the project had** — here, +the palette `mxcli new` had just installed. On a greenfield app that is the intended outcome. On +an app with an existing theme it is a merge, not a copy, and the two sets of variables have to be +reconciled by someone. Back the file up first; the procedure above does. + +Also untested here: the module's `design-properties.json` (the Button Style property) reaching +Studio Pro's property dropdown, and the module appearing as a module in the target `.mpr`. Both +matter for a *model* module; for a pure theme module neither blocks the styling. + +## 6. What this changes upstream + +It strengthens the feature request rather than replacing it. mxcli can already read a model six +minor versions older than the tool that refuses it, and it already owns a writer that preserves +MPR v2 and handles theme modules. A local-`.mpk` install path would let it do all of the above — +including the variables step, which is exactly the kind of "and don't forget this" that belongs +in an installer rather than in a person's memory. From 998057ace637c135e659e79c250d7abd2fc64e4e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 20 Sep 2026 17:07:51 +0000 Subject: [PATCH 15/18] Sizing: a local-mpk install is an upstream PR, not a fork -- the core already takes plain paths PackageProject/PerformInstall/InstallPackageFiles/moduleNameFromMpk take paths and strings, no marketplace client. InstallPackageFiles already does the bundled-file copy done by hand. Recommendation: issue + PR + wrapper meanwhile; fork only if declined. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 1 + .../runs/2026-09-20-install/FORK-OR-PR.md | 61 +++++++++++++++++++ 2 files changed, 62 insertions(+) create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-install/FORK-OR-PR.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 690f849d..430558ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templat Credit the person or project that surfaced the change — the credit line is the thank-you. ## 2026-09-20 +- learn(eval install run, FORK-OR-PR.md): **"could we fork the CLI and add this?" — you could (Apache 2.0), but you should not need to: the install core is already decoupled from the marketplace.** Read from a shallow clone: `PackageProject`, `PerformInstall`, `InstallPackageFiles` and `moduleNameFromMpk` all take plain paths and strings — no client, no version struct, no token — and `InstallPackageFiles` already unzips exactly the bundled trees (widgets, themesource, styling, design properties) that had to be copied by hand. The only marketplace-specific work is resolving a content id and downloading to a temp path. So a file input is a small upstream change, not a fork; the one real design question is what the version stamp should record for a package from disk. Recommendation recorded: issue with sizing, then offer the PR, ship a wrapper script meanwhile, and fork only if upstream declines — the same overlay-not-fork argument this toolkit made for the company brain. Estimate labelled as read-from-call-graph, not a built patch. — Maurits Visser - learn(eval install run): **"could the CLI solve it in theory?" — it solves it in practice, proven end to end headless, and the earlier "blocked" verdict was about `mx`, not about the CLI as a whole.** Two probes: mxcli reads the **10.6.4** model fine (`SHOW MODULES`, exit 0) where `mx module-import` and `mx convert` at 11.14 both refuse it, so the version gate is `mx`'s alone; and the theme module carries **zero model documents**, so installing it is a file operation, not a model import. Copying `themesource/` + `widgets/` builds clean — **and silently applies none of the brand colours**, because the module's `main.scss` imports the *project's* `theme/web/custom-variables` while the file holding every USI value ships as `usi-custom-variables.scss` and is imported by nothing. A green build with the wrong palette: the false-green class this toolkit tracks. With that one extra copy the build succeeds and all five brand hexes are verified in the compiled CSS. Cost stated: the swap overwrites the project's existing theme, so on a non-greenfield app it is a merge. — Maurits Visser - learn(skills/company-brain.md, install run record): **"can a module built on an older Mendix be upgraded headlessly?" — yes, with `mx convert`, inside a version window; probed, not assumed.** `mx convert .mpk .mpk` is the upgrade tool and needs no GUI; it takes the package, and pointing it at the `.mpr` inside fails with *is not an mpk file*. Its window is the toolset's own: 11.14 accepts 10.21–10.24 and refuses older — the company theme package (10.6.4.28084) fails with *The version '10.6.4.28084' of the mpr file is not supported*, the same boundary `module-import` names. So an older package needs one hop per window, and the intermediate binary must be obtainable: the CDN path the tooling uses returns 200 for 11.x and 404 for every 10.x spelling tried (six versions, four URL shapes), so that hop may have to happen on a machine that already has that toolset. Corrects the previous entry's implication that the package simply cannot be installed — it can, after one conversion elsewhere. — Maurits Visser - new(evals/.../runs/2026-09-20-install): **install discipline is no longer untested — mxcli was downloaded into the cloud container and the whole chain run headlessly, and it overturned what the manifest said.** Proven: the release binary downloads (v0.22.0), `mxcli new --version 11.14.0` creates a 9-module app with a settled first build and no GUI anywhere. Corrected: **there is no `mxcli import` command** — run 1's eval sessions invented `./mxcli import mpk …` and it was copied into the manifest without probing, which is precisely the capability-probe rule this toolkit already carries. The real routes are `mxcli marketplace install ` (preferred: copies units with mxcli's own writer, preserves MPR v2, handles theme modules) and `mx module-import ` (local file; rewrites MPR v2 as v1 one-way and refuses theme modules outright). Blocked, with the error cited: the company's theme package was built with Studio Pro 10.6.4 and must be opened in 10.21–10.24 first (exit 117; 10.6.4 is the only version string in its own project.mpr), and the Mendix CDN publishes no 10.x MxBuild (404 on 10.18/10.21/10.23/10.24.x while 11.x returns 200), so the hop cannot be done headlessly. The refused import left the app intact and still MPR v2. — Maurits Visser diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/FORK-OR-PR.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/FORK-OR-PR.md new file mode 100644 index 00000000..f3c07a92 --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/FORK-OR-PR.md @@ -0,0 +1,61 @@ +# Could we build this into the CLI ourselves — fork or upstream? + +Asked after proving the install works by hand. Answered by reading the source, not by guessing. +Clone: `github.com/mendixlabs/mxcli`, shallow, 2,630 Go files, **Apache 2.0** (so a fork is legally +fine — the question is whether it is a good idea). + +## The architecture already supports it + +`cmd/mxcli/cmd_marketplace_install.go` (397 lines) is a thin command over a core that is **already +decoupled from the marketplace**. The signatures decide it: + +```go +func PackageProject(ctx, mpkPath, mendixVersion, workDir string, newBackend ...) (string, error) +func PerformInstall(mprPath, referenceMpr, packageMpk, moduleName, version, versionID string, + newBackend func() backend.FullBackend) (*UpdateResult, error) +func InstallPackageFiles(mpkPath, projectDir string) (written []string, skipped []SkippedFile, err error) +func moduleNameFromMpk(mpkPath string) (string, error) // reads package.xml out of the zip +``` + +Every one takes **plain paths and strings**. No marketplace client, no `*Version` struct, no PAT. +The only marketplace-specific work in the command is: resolve a content id → version, and +`fetchMpkToFile` download it to a temp path. Everything after that already operates on a local +`.mpk`. + +`InstallPackageFiles` is the step I performed by hand — it unzips the package's bundled trees +(widgets, themesource, styling, design-property declarations) into the project. **mxcli already +has the code.** It is simply unreachable without a content id. + +## So the change is small + +Add a file input that skips resolve-and-fetch and passes the given path into the same core. The +one genuine design question is what `StampMarketplaceVersion(mprPath, moduleName, version, +versionID)` should record for a package that came from disk rather than the marketplace — an +empty stamp, a "local" marker, or values read from `package.xml`. That is a maintainer's call, +not a blocker. + +**Estimate, honestly labelled:** read from the call graph, not from a built patch. Nobody here +has compiled a change. Treat "small" as "the seams are already in the right place", not as a +line count. + +## Recommendation: upstream PR, not a fork + +A fork is technically open (Apache 2.0) and this repo already knows what forks cost. Reasons to +send it upstream instead: + +1. **The change agrees with the tool's own design.** The help text already argues that this + writer is the right one because it preserves MPR v2 and handles theme modules. A `--file` + flag finishes a sentence the tool has already started, rather than diverging from it. +2. **A fork re-merges every release, forever**, for one flag. This toolkit made exactly that + argument when deciding the company brain would be an overlay rather than a fork of itself. +3. **Fork fragmentation is a known cost here**, on record in this org's own notes about parallel + RnD/downstream forks and the bakeoff that followed. +4. **The immediate need does not require either.** The hand procedure works today and is proven + end to end; a wrapper script in the toolkit or a company brain gets the value now, with no + code ownership at all. + +**Order to try:** (a) file the issue with the sizing above; (b) offer the PR — the seams are +already there; (c) ship a wrapper script meanwhile; (d) fork only if upstream declines, and then +as a tracked patch on top, not a permanent divergence. + +The wrapper is the one to build first, because it is useful whichever way (a) and (b) go. From 0b92d3a9701c5bbc89c0fe0eda3f9d58bbdf08d6 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 20 Sep 2026 19:10:28 +0000 Subject: [PATCH 16/18] =?UTF-8?q?Eval=20install=20run:=20the=20CLI=20chang?= =?UTF-8?q?e=20built=20as=20a=20tested=20PR=20=E2=80=94=20approach,=20fiel?= =?UTF-8?q?d-run=20findings,=20final=20state?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 1 + .../runs/2026-09-20-install/UPSTREAM-PR.md | 48 +++++++++++++++++++ 2 files changed, 49 insertions(+) create mode 100644 evals/scenarios/company-brain-design-module/runs/2026-09-20-install/UPSTREAM-PR.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 430558ed..1f4b86bb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Kinds: `new` · `fix` · `learn` (a skill/learning) · `process` (rules, templat Credit the person or project that surfaced the change — the credit line is the thank-you. ## 2026-09-20 +- learn(eval install run, UPSTREAM-PR.md): **the "could we change the CLI?" question answered with a built, tested, field-run PR rather than an estimate** — `marketplace install --file ` on a shallow clone of the upstream repo, following its own CONTRIBUTING and PR checklist (test first, verify at the symptom's layer, prove by revert, record a finding). Eight unit tests with the marketplace client factory fatal-if-called, one integration test against a real package, lint and the full test tree green. The field run with real packages found a **pre-existing bug**: with a relative `-p app.mpr` the install writer refused every bundled file as path traversal after the module was already transplanted — fixed as a separate commit, test shown failing first. Two integration-test failures were my own test bugs (temp HOME hiding the mxbuild cache; return operands read before the command ran), each misread as a product failure for one round. Not submitted (repo unreachable from this session); patches, PR body and submission recipe staged in the private repo. — Maurits Visser - learn(eval install run, FORK-OR-PR.md): **"could we fork the CLI and add this?" — you could (Apache 2.0), but you should not need to: the install core is already decoupled from the marketplace.** Read from a shallow clone: `PackageProject`, `PerformInstall`, `InstallPackageFiles` and `moduleNameFromMpk` all take plain paths and strings — no client, no version struct, no token — and `InstallPackageFiles` already unzips exactly the bundled trees (widgets, themesource, styling, design properties) that had to be copied by hand. The only marketplace-specific work is resolving a content id and downloading to a temp path. So a file input is a small upstream change, not a fork; the one real design question is what the version stamp should record for a package from disk. Recommendation recorded: issue with sizing, then offer the PR, ship a wrapper script meanwhile, and fork only if upstream declines — the same overlay-not-fork argument this toolkit made for the company brain. Estimate labelled as read-from-call-graph, not a built patch. — Maurits Visser - learn(eval install run): **"could the CLI solve it in theory?" — it solves it in practice, proven end to end headless, and the earlier "blocked" verdict was about `mx`, not about the CLI as a whole.** Two probes: mxcli reads the **10.6.4** model fine (`SHOW MODULES`, exit 0) where `mx module-import` and `mx convert` at 11.14 both refuse it, so the version gate is `mx`'s alone; and the theme module carries **zero model documents**, so installing it is a file operation, not a model import. Copying `themesource/` + `widgets/` builds clean — **and silently applies none of the brand colours**, because the module's `main.scss` imports the *project's* `theme/web/custom-variables` while the file holding every USI value ships as `usi-custom-variables.scss` and is imported by nothing. A green build with the wrong palette: the false-green class this toolkit tracks. With that one extra copy the build succeeds and all five brand hexes are verified in the compiled CSS. Cost stated: the swap overwrites the project's existing theme, so on a non-greenfield app it is a merge. — Maurits Visser - learn(skills/company-brain.md, install run record): **"can a module built on an older Mendix be upgraded headlessly?" — yes, with `mx convert`, inside a version window; probed, not assumed.** `mx convert .mpk .mpk` is the upgrade tool and needs no GUI; it takes the package, and pointing it at the `.mpr` inside fails with *is not an mpk file*. Its window is the toolset's own: 11.14 accepts 10.21–10.24 and refuses older — the company theme package (10.6.4.28084) fails with *The version '10.6.4.28084' of the mpr file is not supported*, the same boundary `module-import` names. So an older package needs one hop per window, and the intermediate binary must be obtainable: the CDN path the tooling uses returns 200 for 11.x and 404 for every 10.x spelling tried (six versions, four URL shapes), so that hop may have to happen on a machine that already has that toolset. Corrects the previous entry's implication that the package simply cannot be installed — it can, after one conversion elsewhere. — Maurits Visser diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/UPSTREAM-PR.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/UPSTREAM-PR.md new file mode 100644 index 00000000..38c3d0ad --- /dev/null +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-install/UPSTREAM-PR.md @@ -0,0 +1,48 @@ +# From "could we change the CLI?" to a tested PR — 2026-09-20 + +The fork-or-PR note said the seams were in the right place. This proves it: the change was +built on a shallow clone of `mendixlabs/mxcli`, tested at three layers, and field-run against +real packages with the built binary. **Not submitted** — this session cannot reach that repo; the +two patches, the PR body and a submission recipe are staged in the private repo under +`gh-issues-ready/mxcli-pr-local-mpk-install/`. + +## The approach, and why this order + +1. **Read the repo's own rules first** (`CONTRIBUTING.md`, `CLAUDE.md`'s PR checklist): issue + before code, test written first, verify at the layer the symptom lives in, prove a fix by + revert, record a finding. The PR follows all of them, so a maintainer reviews the change and + not the process. +2. **Baseline before touching anything**: grammar generated, clean tree built, vetted, tested. + Without that, a failure later cannot be attributed. +3. **Smallest honest change.** A file input through the existing writer. The 10.6.4 case stays + refused by `mx`'s window and is documented with the remedy, not worked around. +4. **Field run with real packages, not fixtures**, using a module exported by `mx` itself from one + app and installed into another, plus a real widget, plus the real 10.6.4 theme package for the + refusal path. + +## What the field run found that no unit test would have + +- **A pre-existing bug in the install writer**: with a relative `-p app.mpr` every bundled file was + refused as a path-traversal attempt, *after* the module had been transplanted — a half install + reporting failure. The traversal tests only ever used absolute temp dirs, so the root was + canonical by accident. Fixed as its own commit, test shown failing first, finding recorded in + the repo's log. +- **Two bugs in my own integration test**, both mine: a temp `HOME` (copied from the unit-test + helper) hid the mxbuild cache; and `return out.String(), cmd.Execute()` reads the buffer before + the command runs. Both were misread as product failures for one round each. The lesson is the + toolkit's own: an instrument's first run tests the instrument. + +## Final state + +| Check | Result | +|---|---| +| 8 unit tests, new | pass; client factory fatal-if-called proves no PAT | +| 1 unit test for the fix | fails on unpatched code with the reported message, passes on patched | +| integration test, real package | pass: module in model, no marketplace stamp, MPR v2 preserved, 5 bundled files verified, second run reports | +| `make lint-go`, `go test ./...`, `make check-findings`, `make sync-all` | all green / nothing generated | +| field run, relative `-p` | 4 units, 6 files, `mx check` 0 errors, Source column empty | +| 10.6.4 theme package | refused with `mx`'s message, app untouched | + +The company-brain component manifest's install field can now say exactly what works: `mxcli +marketplace install --file .mpk -p app.mpr` on this branch, or the hand procedure recorded +in `CLI-CAN-DO-IT.md` until it merges. From 1a2d94df8aa6d36b3fda3099590a0ce70f0cd0fd Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 21 Sep 2026 07:34:48 +0000 Subject: [PATCH 17/18] evals: genericize container paths in the company-brain run transcripts The leak guard's home-directory probe is right to flag them, even though they are a cloud container's paths and not a person's machine. and carry the same information. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- .../runs/2026-09-19/control-1.md | 32 ++++++++--------- .../runs/2026-09-19/control-2.md | 4 +-- .../runs/2026-09-19/control-3.md | 18 +++++----- .../runs/2026-09-19/treatment-1.md | 8 ++--- .../runs/2026-09-19/treatment-2.md | 28 +++++++-------- .../runs/2026-09-19/treatment-3.md | 4 +-- .../2026-09-20-real/after-fix/treatment-1.md | 34 +++++++++---------- .../2026-09-20-real/after-fix/treatment-2.md | 16 ++++----- .../2026-09-20-real/after-fix/treatment-3.md | 26 +++++++------- .../runs/2026-09-20-real/control-1.md | 2 +- .../runs/2026-09-20-real/control-2.md | 12 +++---- .../runs/2026-09-20-real/control-3.md | 22 ++++++------ .../runs/2026-09-20-real/treatment-1.md | 20 +++++------ .../runs/2026-09-20-real/treatment-2.md | 22 ++++++------ .../runs/2026-09-20-real/treatment-3.md | 20 +++++------ 15 files changed, 134 insertions(+), 134 deletions(-) diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-1.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-1.md index 30913bf9..db5549d2 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-1.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-1.md @@ -19,22 +19,22 @@ This is read-only (UC001 postcondition: "displayed from governed APIs" — no cr ## Files I read -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop` (directory listing) -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/PROJECT.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/CLAUDE.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/CLAUDE.local.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop` (find for RESUME.md / *.local.md / directory tree — confirmed no `docs/progress/RESUME.md` and no `design/` folder) -- `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (listing) + `/home/user/mxcli-project-toolkit/skills/ui-preflight-pages.md` (listing) -- `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (full read) -- `/home/user/mxcli-project-toolkit/skills/ui-preflight-pages.md` (partial read, Steps 1–2) -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/MPR-PLACEHOLDER.txt` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/index.json` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (domainEntities, then pages) -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (top-level keys) -- `/home/user/mxcli-project-toolkit/skills/company-brain.md` (grep + full head read) -- `/home/user/mxcli-project-toolkit` (find for `*company-brain*` references) -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/*.md` (grep for company-brain / design system mentions) -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/CLAUDE.local.md` (grep count for `COMPANY-BRAIN:BEGIN`) +- `/cb-eval/control/USI_Routing_Workshop` (directory listing) +- `/cb-eval/control/USI_Routing_Workshop/PROJECT.md` +- `/cb-eval/control/USI_Routing_Workshop/CLAUDE.md` +- `/cb-eval/control/USI_Routing_Workshop/CLAUDE.local.md` +- `/cb-eval/control/USI_Routing_Workshop` (find for RESUME.md / *.local.md / directory tree — confirmed no `docs/progress/RESUME.md` and no `design/` folder) +- `/mxcli-project-toolkit/skills/design-artifacts.md` (listing) + `/mxcli-project-toolkit/skills/ui-preflight-pages.md` (listing) +- `/mxcli-project-toolkit/skills/design-artifacts.md` (full read) +- `/mxcli-project-toolkit/skills/ui-preflight-pages.md` (partial read, Steps 1–2) +- `/cb-eval/control/USI_Routing_Workshop/MPR-PLACEHOLDER.txt` +- `/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/index.json` +- `/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (domainEntities, then pages) +- `/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (top-level keys) +- `/mxcli-project-toolkit/skills/company-brain.md` (grep + full head read) +- `/mxcli-project-toolkit` (find for `*company-brain*` references) +- `/cb-eval/control/USI_Routing_Workshop/*.md` (grep for company-brain / design system mentions) +- `/cb-eval/control/USI_Routing_Workshop/CLAUDE.local.md` (grep count for `COMPANY-BRAIN:BEGIN`) ## Open questions for the user diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-2.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-2.md index 5a8fcca0..a829eaf8 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-2.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-2.md @@ -21,8 +21,8 @@ Not built yet — this is a plan only, pending the design-system question below. - `intake.md` - `triage.md` - `analysis/knowledge-base/brd/F001-routing-overview.brd.json` -- `/home/user/mxcli-project-toolkit/skills/company-brain.md` -- `/home/user/mxcli-project-toolkit/skills/conversion-runbook.md` (grep on Stage 3/Stage 5/design system) +- `/mxcli-project-toolkit/skills/company-brain.md` +- `/mxcli-project-toolkit/skills/conversion-runbook.md` (grep on Stage 3/Stage 5/design system) - `widgets-inventory.txt` - directory listing (no `design/`, `architecture/`, `mdlsource/`, `mxcli` binary, or `.mpr` file present — only `MPR-PLACEHOLDER.txt`) diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-3.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-3.md index afe61b91..90d8a703 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-3.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/control-3.md @@ -27,15 +27,15 @@ This is a plan, not a build — no MDL was written and the project was not modif ## Files I read -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/PROJECT.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/CLAUDE.local.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/intake.md` -- (directory listing) `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/` — searched for `design`, `wireframe`, `*.css`, `*brd*` -- `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` -- `/home/user/mxcli-project-toolkit/skills/company-brain.md` -- (directory listing) `/home/user`, `/home/user/mxcli-project-toolkit` top-level -- `/home/user/mxcli-project-toolkit/skills/conversion-runbook.md` (grepped for Stage 3/4/5 sections) -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +- `/cb-eval/control/USI_Routing_Workshop/PROJECT.md` +- `/cb-eval/control/USI_Routing_Workshop/CLAUDE.local.md` +- `/cb-eval/control/USI_Routing_Workshop/intake.md` +- (directory listing) `/cb-eval/control/USI_Routing_Workshop/` — searched for `design`, `wireframe`, `*.css`, `*brd*` +- `/mxcli-project-toolkit/skills/design-artifacts.md` +- `/mxcli-project-toolkit/skills/company-brain.md` +- (directory listing) ``, `/mxcli-project-toolkit` top-level +- `/mxcli-project-toolkit/skills/conversion-runbook.md` (grepped for Stage 3/4/5 sections) +- `/cb-eval/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (Note: `CLAUDE.md` and `AGENTS.md` in the project root were present but not opened directly — their content is identical to the toolkit-generated `CLAUDE.md`/`CLAUDE.local.md` pattern already covered via `CLAUDE.local.md` and the toolkit's own `CLAUDE.md`, which was supplied in the system context rather than re-read from disk.) diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-1.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-1.md index 86bc7e13..f496cbc9 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-1.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-1.md @@ -25,11 +25,11 @@ Target: overview page for the routing catalogue entity, per BRD `F001-routing-ov ## Files I read -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop` (directory listing) +- `/cb-eval/treatment/USI_Routing_Workshop` (directory listing) - `.../USI_Routing_Workshop/CLAUDE.md` - `.../USI_Routing_Workshop/CLAUDE.local.md` - `.../USI_Routing_Workshop` — searched for `docs/progress/RESUME.md` (does not exist) -- `/tmp/claude-0/.../scratchpad/cb-eval/usi-brain/` (directory listing) +- `/cb-eval/usi-brain/` (directory listing) - `usi-brain/ROUTING.md` - `usi-brain/README.md` - `usi-brain/components/` and `usi-brain/skills/` (directory listing) @@ -41,8 +41,8 @@ Target: overview page for the routing catalogue entity, per BRD `F001-routing-ov - `.../USI_Routing_Workshop/widgets-inventory.txt` - `.../USI_Routing_Workshop/analysis/` (recursive file listing) - `.../USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` -- `/home/user/mxcli-project-toolkit/skills/ui-preflight-pages.md` -- `/home/user/mxcli-project-toolkit/skills/conversion-runbook.md` (grepped for stage headings) +- `/mxcli-project-toolkit/skills/ui-preflight-pages.md` +- `/mxcli-project-toolkit/skills/conversion-runbook.md` (grepped for stage headings) ## Open questions for the user diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-2.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-2.md index a5d44d57..79a27447 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-2.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-2.md @@ -43,20 +43,20 @@ before the first widget is drafted — see Open questions. ## Files I read -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/CLAUDE.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/CLAUDE.local.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/PROJECT.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/` (directory listing, `find`) -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/` (directory listing, `ls`) -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/ROUTING.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/components/usi.USIDesignSystem.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/skills` (directory listing) -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/skills/house-page-conventions.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/usi-brain/components/README.md` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/widgets-inventory.txt` -- `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/treatment/USI_Routing_Workshop/triage.md` -- `/home/user/mxcli-project-toolkit/skills/ui-preflight-pages.md` +- `/cb-eval/treatment/USI_Routing_Workshop/CLAUDE.md` +- `/cb-eval/treatment/USI_Routing_Workshop/CLAUDE.local.md` +- `/cb-eval/treatment/USI_Routing_Workshop/PROJECT.md` +- `/cb-eval/treatment/USI_Routing_Workshop/` (directory listing, `find`) +- `/cb-eval/usi-brain/` (directory listing, `ls`) +- `/cb-eval/usi-brain/ROUTING.md` +- `/cb-eval/usi-brain/components/usi.USIDesignSystem.md` +- `/cb-eval/usi-brain/skills` (directory listing) +- `/cb-eval/usi-brain/skills/house-page-conventions.md` +- `/cb-eval/usi-brain/components/README.md` +- `/cb-eval/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +- `/cb-eval/treatment/USI_Routing_Workshop/widgets-inventory.txt` +- `/cb-eval/treatment/USI_Routing_Workshop/triage.md` +- `/mxcli-project-toolkit/skills/ui-preflight-pages.md` ## Open questions for the user diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-3.md b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-3.md index dacaa395..b0e734e1 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-3.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-19/treatment-3.md @@ -43,10 +43,10 @@ I have **not** written any MDL, imported the MPK, or touched the project — thi - `/tmp/.../treatment/USI_Routing_Workshop/widgets-inventory.txt` - `/tmp/.../treatment/USI_Routing_Workshop/MPR-PLACEHOLDER.txt` - `/tmp/.../treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` -- `/home/user/mxcli-project-toolkit/skills/ui-preflight-pages.md` +- `/mxcli-project-toolkit/skills/ui-preflight-pages.md` - (directory listing only, no file content) `/tmp/.../treatment/USI_Routing_Workshop/analysis/knowledge-base/` tree -(Full paths root at `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval/`.) +(Full paths root at `/cb-eval/`.) ## Open questions for the user diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-1.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-1.md index bbfbe2f6..ec02b2b5 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-1.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-1.md @@ -82,23 +82,23 @@ written, and no MDL/pages were created or modified. ## Files I read -1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop` (ls) -2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.md` -3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.local.md` -4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/PROJECT.md` -5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/AGENTS.md` (diffed against CLAUDE.md — identical) -6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/analysis` (find, listing) -7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real` (ls) and `ROUTING.md` -8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/` (ls) -9. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/USI_Theme_Module.md` -10. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/` (ls) -11. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/house-page-conventions.md` -12. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/README.md` -13. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` -14. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (read-only, to check the generic branding-interview process this company brain overrides) -15. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/widgets-inventory.txt` -16. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/intake.md` and `triage.md` (grepped for brand/design/colour/theme/logo — no hits) -17. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/projects.tsv` +1. `/cb-eval3/treatment/USI_Routing_Workshop` (ls) +2. `/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.md` +3. `/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.local.md` +4. `/cb-eval3/treatment/USI_Routing_Workshop/PROJECT.md` +5. `/cb-eval3/treatment/USI_Routing_Workshop/AGENTS.md` (diffed against CLAUDE.md — identical) +6. `/cb-eval3/treatment/USI_Routing_Workshop/analysis` (find, listing) +7. `/usi-brain-real` (ls) and `ROUTING.md` +8. `/usi-brain-real/components/` (ls) +9. `/usi-brain-real/components/USI_Theme_Module.md` +10. `/usi-brain-real/skills/` (ls) +11. `/usi-brain-real/skills/house-page-conventions.md` +12. `/usi-brain-real/components/README.md` +13. `/cb-eval3/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +14. `/mxcli-project-toolkit/skills/design-artifacts.md` (read-only, to check the generic branding-interview process this company brain overrides) +15. `/cb-eval3/treatment/USI_Routing_Workshop/widgets-inventory.txt` +16. `/cb-eval3/treatment/USI_Routing_Workshop/intake.md` and `triage.md` (grepped for brand/design/colour/theme/logo — no hits) +17. `/usi-brain-real/projects.tsv` ## Open questions for the user diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-2.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-2.md index fac83d44..ee82fe00 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-2.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-2.md @@ -87,14 +87,14 @@ task's "preparation only" instruction. ## Files I read -1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.md` -2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/ROUTING.md` -3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/USI_Theme_Module.md` -4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/house-page-conventions.md` -5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/PROJECT.md` -6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/intake.md` -7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/triage.md` -8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +1. `/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.md` +2. `/usi-brain-real/ROUTING.md` +3. `/usi-brain-real/components/USI_Theme_Module.md` +4. `/usi-brain-real/skills/house-page-conventions.md` +5. `/cb-eval3/treatment/USI_Routing_Workshop/PROJECT.md` +6. `/cb-eval3/treatment/USI_Routing_Workshop/intake.md` +7. `/cb-eval3/treatment/USI_Routing_Workshop/triage.md` +8. `/cb-eval3/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (`CLAUDE.local.md`, `AGENTS.md`, and `widgets-inventory.txt` exist in the project root but were not opened — `CLAUDE.md` states it is the canonical instruction file and the others are pointers diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-3.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-3.md index bcc1d171..06d63301 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-3.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/after-fix/treatment-3.md @@ -1,7 +1,7 @@ ## Design system This is a USI app, wired to the USI company brain -(`/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real`, pointed to from +(`/usi-brain-real`, pointed to from `CLAUDE.md`'s Company-brain block). Its routing table says: "Choosing a design system, theme, brand colour or page shell for a USI app → `components/USI_Theme_Module.md`", and the house rule is explicit: **never hand-roll a theme or a second design system for a USI app.** @@ -83,18 +83,18 @@ Stage-3 deliverable (not yet produced — `design/` doesn't exist in this projec ## Files I read -1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/PROJECT.md` -2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/intake.md` -3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.md` -4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.local.md` -5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop` (directory listing — no `docs/progress/RESUME.md` present) -6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real` (directory listing) -7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/ROUTING.md` -8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/README.md` -9. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/USI_Theme_Module.md` -10. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/` and `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/` (directory listings) -11. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/house-page-conventions.md` -12. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval3/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +1. `/cb-eval3/treatment/USI_Routing_Workshop/PROJECT.md` +2. `/cb-eval3/treatment/USI_Routing_Workshop/intake.md` +3. `/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.md` +4. `/cb-eval3/treatment/USI_Routing_Workshop/CLAUDE.local.md` +5. `/cb-eval3/treatment/USI_Routing_Workshop` (directory listing — no `docs/progress/RESUME.md` present) +6. `/usi-brain-real` (directory listing) +7. `/usi-brain-real/ROUTING.md` +8. `/usi-brain-real/README.md` +9. `/usi-brain-real/components/USI_Theme_Module.md` +10. `/usi-brain-real/components/` and `/usi-brain-real/skills/` (directory listings) +11. `/usi-brain-real/skills/house-page-conventions.md` +12. `/cb-eval3/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (The corporate brand-guide PDF, `components/USI-UI-UX-Guide-v2.0.pdf`, was not opened directly — the `USI_Theme_Module.md` manifest already states the reconciliation between the guide's Corporate blue diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-1.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-1.md index c7c9c199..ffd9e9f0 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-1.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-1.md @@ -77,7 +77,7 @@ before it ships. 7. `analysis/knowledge-base/brd/index.json` 8. `CLAUDE.local.md` (this project's) 9. `CLAUDE.md` (this project's, partial — communication style, mxcli reference sections) -10. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` +10. `/mxcli-project-toolkit/skills/design-artifacts.md` 11. (directory listings/greps, not single files: `find analysis -type f`, `grep -ril` for brand/colour/theme/design-system/wireframe/logo terms across the project, `find . -iname "*source*"` / `-iname design` / `-iname sources`) ## Open questions for the user diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-2.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-2.md index a203f2b6..5148df21 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-2.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-2.md @@ -40,13 +40,13 @@ Both screens are read-only per the confirmed scope; no create/edit/delete afford ## Files I read -1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/CLAUDE.md` -2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/CLAUDE.local.md` -3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/PROJECT.md` -4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/triage.md` (partial) +1. `/cb-eval2/control/USI_Routing_Workshop/CLAUDE.md` +2. `/cb-eval2/control/USI_Routing_Workshop/CLAUDE.local.md` +3. `/cb-eval2/control/USI_Routing_Workshop/PROJECT.md` +4. `/cb-eval2/control/USI_Routing_Workshop/triage.md` (partial) 5. Directory listing of `analysis/` (found `brd-report.html`, `knowledge-base/`, `facts.lock.json`; no `design/`, `architecture/`, or `.css` files anywhere in the project) -6. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (Step 0a/0b/1/2/3, read for the process governing design system + branding decisions) -7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (UC001 flow, screens, and the `Routing`/`RoutingVersion` entity attributes used for the page plan) +6. `/mxcli-project-toolkit/skills/design-artifacts.md` (Step 0a/0b/1/2/3, read for the process governing design system + branding decisions) +7. `/cb-eval2/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (UC001 flow, screens, and the `Routing`/`RoutingVersion` entity attributes used for the page plan) Not read: `AGENTS.md` and `intake.md`/`widgets-inventory.txt` (redundant with CLAUDE.md / out of scope for a design-only question), and the rest of the BRD/KB files (deferred BRDs are out of the confirmed F001-only scope). diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-3.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-3.md index d82a5ef3..c63ade61 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-3.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/control-3.md @@ -88,17 +88,17 @@ not built that file, per this task's "preparation only" scope. ## Files I read -1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/CLAUDE.md` -2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/CLAUDE.local.md` -3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/PROJECT.md` -4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/intake.md` -5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/triage.md` -6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/widgets-inventory.txt` -7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/analysis/knowledge-base/share/KB_MES20_APIBehaviorSpec.md` (grep only, then targeted read) -8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` -9. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` -10. `/home/user/mxcli-project-toolkit/skills/conversion-runbook.md` (Stage 3 section, via grep) -11. `/home/user/mxcli-project-toolkit/skills/checkpoints/checkpoint-design.md` +1. `/cb-eval2/control/USI_Routing_Workshop/CLAUDE.md` +2. `/cb-eval2/control/USI_Routing_Workshop/CLAUDE.local.md` +3. `/cb-eval2/control/USI_Routing_Workshop/PROJECT.md` +4. `/cb-eval2/control/USI_Routing_Workshop/intake.md` +5. `/cb-eval2/control/USI_Routing_Workshop/triage.md` +6. `/cb-eval2/control/USI_Routing_Workshop/widgets-inventory.txt` +7. `/cb-eval2/control/USI_Routing_Workshop/analysis/knowledge-base/share/KB_MES20_APIBehaviorSpec.md` (grep only, then targeted read) +8. `/cb-eval2/control/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +9. `/mxcli-project-toolkit/skills/design-artifacts.md` +10. `/mxcli-project-toolkit/skills/conversion-runbook.md` (Stage 3 section, via grep) +11. `/mxcli-project-toolkit/skills/checkpoints/checkpoint-design.md` Also listed (not opened as content, only directory-checked): `analysis/knowledge-base/brd/*.deferred.json` filenames, `analysis/knowledge-base/reports/validation-report.md` filename, `analysis/knowledge-base/share/*.md` filenames (grepped for brand/color/theme keywords, no matches beyond the Atlas layout hit above and an unrelated "styled" match in the API spec). diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-1.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-1.md index fde824f0..c367b761 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-1.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-1.md @@ -49,17 +49,17 @@ All widget choices are read-only (no CREATE/CHANGE actions) — matching the con ## Files I read -1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/AGENTS.md` (diffed identical to CLAUDE.md) -2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.md` -3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/PROJECT.md` -4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/intake.md` -5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop` directory listing (`find analysis -type f`) +1. `/cb-eval2/treatment/USI_Routing_Workshop/AGENTS.md` (diffed identical to CLAUDE.md) +2. `/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.md` +3. `/cb-eval2/treatment/USI_Routing_Workshop/PROJECT.md` +4. `/cb-eval2/treatment/USI_Routing_Workshop/intake.md` +5. `/cb-eval2/treatment/USI_Routing_Workshop` directory listing (`find analysis -type f`) 6. Grep of `analysis`, `intake.md`, `triage.md`, `widgets-inventory.txt`, `CLAUDE.md` for color/brand/theme/design-system keywords -7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/share/KB_MES20_FunctionalSpec.md` (grepped) -8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/share/KB_MES20_APIBehaviorSpec.md` (grepped, multiple passes) -9. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (full read) -10. `/home/user/mxcli-project-toolkit/skills/` directory listing (ls, to locate design-related skills) -11. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (full + targeted `pages`/`integrations` sections via python) +7. `/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/share/KB_MES20_FunctionalSpec.md` (grepped) +8. `/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/share/KB_MES20_APIBehaviorSpec.md` (grepped, multiple passes) +9. `/mxcli-project-toolkit/skills/design-artifacts.md` (full read) +10. `/mxcli-project-toolkit/skills/` directory listing (ls, to locate design-related skills) +11. `/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` (full + targeted `pages`/`integrations` sections via python) (Not opened: `.mpr` file, `triage.md` full body, `widgets-inventory.txt` full body, `AGENTS.md` full body — all confirmed via grep/diff to carry no design/brand content beyond what's captured above. No task instructs modifying the project or the toolkit repos, and none were touched.) diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-2.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-2.md index 720aa293..18ece6e9 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-2.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-2.md @@ -3,7 +3,7 @@ **`USI_Theme_Module`** — the company-brain-approved Mendix theme module, not a bespoke or Atlas-default theme. - Source of the rule: this project's `CLAUDE.local.md` wires a USI company brain at - `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real`, whose `ROUTING.md` routes + `/usi-brain-real`, whose `ROUTING.md` routes "choosing a design system, theme, brand colour or page shell for a USI app" to `components/USI_Theme_Module.md`. - That manifest and `skills/house-page-conventions.md` are explicit: **never hand-roll a theme @@ -78,16 +78,16 @@ treats the two as one flow. ## Files I read -1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.md` -2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.local.md` -3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/PROJECT.md` -4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/README.md` -5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/ROUTING.md` -6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/USI_Theme_Module.md` -7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/house-page-conventions.md` -8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/intake.md` -9. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` -10. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/widgets-inventory.txt` +1. `/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.md` +2. `/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.local.md` +3. `/cb-eval2/treatment/USI_Routing_Workshop/PROJECT.md` +4. `/usi-brain-real/README.md` +5. `/usi-brain-real/ROUTING.md` +6. `/usi-brain-real/components/USI_Theme_Module.md` +7. `/usi-brain-real/skills/house-page-conventions.md` +8. `/cb-eval2/treatment/USI_Routing_Workshop/intake.md` +9. `/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +10. `/cb-eval2/treatment/USI_Routing_Workshop/widgets-inventory.txt` (Directory listings of the project root and `analysis/` were also run but are not file reads.) diff --git a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-3.md b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-3.md index 2b15cbe8..712284ae 100644 --- a/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-3.md +++ b/evals/scenarios/company-brain-design-module/runs/2026-09-20-real/treatment-3.md @@ -41,16 +41,16 @@ Buttons throughout: theme's `usi-blue` for primary actions (Search, row navigati ## Files I read -1. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.md` -2. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.local.md` -3. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/ROUTING.md` -4. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/components/USI_Theme_Module.md` -5. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/PROJECT.md` -6. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis` (directory listing) -7. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` -8. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/usi-brain-real/skills/house-page-conventions.md` -9. `/home/user/mxcli-project-toolkit/skills/design-artifacts.md` (partial, for wireframe/branding-interview convention — superseded here by the company brain's fixed theme) -10. `/tmp/claude-0/-home-user/290fe318-2e84-5498-9083-13e06ee724da/scratchpad/cb-eval2/treatment/USI_Routing_Workshop/widgets-inventory.txt` +1. `/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.md` +2. `/cb-eval2/treatment/USI_Routing_Workshop/CLAUDE.local.md` +3. `/usi-brain-real/ROUTING.md` +4. `/usi-brain-real/components/USI_Theme_Module.md` +5. `/cb-eval2/treatment/USI_Routing_Workshop/PROJECT.md` +6. `/cb-eval2/treatment/USI_Routing_Workshop/analysis` (directory listing) +7. `/cb-eval2/treatment/USI_Routing_Workshop/analysis/knowledge-base/brd/F001-routing-overview.brd.json` +8. `/usi-brain-real/skills/house-page-conventions.md` +9. `/mxcli-project-toolkit/skills/design-artifacts.md` (partial, for wireframe/branding-interview convention — superseded here by the company brain's fixed theme) +10. `/cb-eval2/treatment/USI_Routing_Workshop/widgets-inventory.txt` (Also listed, not fully read: `usi-brain-real/` top-level directory and `components/` directory for file inventory.) From 7d779216e23d63d5d9b3a4578dd739f1d571c74e Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 22 Sep 2026 18:10:51 +0000 Subject: [PATCH 18/18] new(existing-app-change): map the app first, ask about findings, park until the change is named Stage 0 in this mode opened with "which slice?" before anyone had seen the app, and the app-analysis instrument (app-facts.sh + app-report.sh) existed but nothing in the change flow ran or read it. Now Stage 0a runs the map without asking (read-only, ~1 min), then asks the user to disposition the top findings (fix / log / accept). The project may park there; Stage 0b scopes the slice and reads its blast radius from the map once the change is named. The manifest owes the map (app-report) in this mode, gate-check reads a mapped-but-unscoped project as Stage 0 PENDING rather than a permanent FAIL, and its Stage 1 hint names Path D. existing-app-assurance Track A starts from the same report. Field run: a live client workflow app, 28 modules, mapped in 76 s, gate-check PENDING/parked with 0 needing attention. Fixture: test-bug03-gates.sh T12 (parked, unmapped, other-mode controls). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Cjxzzw8LVrfLrEmz8YMXQW --- CHANGELOG.md | 1 + bin/gate-check.sh | 13 +++++++++ bin/lib/artifact-manifest.tsv | 5 +++- docs/pipeline-walks.html | 22 +++++++------- skills/app-analysis.md | 14 ++++----- skills/existing-app-assurance.md | 9 +++++- skills/existing-app-change.md | 50 ++++++++++++++++++++++++++++---- tests/wave2/test-bug03-gates.sh | 35 +++++++++++++++++++++- 8 files changed, 123 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 51ab0539..8ba727d1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- new(existing-app-change): **map the app first, ask what to do with the findings, and let a project park until the change is named.** Stage 0 in this mode now opens with the app analysis (`app-facts.sh` + `app-report.sh`, `skills/app-analysis.md`), run without asking since it is read-only and takes about a minute, followed by the question the user owns: fix / log / accept per top finding. Kickoff no longer opens with "which slice?"; the slice and its blast radius (read from the map's tangles and edges, not recomputed) come at Stage 0b, when the change arrives. `artifact-manifest.tsv` owes the map in this mode (`app-report`, Stage 0); gate-check reports a mapped project with no change as Stage 0 `PENDING` instead of a permanent FAIL, and its Stage 1 hint names Path D instead of an extractor. `existing-app-assurance.md` Track A starts from the same report, so an audit that turns into a change does not redo it. Fixture: `test-bug03-gates.sh` T12. Field run: a live client workflow app — 28 modules, one tangle of 7 of 9 own modules, 213 loop microflows, mapped in 76 s; gate-check read it PENDING/parked with 0 needing attention — Maurits Visser - docs(pipeline-walks): **`docs/pipeline-walks.html` — a process diagram per entry mode, with the scripts run at every step.** Shared spine, migration, requirements-driven (incl. the docs-ready fast path), greenfield, change-an-existing-app (opening with the app-mapping step: `SHOW STRUCTURE`, `graph-report`, `lint`, `report`, security matrix, `marketplace diff`), à-la-carte tracks A/A2/B, and the Stage 5 BUILD→GATE→PROVE→LOOK→CONFIRM loop, each as a mermaid flowchart plus a stage/what/scripts table. Linked from the README entry-modes paragraph — Maurits Visser - fix(bin/doctor.sh): **doctor told every Podman user "docker is not installed"** — the section advertised Podman in its advice text ("Rancher Desktop or Podman … are common substitutes") while all four probes ran `docker` only: `docker info`, the `command -v docker` gate, the not-installed warning, and a start hint that said `open -a Docker`. So a machine fully able to run the container lane on Podman, but without the docker shim, was reported broken — and on a team that cannot licence Docker Desktop that reads as "go install software you are not allowed to have" (a colleague's machine-ready status carried "Docker not installed" as a known issue; they may have had Podman all along). Detection is now docker-then-podman (`MXTK_CONTAINER_RUNTIME` forces one), the runtime is **named** in the report (`podman responding — …`), the start hint knows `podman machine start` / `podman.socket`, and the not-installed warning names Podman as the licence-free option instead of implying Docker Desktop is required. Same bounded background/poll/kill probe for both, same 0/1/2 exit contract; `mxcli docker check` invocation deliberately untouched (different repo). **Not field-run** — no container runtime in the authoring container; needs one run on a Mac with Podman and no `docker` on PATH. Driver: the Mendix migration team's Docker Desktop licensing constraint — Maurits Visser - learn(skills/doctor-triage.md): **"doctor.sh says red — what now?" is now on disk instead of in a Slack thread.** Three failures that render identically get separated: my environment is wrong / the toolkit's own self-check is wrong / this line does not apply in my lane. Check the machine before naming a fix — a wrong-arch binary, a missing one and a broken self-check all read the same, which is how *"install Studio Pro 10.24.18"* became the first confident answer to a Linux-ELF mxbuild on a Mac, and how a false `fail (unreadable error file)` (the self-test bug fixed in `c0ea53c`) sent people to audit their own machines. Also: the three toolchain lanes (bundled `mx` — macOS ships one only from Mendix 11 — the Linux-only CDN toolchain, and the container lane, runtime-agnostic although doctor still probes only `docker`), a derivation for which FAILs block rather than a list that rots, the N+M-lines-dispositioned bound, and the VM/arch caveat. From the macOS onboarding thread of 2026-09-22 — Yvann, and the four people in it diff --git a/bin/gate-check.sh b/bin/gate-check.sh index 609d5aca..a902c938 100755 --- a/bin/gate-check.sh +++ b/bin/gate-check.sh @@ -824,6 +824,12 @@ check_stage_0() { }' "$f") if [ -z "$signer" ]; then echo "FAIL|no non-empty 'Confirmed by:' line inside the '## Sign-off' section of $f — anything the user actually said is enough (\"confirmed in chat\", \"agreed, move on\"); it only has to not be the shipped placeholder" + elif printf '%s' "$signer" | grep -q '\[' && [ "$ENTRY_MODE" = "existing-app-change" ] \ + && [ -s "$PROJECT_DIR/analysis/app-report.json" ]; then + # Mapped and waiting for a change is a normal state in this mode (existing-app-change.md + # §"Map the app first"): there is no slice to sign off yet, so this is not-started, not wrong. + # Without it a parked project read "needs attention" forever (existing-app field run, 2026-09-22). + echo "PENDING|app mapped (analysis/app-report.html), waiting on the change — triage.md is signed off at Stage 0b, once the user names the slice and its blast radius is written ($f, ## Sign-off)" elif printf '%s' "$signer" | grep -q '\['; then echo "FAIL|'Confirmed by:' still holds the shipped placeholder: \"$signer\" — replace it with whatever the user actually said (\"confirmed in chat 2026-08-20\" is fine; a full name is not required) ($f, ## Sign-off)" else @@ -837,6 +843,13 @@ check_stage_1() { # extractor it was never going to run (2026-08-20). "No knowledge-base directory" is true in # every mode; what to DO about it is not, and the KB for a document corpus is built by a # skill, not a pipeline. Name both routes rather than assuming the migration one. + if [ "$ENTRY_MODE" = "existing-app-change" ]; then + # Neither route above exists in this mode: there is no source to extract, only the live + # model, and Stage 1 cannot be scoped until the user has named the change (existing-app + # field run, 2026-09-22 — the generic hint pointed at an extractor the mode never runs). + echo "PENDING|no knowledge-base directory yet — existing app: this stage reads the live model (Path D, skills/existing-app-change.md), scoped to the change and its blast radius. Waiting on the change is a normal state: map the app first (analysis/app-report.html, skills/app-analysis.md), then scope when the user names the change" + return + fi echo "PENDING|no knowledge-base directory yet — migration: run the extraction pipeline; requirements-driven: build the KB from the document corpus per skills/kb-generation.md (Path B). Either way it lands at analysis//knowledge-base/" return fi diff --git a/bin/lib/artifact-manifest.tsv b/bin/lib/artifact-manifest.tsv index ec138e56..a0a64c1e 100644 --- a/bin/lib/artifact-manifest.tsv +++ b/bin/lib/artifact-manifest.tsv @@ -43,7 +43,9 @@ # migration,requirements,greenfield,existing-app-change # (conversion-runbook.md §"Entry Modes", its table; existing-app-change runs # P–6 per slice, so it owes every row but coverage-ledger — that mode's ledger -# is `mxcli brain plan`, existing-app-change.md §"Coverage in this mode"). +# is `mxcli brain plan`, existing-app-change.md §"Coverage in this mode") — plus +# app-report, the one row only it owes: the map of the live app that Stage 0's +# blast radius is read from (existing-app-change.md §"Map the app first"). # À-la-carte projects # run no stages and are excused wholesale by artifact-check.sh, same test as # obligation-check.sh. @@ -70,6 +72,7 @@ artifact stage paths producer consumers modes absence intake P intake.md bin/init-project.sh scaffold + the Stage-P kickoff interview (conversion-runbook.md §2 Stage P) bin/gate-check.sh:596 (check_stage_P), bin/gate-check.sh:497 (entry-mode fallback read) migration,requirements,greenfield,existing-app-change gate register P PROJECT.md bin/init-project.sh (empty register); every checkpoint writes into it (CLAUDE.md §One decision register) bin/gate-check.sh:285 (register resolution feeding every ✋ gate), bin/lib/obligation-check.sh:151 (waivers/adoption read from it) migration,requirements,greenfield,existing-app-change gate triage 0 triage.md bin/init-project.sh scaffold, filled per skills/source-triage.md; N/A rows recorded, never deleted (conversion-runbook.md:130) bin/gate-check.sh:684 (check_stage_0 sign-off test) migration,requirements,greenfield,existing-app-change gate +app-report 0 analysis/app-report.json|analysis/app-report.html project-bin/app-facts.sh (read-only: full catalog build + mxcli graph-report + loop sweep) then bin/app-report.sh — skills/app-analysis.md Procedure steps 1 and 5 skills/existing-app-change.md §"Map the app first" (the findings interview reads analysis/app-report.json) and §"Blast radius" (reads analysis/app-facts/dependencies.json tangles and edges) existing-app-change report extraction-report 1 analysis/*/knowledge-base/extraction-report.html|analysis/knowledge-base/extraction-report.html|knowledge-base/extraction-report.html bin/extraction-report.sh — one renderer for every entry mode bin/gate-check.sh:725 (check_stage_1) migration,requirements,existing-app-change gate source-ledger 1 analysis/source-ledger.html bin/source-ledger.sh report — every inventoried source file and what consumed it (Stage 1 owes a disposition per row; gate-check blocks 1–2 on the check) bin/gate-check.sh (LEDGER block — the check; the html is the human-facing table of the same rows) migration,requirements,existing-app-change report brds 2 analysis/*/knowledge-base/brd/*.brd.json|analysis/knowledge-base/brd/*.brd.json|knowledge-base/brd/*.brd.json pipeline generators (migration), skills/kb-generation.md (documents), skills/brd-generation.md (by hand) — the reader must not care which bin/lib/discover-brds.sh:43 (the one path authority), bin/gate-check.sh:773 (check_stage_2), bin/brd-report.sh, bin/open-questions.sh migration,requirements,existing-app-change gate diff --git a/docs/pipeline-walks.html b/docs/pipeline-walks.html index 604c2a3f..7e390a8b 100644 --- a/docs/pipeline-walks.html +++ b/docs/pipeline-walks.html @@ -244,17 +244,17 @@

Greenfield Just an idea, or a running s

Change an existing app A live .mpr you are altering, not rebuilding

-

The knowledge base comes from the model itself (Path D). The first real step is mapping the app for the agent: structure, quality, security, marketplace drift. Stage 0 adds the blast radius of the slice. A regression net goes under the app before anything changes, and every gate from Stage 5 on is also a regression gate.

+

The knowledge base comes from the model itself (Path D). The first real step is mapping the whole app, run without asking, followed by the one question the user owns: what to do with each top finding. The project may park there until a change is named; Stage 0b then adds the blast radius of the slice, read from the map. A regression net goes under the app before anything changes, and every gate from Stage 5 on is also a regression gate.

 flowchart TD
-  P["P Kickoff light
init-project.sh, intake from the model
register: Change an existing app CONFIRMED"] --> M["Map the app for the agent"]
-  M --> M1["structure
SHOW MODULES / ENTITIES / ASSOCIATIONS
mxcli graph-report"]
-  M --> M2["quality
mxcli lint + Starlark rules
mxcli report (assess-quality)"]
-  M --> M3["security and drift
SHOW SECURITY MATRIX
mxcli marketplace diff"]
-  M1 --> S0
-  M2 --> S0
-  M3 --> S0
-  S0["0 Triage ✋ two questions:
which slice, what is its blast radius"] --> S0a["blast radius into triage.md
associations, microflows (SEARCH),
pages (SHOW PAGES IN), module tangle,
published / consumed services"]
+  P["P Kickoff light
./mxcli in place, then init-project.sh
open question: what is driving it?
register: Change an existing app CONFIRMED"] --> M["0a Map the app, not asked, just run
bin/app-facts.sh then bin/app-report.sh
(app-analysis.md, about a minute)"]
+  M --> F["show the report, ask per top finding:
fix in this work / log / accept
answers into PROJECT.md"]
+  F --> PK{"change named yet?"}
+  PK -- "not yet" --> PARK["parked: a normal state
Stage 1 PENDING, nothing nags"]
+  PARK -.->|change arrives| S0
+  PK -- "only audits / tests wanted" --> AS["switch to existing-app-assurance
Track A starts from the same report"]
+  PK -- "yes" --> S0
+  S0["0b Triage ✋ which slice,
what is its blast radius"] --> S0a["blast radius into triage.md, read from the map:
dependencies.json tangles and edges,
then associations, SEARCH, SHOW PAGES IN,
published / consumed services"]
   S0a --> S0b["capability map from SHOW MODULES
extraction rows N/A
CAC-1, sign-off"]
   S0b --> RN["Regression net under the app
Track B baseline from existing-app-assurance"]
   RN --> S1["1 Analysis Path D
DESCRIBE ENTITY, SHOW MICROFLOWS IN,
graph-report scoped to slice + blast radius
counts recorded, Path A = N/A, CAC-1b"]
@@ -277,8 +277,8 @@ 
Change an existing app A live .mpr yo
   

-    
-    
+    
+    
diff --git a/skills/app-analysis.md b/skills/app-analysis.md
index 6fee454a..ad213ff4 100644
--- a/skills/app-analysis.md
+++ b/skills/app-analysis.md
@@ -299,12 +299,12 @@ decided otherwise.
 
 - `skills/module-dependency-review.md`, `skills/microflow-loop-antipatterns.md` (the judgement)
 - `skills/existing-app-assurance.md` (the à-la-carte audit this dossier is the map for)
-- `skills/existing-app-change.md` (the change slice, whose Stage 0 blast radius covers the same
-  ground this dossier already measured). **Not yet wired, as of 2026-09-16.** Neither of those two
-  files mentions the dossier, the facts or this skill, so nothing reads what this produces except
-  `bin/app-report.sh`. The change slice still recomputes its blast radius by hand. Until that is
-  joined, a dossier finding reaches a change slice only because a person carried it, and this
-  section describes an intended relationship, not an implemented one. Do not write a skill
-  description that claims otherwise.
+- `skills/existing-app-change.md` — **wired 2026-09-22.** Its Stage 0a runs this procedure before any
+  slice is named, asks the user to disposition the top findings, and reads the blast radius for Stage
+  0b from `analysis/app-facts/dependencies.json`. In that mode `bin/lib/artifact-manifest.tsv` owes
+  the report (`app-report`, Stage 0), so a map nobody ran reports PENDING. `existing-app-assurance.md`
+  Track A starts from the same report. What is still NOT wired: the dossier itself
+  (`architecture/app-dossier.md`) is not owed by anything, only the rendered report; and nothing
+  re-runs the facts at Stage 6 to show a change left the tangle and loop counts no worse.
 - `skills/lint-that-actually-runs.md`, `skills/report-schema.md`, `skills/measured-claims.md`
 - `skills/skills-over-scripts.md`: the instrument fetches, this file judges; keep it that way
diff --git a/skills/existing-app-assurance.md b/skills/existing-app-assurance.md
index 05e70d63..7f429d13 100644
--- a/skills/existing-app-assurance.md
+++ b/skills/existing-app-assurance.md
@@ -32,6 +32,13 @@ Not for building anything new. Two different places to go, depending on what "ne
 
 ## Track A — Analyze / audit the model
 
+**Start with the map.** `bin/app-facts.sh` then `/bin/app-report.sh`, per `app-analysis.md`:
+inventory, module tangles, loop-risk microflows and dead elements in about a minute, rendered as
+`analysis/app-report.html` with a fix-first list. Its Security and Lint sections report **fault** until
+you fill them from the rows below into `architecture/app-dossier.md` (§5, §6) and re-render — the
+report is the findings list the rows add to, not a separate deliverable. The same map is Stage 0a of
+`existing-app-change.md`, so an audit that turns into a change does not redo it.
+
 | Question | Tool |
 |---|---|
 | What's in the model? | `SHOW MODULES / ENTITIES / ASSOCIATIONS`, `DESCRIBE ENTITY`, `SEARCH` — see `query-the-model.md` |
@@ -41,7 +48,7 @@ Not for building anything new. Two different places to go, depending on what "ne
 | Overall quality scan | `assess-quality.md` (bundled) |
 | Have installed marketplace modules been locally edited? What would an upgrade overwrite? | `mxcli marketplace diff  -p app.mpr [--to VERSION] [--json]` (≥ v0.18; content-id via the module's AppStoreGuid) — reports per-element local edits and upgrade collisions; an element it cannot describe is reported **unknown, never unchanged**, and `verified:false` in the JSON means "no modifications found" is not a conclusion |
 
-**Deliverable:** a findings report (markdown or HTML — reuse `toolkit-guide.html`'s tokens), each finding with evidence (the query/lint output) and a proposed disposition: fix now / log / accept. Triage the list *with the user* — dispositions are their call.
+**Deliverable:** `analysis/app-report.html`, re-rendered after the dossier carries what the rows above found — each finding with evidence (the query/lint output) and a proposed disposition: fix now / log / accept. Show the report, then triage the list *with the user* — dispositions are their call, and asking for them is the step that ends Track A.
 
 ## Track A2 — Audit the design system (bounded: an afternoon, not a re-design)
 
diff --git a/skills/existing-app-change.md b/skills/existing-app-change.md
index c3ebde19..bce3e7d2 100644
--- a/skills/existing-app-change.md
+++ b/skills/existing-app-change.md
@@ -69,6 +69,41 @@ from git history of a `.mpr`, which is not a readable record.
 
 ---
 
+## Map the app first — before anyone names a slice
+
+**Do not open with "which slice?".** At kickoff most people do not know the change yet, or know it
+vaguely ("the approval flow is slow"), and a slice named before anyone has seen the app is a guess.
+Intake Q2 (*what is driving it?*) is the open question, and a change, if they have one, arrives in
+their own words there or later. Q4 (scope) may legitimately be answered
+`Unverified — how to verify: named when the change request arrives (Stage 0b)`.
+
+**Stage 0a — the map, not asked for, just run.** It is read-only and costs about a minute (field
+run, 2026-09-22: 28 modules, 213 loop microflows, 76 s), so asking "shall I analyse the app?" is a
+question with one sane answer. Run it right after intake, per `app-analysis.md` Procedure:
+
+```bash
+bin/app-facts.sh                                  # in the project root; exit 2 = facts not trustworthy, fix and rerun
+/bin/app-report.sh                       # analysis/app-report.html + app-report.json
+```
+
+**Then ask what to do with the findings — that is the question the human owns.** Read
+`analysis/app-report.json` (`fix_first`, severity counts), show the report, and put the top findings
+to the user, one disposition each: **fix as part of this work / log for later / accept, with a
+reason**. Record the answers in `PROJECT.md`. A finding nobody dispositioned is the same silence the
+gates exist to prevent. What the field run put on that list: one tangle holding 7 of 9 own modules,
+and a transaction started inside a loop in the app's core decision microflow — neither was in anyone's
+change request, and both decide how risky any change there is.
+
+**Then it may park.** A mapped app with no change yet is a normal state, not a failing one — gate-check
+reports Stage 1 `PENDING` with "Waiting on the change is a normal state", and nothing nags. If after
+the map the user never wants a change, only audits or a regression net, the mode was wrong: switch to
+`existing-app-assurance.md`, whose Track A starts from the same report.
+
+**Stage 0b — when the change arrives:** scope the slice, then compute its blast radius *from the facts
+already collected* (below), not from scratch.
+
+---
+
 ## Where the knowledge base comes from — Path D
 
 `conversion-runbook.md` Stage 1 names three paths: **A** code → AST extractors, **B** documents → LLM
@@ -103,7 +138,10 @@ Still record the counts. `SHOW MODULES` returning 6 when the person who asked fo
 In every other mode, Stage 0 asks *what do we build first*. Here it asks a second question that has no
 equivalent elsewhere: **what does this change touch that nobody mentioned?**
 
-Work outward from the entities the change affects:
+Work outward from the entities the change affects. Start from the map (Stage 0a): a module in
+`analysis/app-facts/dependencies.json` → `tangles` puts the whole tangle in the radius, and `edges`
+names who depends on the module you change. The queries below confirm and extend that; they do not
+repeat it.
 
 1. **Associations** — `SHOW ASSOCIATIONS`, then `DESCRIBE ENTITY` each end. An entity you are changing
    that is on the far end of an association from a module nobody mentioned is your blast radius.
@@ -111,8 +149,9 @@ Work outward from the entities the change affects:
    A change to a validation rule lands in every flow that writes the entity, not only the one on the
    screen you were shown.
 3. **Pages bound to them** — `SHOW PAGES IN `, and check which are bound to the entity.
-4. **Module dependencies** — `mxcli graph-report`. If the module you are changing is in a tangle, the
-   blast radius is the tangle.
+4. **Module dependencies** — already in the map (`dependencies.json`); rerun `bin/app-facts.sh` only
+   if the model changed since. If the module you are changing is in a tangle, the blast radius is the
+   tangle.
 5. **Published and consumed services** — a changed entity behind a published REST service is a
    contract change, and the consumer is not in the `.mpr`.
 
@@ -126,8 +165,8 @@ least able to give you — they know what they want, not what it touches.
 
 | Stage | Runs? | What changes in this mode |
 |---|---|---|
-| **P — Kickoff** | Yes, light | `bin/init-project.sh` as normal. Many intake questions are already answered by the app existing — answer them from the model, not by asking. Record entry mode `Change an existing app` `CONFIRMED` in `PROJECT.md`. |
-| **0 — Triage & Scope ✋** | **Yes, always** | Two questions, not one: which slice, and its blast radius (above). The Coverage Matrix's *extraction* rows are N/A — Path D has no extractor to choose. The Business Capability Map is built from `SHOW MODULES` + the change request. **CAC-1 runs.** |
+| **P — Kickoff** | Yes, light | `bin/init-project.sh` as normal — **with `./mxcli` already in the project root**, because init runs `mxcli init` and skips it when the binary is missing. Many intake questions are already answered by the app existing — answer them from the model, not by asking. The scope question is open: the change may not be known yet. Record entry mode `Change an existing app` `CONFIRMED` in `PROJECT.md`. |
+| **0 — Triage & Scope ✋** | **Yes, always, in two halves** | **0a:** map the app and disposition its findings with the user (above) — owed as the `app-report` artifact. May park here. **0b**, once the change is named: which slice, and its blast radius (above), read from the map. The Coverage Matrix's *extraction* rows are N/A — Path D has no extractor to choose. The Business Capability Map is built from `SHOW MODULES` + the change request. **CAC-1 runs.** |
 | **1 — Analysis** | Yes, **Path D** | Query the model into the knowledge base, scoped to the slice **plus** its blast radius. Path A is declared not-applicable with attribution, not "skipped". Path C (SME) matters more here than anywhere: the model tells you what the app does and nobody wrote down why. **CAC-1b runs** — its scope-out diff is the slice-vs-app statement. |
 | **2 — Requirements** | Yes, **slice only** | One BRD per capability *being changed*, each carrying **as-is** and **to-be**. Do not BRD untouched capabilities; record explicitly that you did not, and why. **CAC-2 and CAC-3 run.** |
 | **3 — Architecture & Design ✋** | **Conditionally** | Run it in full if the change crosses module boundaries, adds an integration, or alters the domain model. Otherwise it collapses to: which existing module owns this, and does that still hold. **Never invent new module boundaries for an app that already has them** — `modularize-domain.md` is being used to *check* a boundary here, not to draw one. Wireframes only for screens that change; the design system is the app's existing styling, captured, not designed. **CAC-4 runs, scoped to what changes.** |
@@ -184,6 +223,7 @@ it is the only entry mode where that is true.
 
 - `PROJECT.md` — gate register: entry mode `Change an existing app` `CONFIRMED`, blast radius confirmed, Stage 7 marked N/A with its reason.
 - `docs/brain/` — `mxcli brain init` run once; the slice under `plan/`, and the as-is facts nobody wrote down captured as decisions anchored to the modules they are about. `brain check` green at every commit.
+- `analysis/app-report.html` + `.json` — the map of the whole app (`app-analysis.md`), with every top finding dispositioned in `PROJECT.md`.
 - `triage.md` — slice + blast radius, signed off.
 - Knowledge base — Path D, scoped, with counts recorded: modules, and per in-scope module the entity, page and microflow totals `SHOW …` returned, so a reader can see the slice against the app.
 - BRDs for the changed slice, each with as-is and to-be — and one line saying how many capabilities were *not* BRD'd and why.
diff --git a/tests/wave2/test-bug03-gates.sh b/tests/wave2/test-bug03-gates.sh
index 79af95cb..c8a81f70 100755
--- a/tests/wave2/test-bug03-gates.sh
+++ b/tests/wave2/test-bug03-gates.sh
@@ -1,6 +1,7 @@
 #!/usr/bin/env bash
 # Fixture for wave-2 #3: the unanchored substring CONTENT gates — Stage 0 (triage sign-off),
-# Stage 2 (validation stop condition) and Stage 7 (cutover decision row).
+# Stage 2 (validation stop condition) and Stage 7 (cutover decision row); T12 adds the
+# existing-app-change parked state (Stage 0) and that mode's Stage 1 hint.
 #
 # Stage P is covered by test-stage-p.sh and is deliberately not retested here.
 #
@@ -174,6 +175,38 @@ V="$(verdict "$P" 7)"
 case "$V" in *WAIVED*) ok "the short token 'existing-app' still waives stage 7" ;;
              *) bad "existing-app short token no longer waives stage 7: $V" ;; esac
 
+
+echo "== T12: existing-app mode — a mapped app waiting on its change is PENDING, not FAIL =="
+# existing-app-change.md §"Map the app first": the map runs before any slice is named, and the
+# project may park there. The parked arm is narrow: this mode, the untouched placeholder, AND a
+# rendered map. Drop any one and the old FAIL must come back.
+mkexisting() {
+  d="$WORK/$1"; mkdir -p "$d/analysis"
+  printf 'Toolkit commit: none\n\n| Stage | Decision | Status | Notes |\n|---|---|---|---|\n\nEntry mode: %s\n' "$2" > "$d/PROJECT.md"
+  printf '# Triage\n\n## Sign-off\n\nConfirmed by: [user] on [date]\n' > "$d/triage.md"
+  echo "$d"
+}
+P="$(mkexisting t12-parked 'Change an existing app')"; printf '{}\n' > "$P/analysis/app-report.json"
+V="$(verdict "$P" 0)"
+case "$V" in *PENDING*'waiting on the change'*) ok "mapped + unsigned triage in this mode reads PENDING, parked" ;;
+             *) bad "parked existing-app project not PENDING: $V" ;; esac
+V="$(verdict "$P" 1)"
+case "$V" in *'Path D'*) ok "Stage 1 hint names the live-model path in this mode" ;;
+             *) bad "Stage 1 hint still points at extractors in existing-app mode: $V" ;; esac
+
+P="$(mkexisting t12-unmapped 'Change an existing app')"
+V="$(verdict "$P" 0)"
+case "$V" in *FAIL*) ok "no map yet: the placeholder still FAILs in this mode" ;;
+             *) bad "unmapped existing-app project escaped the placeholder FAIL: $V" ;; esac
+
+P="$(mkexisting t12-migration 'Migration')"; printf '{}\n' > "$P/analysis/app-report.json"
+V="$(verdict "$P" 0)"
+case "$V" in *FAIL*) ok "another mode with a stray app-report.json still FAILs the placeholder" ;;
+             *) bad "the parked arm leaked into migration mode: $V" ;; esac
+V="$(verdict "$P" 1)"
+case "$V" in *'Path D'*) bad "migration project got the existing-app Stage 1 hint: $V" ;;
+             *) ok "other modes keep the extractor / kb-generation hint" ;; esac
+
 printf '\n%s: %d ok, %d FAIL\n' "$(basename "$0")" "$PASS" "$FAIL"
 rm -rf "$WORK"
 [ "$FAIL" -eq 0 ]


     
StepWhat it doesScripts and commands

     
P lightScaffold. Intake answers come from the model, not from the user's memory. Run the machine check once.bin/doctor.sh
bin/init-project.sh <project>
Map the appGive the agent a picture of the whole app before scoping: module graph and dependency tangle, current quality score, lint findings, security matrix, marketplace modules that drifted from their published version. The findings feed the blast-radius answer and the regression baseline../mxcli -p app.mpr -c "SHOW MODULES"
./mxcli -p app.mpr -c "SHOW STRUCTURE DEPTH 2"
./mxcli graph-report -p app.mpr (graph-analysis.md)
./mxcli lint -p app.mpr
./mxcli report -p app.mpr --format html
./mxcli -p app.mpr -c "SHOW SECURITY MATRIX"
./mxcli marketplace diff <content-id> -p app.mpr (v0.18+)
0 Triage ✋Two questions: which slice, and what is its blast radius. The radius is written into triage.md as its own section, with counts. Capability map from the module list. CAC-1 and sign-off.SHOW ASSOCIATIONS IN <Module> · DESCRIBE ENTITY M.E
SEARCH '<entity>' · SHOW PAGES IN <Module>
SHOW REFERENCES OF M.E · SHOW IMPACT OF M.E
./mxcli graph-report for the tangle
bin/gate-check.sh <project> 0
0a Map the appRun, do not ask: it is read-only and takes about a minute. Inventory, module tangles, loop-risk microflows, dead elements, with a fix-first list. Show the report and ask the user to disposition each top finding (fix in this work / log / accept); answers go to PROJECT.md. The project may then park until a change is named. Owed in this mode as the app-report artifact. Lint, security matrix and marketplace drift fill the report's Security and Lint sections.bin/app-facts.sh (app-analysis.md)
bin/app-report.sh <project>
./mxcli lint -p app.mpr
./mxcli -p app.mpr -c "SHOW SECURITY MATRIX"
./mxcli marketplace diff <content-id> -p app.mpr (v0.18+)
0b Triage ✋Once the change is named: which slice, and what is its blast radius, starting from the map's tangles and edges. The radius is written into triage.md as its own section, with counts. Capability map from the module list. CAC-1 and sign-off.SHOW ASSOCIATIONS IN <Module> · DESCRIBE ENTITY M.E
SEARCH '<entity>' · SHOW PAGES IN <Module>
SHOW REFERENCES OF M.E · SHOW IMPACT OF M.E
analysis/app-facts/dependencies.json for the tangle
bin/gate-check.sh <project> 0

     
Regression netTrack B from the assurance skill, before any change: harness, action inventory, one journey per action, DB assertions, wiring sweep, LOOK pass, committed baseline.see Track B below
project-bin/coverage-preflight.sh --assess --module <M>

     
1 Path DQuery the model into the knowledge base, scoped to slice plus blast radius. Record counts. Path A declared N/A with attribution. Path C matters most here: the people who know why the app is the way it is.SHOW ENTITIES IN · DESCRIBE ENTITY · SHOW MICROFLOWS IN
DESCRIBE MICROFLOW · DESCRIBE PAGE
bin/extraction-report.sh <project>
bin/gate-check.sh <project> 1

     
2 RequirementsOne as-is / to-be BRD per capability being changed. Nothing for the untouched rest.bin/brd-report.sh · bin/open-questions.sh --stage 2