feat(compose-agent): write-mode acceptance evals + CI validation (4.4.0)

CHANGELOG.md

@@ -2,6 +2,17 @@
 
 Full release history for the Compose Skill Suite. The newest release is summarised under **What's new** in the [README](./README.md).
 
+### 4.4.0 — 2026-07-07
+
+**`compose-agent` — write-mode acceptance evals.**
+
+Closes the last follow-up from the 4.3.1 cross-review: the authoring skill had no evals of its own, so its new rules were only exercised through the audit's acceptance cases.
+
+- **New suite.** `skills/compose-agent/evals/evals.json` — five write-mode cases that ask the skill to *write* Compose and grade the result. Cases 0–2 lock in the 4.3.x authoring rules (cross-phase back-write avoidance, Strong Skipping false-lead avoidance, snapshot self-invalidation avoidance); cases 3–4 cover phase-correct reads and lifecycle-aware Flow collection. Unlike the audit evals (which grade an audit of existing code), these grade generated code.
+- **CI enforcement.** `bin/ci` now structurally validates every `skills/<name>/evals/evals.json` (JSON, unique numeric ids, required fields, non-empty expectations) and asserts the compose-agent suite keeps covering the three 4.3.x authoring rules — so the authoring path can't silently drift from the audit scoring path.
+- **`SKILL.md`.** New *Acceptance Evals* section pointing at the suite.
+- **Versions.** `compose-agent` → `4.4.0`. `jetpack-compose-audit` unchanged at `4.3.2`.
+
 ### 4.3.2 — 2026-07-07
 
 **Patch — one missed Strong Skipping caveat.**

README.md

@@ -13,7 +13,7 @@
   <img alt="Claude Code plugin" src="https://img.shields.io/badge/Claude%20Code-plugin-111827">
 </p>
 
-**`compose-agent` 4.3.2 · 2026-07-07** — Authoring guidance so the coding-agent skill *warns while you write*: **Never Back-Write Across Phases** (layout callbacks / snapshot-collection mutation feeding composition) and **Optimizations That Do Nothing** (the false leads). The 4.3.0 audit catches these after the fact; 4.3.1 stops you writing them.
+**`compose-agent` 4.4.0 · 2026-07-07** — Authoring guidance so the coding-agent skill *warns while you write* — **Never Back-Write Across Phases** and **Optimizations That Do Nothing** — now backed by its own [write-mode acceptance evals](./skills/compose-agent/evals/evals.json) (CI-enforced to track the audit's rules). The 4.3.0 audit catches these after the fact; `compose-agent` stops you writing them.
 
 **`jetpack-compose-audit` 4.3.2 · 2026-07-07** — Cross-phase back-write detection (axis 3: layout callbacks writing state read in composition), a related composition-phase self-invalidation check (snapshot collections mutated in a composable body), and a **False Leads** scoring guard so the auditor stops crediting no-op "recomposition fixes." Adapted from [`chrisbanes/skills`](https://github.com/chrisbanes/skills) (Apache-2.0).
 
@@ -31,6 +31,12 @@ Authored and cross-reviewed with every frontier model — Claude Opus 4.8, GPT-5
 
 ## What's new
 
+### 4.4.0 — 2026-07-07
+
+**`compose-agent` — write-mode acceptance evals.** New [`skills/compose-agent/evals/evals.json`](./skills/compose-agent/evals/evals.json): five cases that ask the skill to *write* Compose and grade the output (cross-phase back-write avoidance, Strong Skipping false leads, snapshot self-invalidation, phase-correct reads, lifecycle-aware Flow collection). `bin/ci` now validates every eval suite's structure and asserts the compose-agent cases keep covering the 4.3.x authoring rules — so the authoring path can't drift from the audit path. `compose-agent` → `4.4.0`; `jetpack-compose-audit` stays `4.3.2`.
+
+For release detail, see [`docs/release-notes-4.4.0.md`](./docs/release-notes-4.4.0.md).
+
 ### 4.3.2 — 2026-07-07
 
 **Patch.** Fixed one Strong Skipping caveat in `jetpack-compose-audit/scoring.md` that the 4.3.1 sweep missed (its "opt-out path" wording dodged the string match): `@NonSkippableComposable` opts a composable out of *skipping* but does not disable *lambda memoization* — only `@DontMemoize` / SSM-off does. Both skills → `4.3.2`.

bin/ci

@@ -162,6 +162,94 @@ const auditVersion = JSON.parse(readFileSync('skills/jetpack-compose-audit/.clau
 if (!auditSkill.includes(`**Skill version:** ${auditVersion}`)) {
   throw new Error('jetpack-compose-audit SKILL.md version does not match manifest version');
 }
+
+// Eval suites: structural validation for every skills/<name>/evals/evals.json.
+const evalSuites = new Map();
+for (const evalFile of [...files].filter((file) => file.endsWith('/evals/evals.json'))) {
+  const skillName = evalFile.split('/')[1];
+  let suite;
+  try {
+    suite = JSON.parse(readFileSync(evalFile, 'utf8'));
+  } catch (error) {
+    throw new Error(`${evalFile} is not valid JSON: ${error.message}`);
+  }
+  if (suite.skill_name !== skillName) {
+    throw new Error(`${evalFile} skill_name must be "${skillName}", got "${suite.skill_name}"`);
+  }
+  if (!Array.isArray(suite.evals) || suite.evals.length === 0) {
+    throw new Error(`${evalFile} must have a non-empty "evals" array`);
+  }
+  const seenIds = new Set();
+  for (const item of suite.evals) {
+    if (typeof item.id !== 'number' || seenIds.has(item.id)) {
+      throw new Error(`${evalFile} eval ids must be unique numbers (offender: ${JSON.stringify(item.id)})`);
+    }
+    seenIds.add(item.id);
+    for (const field of ['prompt', 'expected_output']) {
+      if (typeof item[field] !== 'string' || !item[field].trim()) {
+        throw new Error(`${evalFile} eval ${item.id} is missing a non-empty "${field}"`);
+      }
+    }
+    if (!Array.isArray(item.files)) {
+      throw new Error(`${evalFile} eval ${item.id} must have a "files" array`);
+    }
+    if (!Array.isArray(item.expectations) || item.expectations.length === 0) {
+      throw new Error(`${evalFile} eval ${item.id} must have a non-empty "expectations" array`);
+    }
+    if (!item.expectations.every((expectation) => typeof expectation === 'string' && expectation.trim())) {
+      throw new Error(`${evalFile} eval ${item.id} expectations must all be non-empty strings`);
+    }
+  }
+  evalSuites.set(skillName, suite);
+}
+
+// compose-agent write-mode evals must keep pace with the 4.3.x authoring rules,
+// so the authoring path can't silently drift from the audit scoring path. Checked
+// per eval CASE (not whole-file text), so the suite description can't satisfy them.
+const composeSuite = evalSuites.get('compose-agent');
+if (!composeSuite) {
+  throw new Error('compose-agent must ship write-mode evals at skills/compose-agent/evals/evals.json');
+}
+// The three 4.3.x authoring rules, each with the token that identifies its case.
+const authoringRules = [
+  ['cross-phase back-write', /onSizeChanged|onGloballyPositioned|onPlaced|cross-phase/],
+  ['Strong Skipping false lead', /Strong Skipping[\s\S]*memoiz|memoiz[\s\S]*Strong Skipping/i],
+  ['snapshot self-invalidation', /mutableStateMapOf|mutableStateListOf|self-invalidation/],
+];
+// Each rule must be covered by its own DISTINCT case — so one case stuffed with all
+// three keywords can't fake coverage.
+const usedIds = new Set();
+const matchedByRule = new Map();
+for (const [label, needle] of authoringRules) {
+  const match = composeSuite.evals.find((item) => !usedIds.has(item.id) && needle.test(JSON.stringify(item)));
+  if (!match) {
+    throw new Error(`compose-agent evals must include a distinct case covering the ${label} authoring rule`);
+  }
+  usedIds.add(match.id);
+  matchedByRule.set(label, match);
+}
+// The Strong Skipping false-lead case must cover BOTH subcases: the auto-memoized
+// lambda AND the remember(index) pure-expression no-op. Checked on the same case the
+// Strong Skipping rule matched above, not a fresh lookup.
+const falseLeadCase = matchedByRule.get('Strong Skipping false lead');
+if (!/remember\(index\)|remember\(i\)/.test(JSON.stringify(falseLeadCase))) {
+  throw new Error('compose-agent Strong Skipping false-lead case must cover both the auto-memoized-lambda and the remember(index) pure-expression subcases');
+}
+// Bidirectional lockstep: the audit must keep cases 12-14 AND they must still cover
+// the same three rules (an id alone is not enough).
+const auditSuite = evalSuites.get('jetpack-compose-audit');
+if (auditSuite) {
+  const auditMirror = [[12, authoringRules[0]], [13, authoringRules[1]], [14, authoringRules[2]]];
+  for (const [id, [label, needle]] of auditMirror) {
+    const auditCase = auditSuite.evals.find((item) => item.id === id);
+    if (!auditCase) {
+      throw new Error(`jetpack-compose-audit evals must retain case ${id} (the audit side of the compose-agent write-mode mirror)`);
+    }
+    if (!needle.test(JSON.stringify(auditCase))) {
+      throw new Error(`jetpack-compose-audit case ${id} must still cover the ${label} rule (mirror of compose-agent write-mode evals)`);
+    }
+  }
+}
 NODE
 
 if command -v claude >/dev/null 2>&1; then

docs/release-notes-4.3.1.md

@@ -26,7 +26,7 @@ The same correction lands in `compose-agent/performance.md`: the *Strong Skippin
 
 ## Eval coverage
 
-The authoring rules share the audit acceptance evals (`jetpack-compose-audit/evals/evals.json` cases 12–14) — `compose-agent` mirrors the same rubric. A dedicated compose-agent *write-mode* eval harness is a tracked follow-up.
+The authoring rules share the audit acceptance evals (`jetpack-compose-audit/evals/evals.json` cases 12–14) — `compose-agent` mirrors the same rubric. A dedicated compose-agent *write-mode* eval harness was **delivered in 4.4.0** (`skills/compose-agent/evals/evals.json`, cases 0–2 mirror audit 12–14, CI-locked).
 
 ## Attribution
 

docs/release-notes-4.4.0.md

@@ -0,0 +1,32 @@
+# Release notes — 4.4.0 (2026-07-07)
+
+**`compose-agent` only.** `jetpack-compose-audit` unchanged at `4.3.2`.
+
+Closes the last follow-up flagged across the 4.3.x cross-reviews: `compose-agent` had no evals of its own, so its authoring rules were only ever exercised through the *audit's* acceptance cases (which grade an audit of existing code, not code the skill writes).
+
+## New: write-mode acceptance evals
+
+`skills/compose-agent/evals/evals.json` — five cases that hand the skill a *writing* task and grade the Compose it produces:
+
+- **Case 0 — cross-phase back-write avoidance.** Align a value to a label's measured width without writing `onSizeChanged` state read in composition; expects a layout-phase solution.
+- **Case 1 — Strong Skipping false leads.** Make a row skip under SSM; expects *no* `remember`-wrapped callbacks and *no* `remember(index)` on pure expressions — real levers only (stable types, keys, verified with recomposition counts).
+- **Case 2 — snapshot self-invalidation avoidance.** Build a per-entry lookup with `remember(entries)`, not by mutating a `mutableStateMapOf` during composition.
+- **Case 3 — phase-correct reads.** Feed a per-frame `Float` through a `() -> Float` provider or a layout/draw-phase read, keeping composition stable.
+- **Case 4 — lifecycle-aware collection.** Collect a `StateFlow` with `collectAsStateWithLifecycle()`, side-effect free, modifier-hygienic.
+
+Cases 0–2 mirror the `jetpack-compose-audit` scoring rules 1:1, so the authoring path and the audit path stay in lockstep.
+
+## CI enforcement
+
+`bin/ci` now:
+
+- structurally validates **every** `skills/<name>/evals/evals.json` — valid JSON, unique numeric ids, non-empty `prompt` / `expected_output`, `files` array, non-empty `expectations`;
+- asserts the compose-agent suite keeps covering the three 4.3.x authoring rules (cross-phase back-write, Strong Skipping false lead, snapshot self-invalidation), so a future edit can't quietly drop coverage.
+
+## Note
+
+These evals are acceptance data, not a runner — run a model with the skill loaded against each prompt and check every expectation, the same way `jetpack-compose-audit/evals/evals.json` is used.
+
+## Versions
+
+`compose-agent` → **4.4.0** (SKILL + both `plugin.json`). `jetpack-compose-audit` stays at `4.3.2`.

skills/compose-agent/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "compose-agent",
-  "version": "4.3.2",
+  "version": "4.4.0",
   "description": "Agent skill that helps AI coding assistants write modern Jetpack Compose: correct state, effects, performance-aware modifiers, Navigation 3, Paging 3 in Compose, coroutines on lifecycle, animations, UI tests, focus/keyboard navigation, Compose Multiplatform boundaries, Android launch resources, and idiomatic Kotlin. Targets the mistakes LLMs actually make in Compose code.",
   "author": {
     "name": "Ivan Morgillo"

skills/compose-agent/.cursor-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "compose-agent",
-  "version": "4.3.2",
+  "version": "4.4.0",
   "description": "Agent skill that helps AI coding assistants write modern Jetpack Compose: correct state, effects, performance-aware modifiers, Navigation 3, Paging 3 in Compose, coroutines on lifecycle, animations, UI tests, focus/keyboard navigation, Compose Multiplatform boundaries, Android launch resources, and idiomatic Kotlin. Targets the mistakes LLMs actually make in Compose code.",
   "author": {
     "name": "Ivan Morgillo"

skills/compose-agent/SKILL.md

@@ -6,7 +6,7 @@ allowed-tools: Read, Glob, Grep, Edit, Write, Bash
 argument-hint: "[focus area, e.g. 'state', 'effects', 'navigation', 'paging', 'lifecycle', 'animation', 'testing', 'focus', 'kmp']"
 metadata:
   author: Ivan Morgillo
-  version: "4.3.2"
+  version: "4.4.0"
 ---
 
 # Compose Agent
@@ -178,6 +178,10 @@ If the user needs any of the above, narrow the scope and say so.
 - `references/paging.md` — Paging 3 in Compose: when to page vs plain lists, `collectAsLazyPagingItems`, stable `itemKey`, `LoadState` UI, pull-to-refresh, anti-patterns for paginated lazy lists.
 - `references/kotlin.md` — Kotlin coding conventions and Android Kotlin style the LLM keeps missing.
 
+## Acceptance Evals
+
+`evals/evals.json` holds write-mode acceptance cases — prompts that ask this skill to *write* Compose, plus the expectations the produced code must satisfy. Cases 0–2 mirror the `jetpack-compose-audit` scoring rules 1:1 (cross-phase back-writes, Strong Skipping false leads, snapshot self-invalidation) — `bin/ci` keeps them in lockstep so the authoring path can't drift from the audit path; cases 3–4 add foundational phase-correct reads and lifecycle-aware Flow collection. Run a model with this skill loaded against each prompt and check every expectation.
+
 ## Primary Sources
 
 Every rule in this skill traces back to one of: