diff --git a/CHANGELOG.md b/CHANGELOG.md index 553f132..b867ae7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,17 @@ Full release history for the Compose Skill Suite. The newest release is summarised under **What's new** in the [README](./README.md). +### 4.4.0 — 2026-07-07 + +**`compose-agent` — write-mode acceptance evals.** + +Closes the last follow-up from the 4.3.1 cross-review: the authoring skill had no evals of its own, so its new rules were only exercised through the audit's acceptance cases. + +- **New suite.** `skills/compose-agent/evals/evals.json` — five write-mode cases that ask the skill to *write* Compose and grade the result. Cases 0–2 lock in the 4.3.x authoring rules (cross-phase back-write avoidance, Strong Skipping false-lead avoidance, snapshot self-invalidation avoidance); cases 3–4 cover phase-correct reads and lifecycle-aware Flow collection. Unlike the audit evals (which grade an audit of existing code), these grade generated code. +- **CI enforcement.** `bin/ci` now structurally validates every `skills//evals/evals.json` (JSON, unique numeric ids, required fields, non-empty expectations) and asserts the compose-agent suite keeps covering the three 4.3.x authoring rules — so the authoring path can't silently drift from the audit scoring path. +- **`SKILL.md`.** New *Acceptance Evals* section pointing at the suite. +- **Versions.** `compose-agent` → `4.4.0`. `jetpack-compose-audit` unchanged at `4.3.2`. + ### 4.3.2 — 2026-07-07 **Patch — one missed Strong Skipping caveat.** diff --git a/README.md b/README.md index 16c3b0b..bde2515 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ Claude Code plugin

-**`compose-agent` 4.3.2 · 2026-07-07** — Authoring guidance so the coding-agent skill *warns while you write*: **Never Back-Write Across Phases** (layout callbacks / snapshot-collection mutation feeding composition) and **Optimizations That Do Nothing** (the false leads). The 4.3.0 audit catches these after the fact; 4.3.1 stops you writing them. +**`compose-agent` 4.4.0 · 2026-07-07** — Authoring guidance so the coding-agent skill *warns while you write* — **Never Back-Write Across Phases** and **Optimizations That Do Nothing** — now backed by its own [write-mode acceptance evals](./skills/compose-agent/evals/evals.json) (CI-enforced to track the audit's rules). The 4.3.0 audit catches these after the fact; `compose-agent` stops you writing them. **`jetpack-compose-audit` 4.3.2 · 2026-07-07** — Cross-phase back-write detection (axis 3: layout callbacks writing state read in composition), a related composition-phase self-invalidation check (snapshot collections mutated in a composable body), and a **False Leads** scoring guard so the auditor stops crediting no-op "recomposition fixes." Adapted from [`chrisbanes/skills`](https://github.com/chrisbanes/skills) (Apache-2.0). @@ -31,6 +31,12 @@ Authored and cross-reviewed with every frontier model — Claude Opus 4.8, GPT-5 ## What's new +### 4.4.0 — 2026-07-07 + +**`compose-agent` — write-mode acceptance evals.** New [`skills/compose-agent/evals/evals.json`](./skills/compose-agent/evals/evals.json): five cases that ask the skill to *write* Compose and grade the output (cross-phase back-write avoidance, Strong Skipping false leads, snapshot self-invalidation, phase-correct reads, lifecycle-aware Flow collection). `bin/ci` now validates every eval suite's structure and asserts the compose-agent cases keep covering the 4.3.x authoring rules — so the authoring path can't drift from the audit path. `compose-agent` → `4.4.0`; `jetpack-compose-audit` stays `4.3.2`. + +For release detail, see [`docs/release-notes-4.4.0.md`](./docs/release-notes-4.4.0.md). + ### 4.3.2 — 2026-07-07 **Patch.** Fixed one Strong Skipping caveat in `jetpack-compose-audit/scoring.md` that the 4.3.1 sweep missed (its "opt-out path" wording dodged the string match): `@NonSkippableComposable` opts a composable out of *skipping* but does not disable *lambda memoization* — only `@DontMemoize` / SSM-off does. Both skills → `4.3.2`. diff --git a/bin/ci b/bin/ci index 7487da7..706ad1c 100755 --- a/bin/ci +++ b/bin/ci @@ -162,6 +162,94 @@ const auditVersion = JSON.parse(readFileSync('skills/jetpack-compose-audit/.clau if (!auditSkill.includes(`**Skill version:** ${auditVersion}`)) { throw new Error('jetpack-compose-audit SKILL.md version does not match manifest version'); } + +// Eval suites: structural validation for every skills//evals/evals.json. +const evalSuites = new Map(); +for (const evalFile of [...files].filter((file) => file.endsWith('/evals/evals.json'))) { + const skillName = evalFile.split('/')[1]; + let suite; + try { + suite = JSON.parse(readFileSync(evalFile, 'utf8')); + } catch (error) { + throw new Error(`${evalFile} is not valid JSON: ${error.message}`); + } + if (suite.skill_name !== skillName) { + throw new Error(`${evalFile} skill_name must be "${skillName}", got "${suite.skill_name}"`); + } + if (!Array.isArray(suite.evals) || suite.evals.length === 0) { + throw new Error(`${evalFile} must have a non-empty "evals" array`); + } + const seenIds = new Set(); + for (const item of suite.evals) { + if (typeof item.id !== 'number' || seenIds.has(item.id)) { + throw new Error(`${evalFile} eval ids must be unique numbers (offender: ${JSON.stringify(item.id)})`); + } + seenIds.add(item.id); + for (const field of ['prompt', 'expected_output']) { + if (typeof item[field] !== 'string' || !item[field].trim()) { + throw new Error(`${evalFile} eval ${item.id} is missing a non-empty "${field}"`); + } + } + if (!Array.isArray(item.files)) { + throw new Error(`${evalFile} eval ${item.id} must have a "files" array`); + } + if (!Array.isArray(item.expectations) || item.expectations.length === 0) { + throw new Error(`${evalFile} eval ${item.id} must have a non-empty "expectations" array`); + } + if (!item.expectations.every((expectation) => typeof expectation === 'string' && expectation.trim())) { + throw new Error(`${evalFile} eval ${item.id} expectations must all be non-empty strings`); + } + } + evalSuites.set(skillName, suite); +} + +// compose-agent write-mode evals must keep pace with the 4.3.x authoring rules, +// so the authoring path can't silently drift from the audit scoring path. Checked +// per eval CASE (not whole-file text), so the suite description can't satisfy them. +const composeSuite = evalSuites.get('compose-agent'); +if (!composeSuite) { + throw new Error('compose-agent must ship write-mode evals at skills/compose-agent/evals/evals.json'); +} +// The three 4.3.x authoring rules, each with the token that identifies its case. +const authoringRules = [ + ['cross-phase back-write', /onSizeChanged|onGloballyPositioned|onPlaced|cross-phase/], + ['Strong Skipping false lead', /Strong Skipping[\s\S]*memoiz|memoiz[\s\S]*Strong Skipping/i], + ['snapshot self-invalidation', /mutableStateMapOf|mutableStateListOf|self-invalidation/], +]; +// Each rule must be covered by its own DISTINCT case — so one case stuffed with all +// three keywords can't fake coverage. +const usedIds = new Set(); +const matchedByRule = new Map(); +for (const [label, needle] of authoringRules) { + const match = composeSuite.evals.find((item) => !usedIds.has(item.id) && needle.test(JSON.stringify(item))); + if (!match) { + throw new Error(`compose-agent evals must include a distinct case covering the ${label} authoring rule`); + } + usedIds.add(match.id); + matchedByRule.set(label, match); +} +// The Strong Skipping false-lead case must cover BOTH subcases: the auto-memoized +// lambda AND the remember(index) pure-expression no-op. Checked on the same case the +// Strong Skipping rule matched above, not a fresh lookup. +const falseLeadCase = matchedByRule.get('Strong Skipping false lead'); +if (!/remember\(index\)|remember\(i\)/.test(JSON.stringify(falseLeadCase))) { + throw new Error('compose-agent Strong Skipping false-lead case must cover both the auto-memoized-lambda and the remember(index) pure-expression subcases'); +} +// Bidirectional lockstep: the audit must keep cases 12-14 AND they must still cover +// the same three rules (an id alone is not enough). +const auditSuite = evalSuites.get('jetpack-compose-audit'); +if (auditSuite) { + const auditMirror = [[12, authoringRules[0]], [13, authoringRules[1]], [14, authoringRules[2]]]; + for (const [id, [label, needle]] of auditMirror) { + const auditCase = auditSuite.evals.find((item) => item.id === id); + if (!auditCase) { + throw new Error(`jetpack-compose-audit evals must retain case ${id} (the audit side of the compose-agent write-mode mirror)`); + } + if (!needle.test(JSON.stringify(auditCase))) { + throw new Error(`jetpack-compose-audit case ${id} must still cover the ${label} rule (mirror of compose-agent write-mode evals)`); + } + } +} NODE if command -v claude >/dev/null 2>&1; then diff --git a/docs/release-notes-4.3.1.md b/docs/release-notes-4.3.1.md index dad5c86..01d9716 100644 --- a/docs/release-notes-4.3.1.md +++ b/docs/release-notes-4.3.1.md @@ -26,7 +26,7 @@ The same correction lands in `compose-agent/performance.md`: the *Strong Skippin ## Eval coverage -The authoring rules share the audit acceptance evals (`jetpack-compose-audit/evals/evals.json` cases 12–14) — `compose-agent` mirrors the same rubric. A dedicated compose-agent *write-mode* eval harness is a tracked follow-up. +The authoring rules share the audit acceptance evals (`jetpack-compose-audit/evals/evals.json` cases 12–14) — `compose-agent` mirrors the same rubric. A dedicated compose-agent *write-mode* eval harness was **delivered in 4.4.0** (`skills/compose-agent/evals/evals.json`, cases 0–2 mirror audit 12–14, CI-locked). ## Attribution diff --git a/docs/release-notes-4.4.0.md b/docs/release-notes-4.4.0.md new file mode 100644 index 0000000..157ec41 --- /dev/null +++ b/docs/release-notes-4.4.0.md @@ -0,0 +1,32 @@ +# Release notes — 4.4.0 (2026-07-07) + +**`compose-agent` only.** `jetpack-compose-audit` unchanged at `4.3.2`. + +Closes the last follow-up flagged across the 4.3.x cross-reviews: `compose-agent` had no evals of its own, so its authoring rules were only ever exercised through the *audit's* acceptance cases (which grade an audit of existing code, not code the skill writes). + +## New: write-mode acceptance evals + +`skills/compose-agent/evals/evals.json` — five cases that hand the skill a *writing* task and grade the Compose it produces: + +- **Case 0 — cross-phase back-write avoidance.** Align a value to a label's measured width without writing `onSizeChanged` state read in composition; expects a layout-phase solution. +- **Case 1 — Strong Skipping false leads.** Make a row skip under SSM; expects *no* `remember`-wrapped callbacks and *no* `remember(index)` on pure expressions — real levers only (stable types, keys, verified with recomposition counts). +- **Case 2 — snapshot self-invalidation avoidance.** Build a per-entry lookup with `remember(entries)`, not by mutating a `mutableStateMapOf` during composition. +- **Case 3 — phase-correct reads.** Feed a per-frame `Float` through a `() -> Float` provider or a layout/draw-phase read, keeping composition stable. +- **Case 4 — lifecycle-aware collection.** Collect a `StateFlow` with `collectAsStateWithLifecycle()`, side-effect free, modifier-hygienic. + +Cases 0–2 mirror the `jetpack-compose-audit` scoring rules 1:1, so the authoring path and the audit path stay in lockstep. + +## CI enforcement + +`bin/ci` now: + +- structurally validates **every** `skills//evals/evals.json` — valid JSON, unique numeric ids, non-empty `prompt` / `expected_output`, `files` array, non-empty `expectations`; +- asserts the compose-agent suite keeps covering the three 4.3.x authoring rules (cross-phase back-write, Strong Skipping false lead, snapshot self-invalidation), so a future edit can't quietly drop coverage. + +## Note + +These evals are acceptance data, not a runner — run a model with the skill loaded against each prompt and check every expectation, the same way `jetpack-compose-audit/evals/evals.json` is used. + +## Versions + +`compose-agent` → **4.4.0** (SKILL + both `plugin.json`). `jetpack-compose-audit` stays at `4.3.2`. diff --git a/skills/compose-agent/.claude-plugin/plugin.json b/skills/compose-agent/.claude-plugin/plugin.json index 96b4634..822dd14 100644 --- a/skills/compose-agent/.claude-plugin/plugin.json +++ b/skills/compose-agent/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "compose-agent", - "version": "4.3.2", + "version": "4.4.0", "description": "Agent skill that helps AI coding assistants write modern Jetpack Compose: correct state, effects, performance-aware modifiers, Navigation 3, Paging 3 in Compose, coroutines on lifecycle, animations, UI tests, focus/keyboard navigation, Compose Multiplatform boundaries, Android launch resources, and idiomatic Kotlin. Targets the mistakes LLMs actually make in Compose code.", "author": { "name": "Ivan Morgillo" diff --git a/skills/compose-agent/.cursor-plugin/plugin.json b/skills/compose-agent/.cursor-plugin/plugin.json index 96b4634..822dd14 100644 --- a/skills/compose-agent/.cursor-plugin/plugin.json +++ b/skills/compose-agent/.cursor-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "compose-agent", - "version": "4.3.2", + "version": "4.4.0", "description": "Agent skill that helps AI coding assistants write modern Jetpack Compose: correct state, effects, performance-aware modifiers, Navigation 3, Paging 3 in Compose, coroutines on lifecycle, animations, UI tests, focus/keyboard navigation, Compose Multiplatform boundaries, Android launch resources, and idiomatic Kotlin. Targets the mistakes LLMs actually make in Compose code.", "author": { "name": "Ivan Morgillo" diff --git a/skills/compose-agent/SKILL.md b/skills/compose-agent/SKILL.md index 8351f17..5af2643 100644 --- a/skills/compose-agent/SKILL.md +++ b/skills/compose-agent/SKILL.md @@ -6,7 +6,7 @@ allowed-tools: Read, Glob, Grep, Edit, Write, Bash argument-hint: "[focus area, e.g. 'state', 'effects', 'navigation', 'paging', 'lifecycle', 'animation', 'testing', 'focus', 'kmp']" metadata: author: Ivan Morgillo - version: "4.3.2" + version: "4.4.0" --- # Compose Agent @@ -178,6 +178,10 @@ If the user needs any of the above, narrow the scope and say so. - `references/paging.md` — Paging 3 in Compose: when to page vs plain lists, `collectAsLazyPagingItems`, stable `itemKey`, `LoadState` UI, pull-to-refresh, anti-patterns for paginated lazy lists. - `references/kotlin.md` — Kotlin coding conventions and Android Kotlin style the LLM keeps missing. +## Acceptance Evals + +`evals/evals.json` holds write-mode acceptance cases — prompts that ask this skill to *write* Compose, plus the expectations the produced code must satisfy. Cases 0–2 mirror the `jetpack-compose-audit` scoring rules 1:1 (cross-phase back-writes, Strong Skipping false leads, snapshot self-invalidation) — `bin/ci` keeps them in lockstep so the authoring path can't drift from the audit path; cases 3–4 add foundational phase-correct reads and lifecycle-aware Flow collection. Run a model with this skill loaded against each prompt and check every expectation. + ## Primary Sources Every rule in this skill traces back to one of: diff --git a/skills/compose-agent/evals/evals.json b/skills/compose-agent/evals/evals.json new file mode 100644 index 0000000..5f6d754 --- /dev/null +++ b/skills/compose-agent/evals/evals.json @@ -0,0 +1,65 @@ +{ + "skill_name": "compose-agent", + "description": "Write-mode acceptance evals for compose-agent. Unlike jetpack-compose-audit's evals (which grade an audit of existing code), these grade code the skill is asked to WRITE. Cases 0-2 are the write-mode mirror of jetpack-compose-audit/evals/evals.json cases 12-14 (cross-phase back-write, Strong Skipping false lead, snapshot self-invalidation) — same rules, opposite direction (write vs audit) — so the authoring path can't drift from the scoring path; cases 3-4 cover foundational phase/lifecycle rules that this skill enforces but the audit does not have dedicated cases for. Run a model with the skill loaded against each prompt and check the produced Compose satisfies every expectation.", + "evals": [ + { + "id": 0, + "prompt": "Write a Compose row that shows a `label` and, to its right, a `value` that starts exactly 8dp after the label ends — the value's left edge must track the label's actual measured width for any label length. Target Kotlin 2.1 with Strong Skipping on.", + "expected_output": "A single custom `Layout` (or `SubcomposeLayout`) that measures the label and places the value at `label.width + 8dp`, keeping the measured width inside the layout phase. It must NOT capture the label's width from `onSizeChanged` / `onGloballyPositioned` into a `mutableStateOf` / `mutableIntStateOf` that the value then reads in composition.", + "files": [], + "expectations": [ + "Do not write the label's measured width from onSizeChanged / onGloballyPositioned / onPlaced into state that the value reads in composition (that is a cross-phase back-write: measure -> write -> recompose).", + "Solve it in the layout phase: a custom Layout, Modifier.layout { }, or SubcomposeLayout that measures the label and places the value at label.width + 8dp — these are correct.", + "Do not use BoxWithConstraints as a substitute for measuring the sibling label: it exposes the parent's constraints, not a sibling's measured size.", + "The value's horizontal offset is computed during measure/layout, not read from a composition-phase State." + ] + }, + { + "id": 1, + "prompt": "This screen targets Kotlin 2.1 with Strong Skipping on by default. Make `RowItem` skip recomposition when unrelated screen state changes:\n@Composable fun Rows(items: List, onOpen: (Row) -> Unit) {\n LazyColumn { itemsIndexed(items) { i, row ->\n RowItem(row, isFirst = i == 0, onClick = { onOpen(row) })\n } }\n}", + "expected_output": "Guidance/code that targets the real levers — `Row` being a stable type and `RowItem` restartable/skippable, stable list keys — and explicitly does NOT add ceremony that does nothing under Strong Skipping: no wrapping `onOpen`/`onClick` in `remember`, no `remember(i)` around `i == 0`.", + "files": [], + "expectations": [ + "Do not wrap the onClick / onOpen lambda in remember to 'stabilize' it — under Strong Skipping the compiler already memoizes lambdas passed to composables (even with unstable captures).", + "Do not add remember(i) / remember(index) around the pure `i == 0` expression as a recomposition fix.", + "Point at real levers: Row as a stable type (data class with stable/immutable fields, no var), RowItem being skippable, and a stable key = on itemsIndexed.", + "Recommend verifying with Layout Inspector recomposition counts or Compose compiler reports rather than asserting a win from the pattern.", + "If the manual-remember lever is mentioned at all, it is correctly scoped to SSM-off or a @DontMemoize path — never presented as helping on the default Strong Skipping track." + ] + }, + { + "id": 2, + "prompt": "Write a composable `Legend(entries: List)` that renders each entry as a colored dot plus its label, where each dot's color comes from a lookup keyed by `entry.id` built from `entries`.", + "expected_output": "A composable that derives the lookup as a plain `Map` with `remember(entries) { ... }` (or inline), reading it to render. It must NOT build the lookup by mutating a `mutableStateMapOf` / `mutableStateListOf` inside the composable body and reading it back in the same composition.", + "files": [], + "expectations": [ + "Build the lookup with remember(entries) { entries.associate { it.id to it.color } } (or derive inline) — a plain immutable Map keyed on entry.id.", + "Do not mutate a mutableStateMapOf / mutableStateListOf (put / putAll / add / clear / [k] =) inside the composable body and read it in the same composition (composition-phase self-invalidation).", + "No snapshot-state writes happen during composition." + ] + }, + { + "id": 3, + "prompt": "Write a `Parent` composable that owns a scroll- or animation-driven `progress: Float` in 0f..1f and passes it to a `ProgressBar` child that fills that fraction of its width. Keep recomposition off the per-frame hot path. Target Strong Skipping.", + "expected_output": "Parent hoists/owns the progress; the fast-changing value reaches ProgressBar as a `() -> Float` provider or is read in the layout/draw phase (`Modifier.graphicsLayer { }`, `Modifier.drawBehind { }`, or `Modifier.layout { }`), so composition stays stable each frame.", + "files": [], + "expectations": [ + "The per-frame progress is read in layout/draw (lambda modifier / graphicsLayer / drawBehind / layout) or passed as a () -> Float provider — not as a plain Float read in ProgressBar's composition.", + "ProgressBar does not recompose on every frame of the scroll/animation.", + "State is hoisted to Parent (or a state holder); if the solution is animation-driven, any Animatable is held in remember and driven from a LaunchedEffect (a scroll-driven solution may instead read LazyListState in a lambda modifier)." + ] + }, + { + "id": 4, + "prompt": "Write a `FeedScreen(viewModel: FeedViewModel)` that renders `viewModel.uiState: StateFlow` (a header plus a list of posts). Assume a normal, non-paged list.", + "expected_output": "Collects the StateFlow with `collectAsStateWithLifecycle()`, keeps the composable side-effect free in composition, forwards a `modifier` to the root, and hoists any UI state correctly.", + "files": [], + "expectations": [ + "Collect the StateFlow with collectAsStateWithLifecycle(), not plain collectAsState().", + "No side effects (I/O, navigation, non-remembered mutation) run directly in the composition body.", + "FeedScreen takes modifier: Modifier = Modifier applied to the outermost layout, and parameter order is data -> modifier -> other.", + "Does not reach for collectAsLazyPagingItems() here — this is a plain StateFlow, not PagingData." + ] + } + ] +} diff --git a/skills/jetpack-compose-audit/SKILL.md b/skills/jetpack-compose-audit/SKILL.md index b5e0a8c..8498e35 100644 --- a/skills/jetpack-compose-audit/SKILL.md +++ b/skills/jetpack-compose-audit/SKILL.md @@ -364,3 +364,7 @@ For medium or large repositories: - `references/canonical-sources.md` — the official URLs every deduction must cite - `references/diagnostics.md` — copy-pasteable Gradle/code snippets for Compose Compiler reports, stability config, baseline profiles, and R8 checks - `scripts/compose-reports.init.gradle` — Gradle init script the skill injects via `--init-script` in Step 4 to generate compiler reports automatically + +## Acceptance Evals + +`evals/evals.json` holds acceptance cases that grade an *audit of existing code* — each `{prompt, expected_output, expectations}` is a snippet plus the findings the audit must produce. Cases 12–14 (cross-phase back-write, Strong Skipping false lead, snapshot self-invalidation) are mirrored, in the opposite direction, by `compose-agent`'s write-mode evals, so the scoring path and the authoring path stay in lockstep. Run a model against each prompt and check every expectation. diff --git a/skills/jetpack-compose-audit/evals/evals.json b/skills/jetpack-compose-audit/evals/evals.json index 04b5042..0cce188 100644 --- a/skills/jetpack-compose-audit/evals/evals.json +++ b/skills/jetpack-compose-audit/evals/evals.json @@ -1,6 +1,6 @@ { "skill_name": "jetpack-compose-audit", - "description": "Acceptance evals for the API-hygiene, effect-correctness, cancellation, and state-boundary behavior introduced in 4.1.x, plus Paging 3 in Compose list/load-state behavior added in 4.2.0 (cases 10-11), plus cross-phase back-write detection and the false-lead scoring guard added in 4.3.0 (cases 12-14; compose-agent gained matching authoring guidance in 4.3.1). These are the audit/review acceptance criteria the skill must preserve; the earlier cases also back compose-agent review behavior. Run a model against each prompt and check it satisfies every expectation.", + "description": "Acceptance evals for the API-hygiene, effect-correctness, cancellation, and state-boundary behavior introduced in 4.1.x, plus Paging 3 in Compose list/load-state behavior added in 4.2.0 (cases 10-11), plus cross-phase back-write detection and the false-lead scoring guard added in 4.3.0 (cases 12-14; compose-agent gained matching authoring guidance in 4.3.1, and a write-mode eval mirror in 4.4.0 — its cases 0-2 correspond to cases 12-14 here). These are the audit/review acceptance criteria the skill must preserve; the earlier cases also back compose-agent review behavior. Run a model against each prompt and check it satisfies every expectation.", "evals": [ { "id": 0,