feat(setup): reconciliation tracking for marketplace installs by potiuk · Pull Request #1324 · apache/magpie

potiuk · 2026-09-21T22:36:50Z

Summary

A marketplace install has no memory of what it was last set up against. Snapshot methods reconcile overrides against a new framework version on /magpie-setup upgrade; the marketplace method has no local lock and no equivalent pass, so nothing notices when a plugin moves underneath a configuration written against an older build.
This adds the missing half: a generated surface_hash: fingerprint on every skill, a reconciled: stamp recording what a project was last configured against, and a free per-skill comparison at the top of every skill run. /magpie-setup reconcile runs the sweep; verify reports it read-only and owns the latest-version comparison; config, adopt, upgrade and unadopt maintain the stamp.
Design, including rejected alternatives and the nine places the build diverged from it: docs/designs/2026-09-21-marketplace-reconciliation-tracking.md.

Type of change

Skill change — the shared pre-flight block, propagated to 65 skills
Tool / bridge contract — new tools/dev/skill-surface-hash.py, validator requirement
Python package — new generator + tests
Cross-cutting — sandbox-aware behaviour; the HITL contract of auto-invoked config
Documentation — specs, design, locks.md, mode-economics.md
CI / dev loop — new skill-surface-hash prek hook

How it works

The fingerprint covers two things — a skill's requires_config: list and its structural anchors (##–#### headings, golden-rule names) across SKILL.md and its detail files, each anchor tagged with its source file so a heading moving between files changes the digest. Deliberately not file content: a reworded paragraph must not tell every adopter their configuration went stale; a renamed step must.

The check is free — the skill reads its own hash from its own frontmatter and the stamp from the lock pre-flight already opens. Three load-bearing properties, each pinned by a test or fixture: the generated pre-flight region is excluded from the fingerprint (so editing shared text doesn't flag every adopter); an unreadable plugin manager is unknown, never absent (inside the sandbox claude plugin list --json returns [], which the old text read as "nothing installed" and would have acted on); and a dev build is a version like any other.

Test plan

prek run --all-files — 33 hooks, exit 0, clean tree.
20 generator tests (determinism, prose-immunity, cross-file anchor moves, #### in, ##### out, subdirectories out) + 471 validator tests.
Graded evals, not print-mode: preflight-reconciliation 7/7, setup 80/83.
The prek hook exercised end to end: a #### heading added to locks.md moves the setup digest and the checker catches it.
Three setup cases still fail, all pre-existing or environmental: preflight-floor/case-6-unadopted (references a predicate never defined, from 2026-09-13), step-m3-baseline-pick/case-3 (unrelated), and one variance case. The suite is not deterministic under grading — three runs of the same tree gave 81/79/80 of 83 with four cases differing. Worth knowing before treating it as a gate.

RFC-AI-0004 compliance

HITL — all proposals. Worth naming: an earlier revision had config writing the committed lock, which pre-flight auto-runs unattended on a justification repeated in 134 shipped files ("only gitignored paths"). It now never touches the lock.
Sandbox — no new host access; the design is shaped by what a sandboxed session can read.
Vendor neutrality / Conversational + correctable — placeholders throughout; read-only and proposal-only.

Notes for reviewers

The cost is the decision. The block adds a median +1,608 tokens to every invocation of all 65 non-setup skills, peaking at +49.0% (ci-runner-audit, 3,281 → 4,889) — stated in mode-economics.md and the design's Risks. Moving detail behind a pointer into locks.md doesn't work: that file is in the plugin cache, unreadable in a sandbox, so the rules would be missing exactly where this is aimed. If the price isn't worth it, narrow or drop the feature rather than trim further.

Five small things I'd fix before or just after merge:

tools/dev/preflight-block.md:87-90 still says skills lives in one store "never both", ten lines above the paragraph now calling that state expected — self-contradicting, in always-on text.
"a reconciled: block exists" (the gate stopping the sweep re-proposing) is undefined against a local file holding only the three always-local keys — a state pre-flight itself creates.
Spec criterion 17 lists two skip conditions; the block has three.
An unattended config run creates a stamp, retiring the one-time sweep for a project never swept. verify reports it; nothing documents it.
A PR deleting or renaming a detail file escapes the PR-scoped prek run and lands a stale digest that fails main — the class lychee already solves by running unconditionally.

Deliberate trade: the sweep is gated on any stamp rather than this skill's entry, which stops a permanent false prompt in the eleven skills that declare no requires_config, at the cost of a skill configured after the last sweep getting no per-skill prompt (backstopped by step 7 and verify).

.last-sync deliberately not bumped — 10 commits behind on unrelated work. The specs here were updated by hand, so the rule is satisfied in substance; bumping would falsely claim the others are synced.

🤖 Generated with Claude Code

A marketplace install has no memory of what it was last set up against. The snapshot methods reconcile overrides against a new framework version on upgrade; the marketplace method has no local lock and no such pass, so nothing notices when a plugin moves underneath a configuration that was written against an older build. The design: `setup` stamps what it reconciled — a version plus a per-skill fingerprint of the surface the project actually resolves — into the committed lock when the project is adopted, or into gitignored local state when it is only configured. A prek hook ships that fingerprint as a `surface_hash:` frontmatter field, so a running skill compares its own hash against the stamp with no file read and no hashing at runtime, which is what makes the check work inside the sandbox. The project-wide sweep and the latest-marketplace-version comparison live in `verify` and a new `reconcile`, and `verify` is suggested on a fortnightly clock kept in local state. Proposed only; nothing is built. Records the five rejected alternatives, including the version-only comparison that this repository's daily dev builds would turn into a prompt nobody can action. Generated-by: Claude Opus 5

The first draft treated a missing stamp as unknown-and-silent, to avoid greeting pre-existing adopters with a prompt they could not evaluate. That makes the blind spot permanent: the projects certain to need reconciling are exactly the ones adopted before the stamp existed, and they would never be offered it. A missing stamp now proposes a one-time full sweep. The sweep needs no baseline, because with nothing to diff against it validates the present instead — do the override anchors still resolve, does every `requires_config` entry resolve through the lookup chain — both answerable from the current tree. The guessed baseline (lock `min_version`, else the last commit touching the committed config mapped through the marketplace clone's history, else local mtimes) only shapes the wording, and is phrased as the estimate it is. Sandboxed sessions sweep the repository side and say plainly that anchor resolution could not be checked there. The proposal is made once: accepted it writes the stamp and the cheap per-skill check takes over, declined it is remembered until the configuration or plugin set changes. Generated-by: Claude Opus 5

Separate the two axes the first draft blurred. Version comparison answers "is there something newer", and a dev build counts: nothing strips `.devN`, rounds to the release segment, or treats a dev-to-dev move as a non-event, and `verify` reports it as the update it is. The fingerprint answers "does it affect this project's configuration", and gates only the prompt. A newer dev build with no surface change is therefore an update that `verify` reports and that no pre-flight interrupts anyone about — both answers correct, neither suppressing the other. Choosing a dev version is choosing frequent change; the design owes that user an honest comparison rather than a rounded one, which is now the second reason the release-segment-only alternative was rejected. Generated-by: Claude Opus 5

Eight tasks. One of them is code — the `surface_hash` generator and its hook, with the tests written first — and one is the validator that makes the field mandatory. The rest are the agentic surfaces: the lock format, the pre-flight comparison, a new `reconcile` sub-action, `verify` gaining the sweep and the version comparison, `config` and `adopt` writing the stamp, and the specs that state the guarantee. Each task carries the exact files, the test or eval fixtures that fail first, the commands to run, and its commit. Two fixes to existing text ride in the pre-flight task because they are in the lines being rewritten: an empty `claude plugin list --json` must read as unknown rather than as no plugins installed, which is what a sandboxed session actually returns, and the version comparison has to say that a dev build counts. Generated-by: Claude Opus 5

Generated-by: Claude Sonnet 4.5

Every SKILL.md now carries a surface_hash: frontmatter field, stamped by tools/dev/skill-surface-hash.py (Task 1 of the marketplace-reconciliation plan). validate_frontmatter now enforces it: a missing surface_hash: or one that isn't sha256: + 16 lowercase hex characters is a HARD violation, so a new skill can't ship without one and a corrupted value is caught. Both error messages point at the generator, never at hand-typing a value. Updated the inline frontmatter fixtures across test_validator.py that build a full, violation-free SKILL.md so they keep asserting on their own concern instead of tripping the new check. Generated-by: Claude Sonnet 4.5

Documents the reconciled: block (version/at/skills, sha256 fingerprints) that setup writes to record what a project's configuration was last checked against, and where it lives depending on adoption state. Also documents how marketplace adopters reach reconciliation through the stamp instead of /magpie-setup upgrade's snapshot walk. Generated-by: Claude Opus 5

The per-skill surface_hash pre-flight check and /magpie-setup reconcile apply to every adopted or configured project regardless of install method, per controller ruling; only the override-walk's trigger differs (snapshot methods additionally reach it via /magpie-setup upgrade). The previous wording in agentic-overrides.md incorrectly framed both as marketplace-exclusive, contradicting locks.md in the same prior commit. Also drops a locks.md callback to a rationale the file never states. Generated-by: Claude Sonnet 4.5

…re-flight Adds a new step to the shared pre-flight block that every skill runs first: compare the skill's own generated `surface_hash` against its entry in the project's `reconciled:` stamp (committed lock, or the local reconciled.json for configured-but-not-adopted projects). A match stays silent; a mismatch distinguishes a `requires_config` change from a moved structural anchor and proposes the matching fix; no entry at all proposes the one-time reconciliation sweep. Declines are remembered per skill via `acknowledged` until the hash moves again. The check is method-agnostic — it applies the same way to a marketplace floor and a snapshot pin. Rides along: an unreadable `claude plugin list --json` (sandboxed plugin cache) is now treated as unknown rather than "nothing installed", so a sandboxed session no longer gets told to install the project's entire floor; the version-comparison paragraph states that a dev build compares like any other version, with the reconciliation prompt gated on the fingerprint rather than the version delta; and a new end-of-run item suggests `/magpie-setup verify` when it has not run in `setup.verify_interval_days` (default 14). Adds the `preflight-reconciliation` eval suite (5 cases) pinning the four outcomes and the update-piggyback rule, propagates the block into every non-setup SKILL.md via check-skill-preflight, and confirms skill-surface-hash produces no change — the pre-flight region stays excluded from the fingerprint by design. Generated-by: Claude Opus 5

…figured, record on shown Fix round 1 on the reconciliation pre-flight check, addressing three Criticals and nine Importants from review. - Stamp key is now a skill's frontmatter `name:` (e.g. `magpie-security-issue-triage`) instead of `<plugin>/<skill>`, which was underivable under a snapshot install (no plugin component) and degraded silently to a false "no entry" sweep proposal. - Step 4 now opens with a gate: no lock, no `.apache-magpie-local/`, and no `.apache-magpie-overrides/` means nothing was ever configured or adopted, so the step is silent and reads nothing — previously an unadopted marketplace install got a spurious sweep proposal on every skill invocation. - `acknowledged` is recorded when a proposal is *shown*, never on a decline the check does not wait for (it always continues into the work in the same turn). Split into `acknowledged.skills` (per-skill, keyed by name and gated on that skill's hash) and `acknowledged.sweep` (project-scoped, gated on the installed plugin version) — a per-skill sweep key would have meant proposing the sweep once per skill invoked rather than once per project. - `requires_config` vs. anchor discrimination no longer claims causation the resolve check cannot support: an unresolved entry is the actionable half regardless of why the hash moved; both fixes are proposed when both apply. - `.apache-magpie-local/reconciled.json`'s shape is now pinned (no `reconciled:` wrapper; carries `version`/`at`/`skills` plus the always-local keys) and its `skills` entries take precedence over the committed lock's when both name the same skill. - Dropped `update_available`/marketplace-clone piggybacking from the eval suite: the block never reads the clone — that comparison is `verify`'s alone, per the block's own step 10. - Trimmed rationale the executing agent does not need (the three-keys aside, "nothing above writes the stamp", the version/reconciliation cross-axis paragraph) and made the local-file read conditional on the lock missing an entry or a proposal being imminent, reusing that read in step 10 — down from a per-invocation +1,168 tokens on the smallest skill to +1,127. Adds a sixth eval case (stamp exists but this skill is absent from it — the more common real-world trigger than no stamp at all) and a new preflight-floor case for the empty-`claude plugin list --json` unknown rule, previously untested. Strips the fixtures' `Result:` paragraphs so the model has to apply the rule rather than read the answer. Generated-by: Claude Sonnet 4.5

… 9-10 Fix round 2 on the reconciliation pre-flight check. Ruling 6 supersedes Ruling 5's precedence half: the re-review found that "local wins when both stores name a skill" was never reachable on the common path, because a lock match short-circuits step 4 before the local file is ever read, and would have been silently false exactly when the lock happened to match. `skills` now lives in exactly one store per project — the committed lock when adopted, the local file when configured but not adopted — matching what the design said from the start. Both `locks.md` and the block now state that invariant in one line, plus one line for the anomaly: a `skills` entry found in both stores is a bug or a hand edit, not a state this framework writes; the local entry wins and `/magpie-setup reconcile` reports it as drift. `locks.md`'s local-file JSON example now shows both shapes — full (unadopted) and always-local-keys-only (adopted) — instead of one ambiguous example. Ruling 5's other half (the flat JSON shape, no `reconciled:` wrapper) is unchanged. Also folds in the remainder of I6: steps 9 and 10's "why this lives here" rationale is now one shared clause instead of two near-duplicate paragraphs. Re-ran the full `evals/setup/` suite (72 cases, print mode) rather than just `preflight-floor/case-6-unadopted`: all cases extract and parse cleanly, including all 8 `preflight-floor` cases; `locks.md`'s own `lock-marketplace-parse` suite is unaffected (its extraction boundary ends before the `reconciled:` block section this round touched). Generated-by: Claude Sonnet 4.5

Implements /magpie-setup reconcile — the one-time project-wide sweep the shared pre-flight block proposes when a skill's own surface-hash check finds no baseline to diff against. Walks every configured or overridden skill, checks override-anchor resolution and requires_config resolution, proposes fixes item by item, and writes the reconciled: stamp (committed lock when adopted, the flat .apache-magpie-local/reconciled.json otherwise). Generated-by: Claude Opus 5

Ruling 7 (Task 5 review round 1): the timing difference from the pre-flight block's per-skill check is deliberate, not a deviation from Ruling 3 — the pre-flight never blocks for an answer, so it has no decline event and records on show; reconcile blocks for a real item-by-item and whole-sweep confirmation, so it has a real decline event to hook, and an abandoned sweep should leave nothing recorded so the proposal returns on the next run. Generated-by: Claude Sonnet 4.5

…rsions Generated-by: Claude Opus 5

config writes the reconciliation stamp for every skill it just configured (Step 3b): current surface_hash, running version, and today's date, landing in the committed lock when the project is already adopted, else in .apache-magpie-local/reconciled.json. adopt does the same for every skill its configuration store now covers (Step 4d) — the one path where the stamp enters git, riding along in the same commit the maintainer is already making. upgrade's override-reconciliation walk (Step 5) writes the same stamp for every override it just confirmed clean, so a snapshot project's stamp does not go stale the moment the walk that checks it finishes. Generated-by: Claude Opus 5

…e local stamp Fix round 1 on the reconciliation-stamp work, per review ruling: - config's reconciliation write (Step 3b) never touches the committed lock, adopted project or not (ruling 8 / C1). Unadopted: still writes version/at/skills to .apache-magpie-local/reconciled.json. Adopted: writes no skills/version/at anywhere; records the per-machine fact as acknowledged.skills["<name>"] instead, the always-local key locks.md already reserves for it. This also closes the worktree gap: config.md carries no main-checkout gate, so an unattended pre-flight run could otherwise have staged into a maintainer's in-progress index from inside a worktree. Reverted the auto-invoke justification, Hard rule 1, and the opening summary to their pre-stamp wording, since under this fix they are true again. Widened the stamp's skip gate from "Step 3 wrote nothing" to the R2 nothing-configured-or-adopted condition, so a fully-configured-but-unstamped project still gets recorded. - adopt's Step 4d now migrates config's local stamp into the lock on the unadopted-to-adopted transition, instead of leaving the same skill named in both stores (ruling 9 / C2). It also records the version Step 2 actually read off the machine, not the (possibly higher, ratcheted) min_version Step 2 wrote to the lock (I2). - upgrade's override-reconciliation walk (Step 5) now also runs reconcile's requires_config resolution check before stamping a skill clean, so an override with intact anchors but an unresolved requires_config entry is left out rather than reported false-clean (I1). Documents that this reconciles overrides, not configuration, and that reconcile is the full pass; the skip gate now matches the surface the walk actually checks. - locks.md: version/at describe "when this block was last written," not "when the project was last fully swept" (I4); the write-owner list now includes upgrade. - Eval fixtures: reworked step-config-stamp's two cases for the new acknowledged.skills-vs-skills split, fixed the base-directory path segment and hash length (M4), added the R2-skip case, and added new step-adopt-stamp and step-upgrade-stamp suites (M5). Generated-by: Claude Sonnet 4.5

Ruling 10 fix round 2. config's Step 3b "already adopted" branch was writing acknowledged.skills["<name>"] for every skill in scope whose requires_config now resolves, including ones this run touched nothing for. acknowledged.skills is a generic gate in preflight-block.md step 4 covering both a stale requires_config finding and a stale-anchor finding without distinguishing them, so a routine no-op config run on an adopted project was silencing anchor-drift findings it never investigated and cannot fix. Narrowed the trigger to what the prose already promised: config writes acknowledged.skills only for a skill whose missing configuration this run actually wrote (Step 3 produced the file). A skill already resolved and untouched this run gets no entry, so its pre-flight finding stays live. The unadopted branch (skills/version/at into .apache-magpie-local/reconciled.json) is unchanged — it was never the problem. Left adopt.md 4d's hash-copy-unchanged migration alone, per the controller's explicit instruction: recording a skill's current hash there would claim a reconciliation adopt never performed. Eval: added step-config-stamp/case-4-adopted-untouched-skill (the regression this ruling closes) and widened output-spec.md's write_stamp definition to cover both zero-write reasons (R2 gate, and the narrowed adopted-branch trigger). 83 cases across 21 steps. Generated-by: Claude Sonnet 4.5

skill-surface-hash.py only ever hashed a skill's own SKILL.md, so an override anchored to a heading in a sibling detail file (setup, pr-management-triage, security-issue-sync, ...) could drift with no signal: the generator would not notice, and reconcile.md's anchor-resolution check only read SKILL.md too. surface_inputs() and surface_hash() now take the skill's directory and fold in every sibling *.md file's anchors (sorted by filename, no recursion into subdirectories), tagging each detail-file anchor with the file it came from so a heading moving between files changes the digest. A skill directory with no detail files still hashes exactly as before. reconcile.md's anchor-resolution check now reads the same set of files. Generated-by: Claude Opus 5

The reconciliation fingerprint now spans a skill's sibling *.md detail files, not only its SKILL.md. Regenerated the 18 multi-file skills whose fingerprint widens as a result (setup, pr-management-triage, pr-management-code-review, security-issue-sync, issue-reproducer, and others); the 57 single-file skills are unaffected. Generated-by: Claude Opus 5

reconcile.md's anchor-resolution check now names SKILL.md and its sibling detail files explicitly, but upgrade.md's Step 5 anchor check (the override-reconciliation pass that rides along with a snapshot refresh) stayed silent on file scope. Since upgrade writes reconciled.skills entries off that check's outcome, the gap reproduced the false-clean failure Task 7 closed: a skill could be stamped reconciled while its override anchors to a detail-file heading that moved. Matched reconcile.md's wording so the two checks describe the same resolution surface. Also add a test for the property the suite didn't cover directly: the same combined anchor text split the opposite way across two detail files must still hash differently, proving the payload tracks which file an anchor came from and not just the anchor text. Generated-by: Claude Sonnet 4.5

Add acceptance criteria to adoption-and-setup.md and marketplace-distribution.md for the surface_hash fingerprint, the reconciled: stamp, the reconcile sub-action, and the config/adopt/ upgrade write split the marketplace-reconciliation plan shipped. Mark the design doc Built and record where the build departed from it (stamp key, widened fingerprint scope, no pre-flight piggyback on the marketplace clone, method-agnostic stamp scope, show-vs-decline recording, config never touching the committed lock's skills map). Note the accepted per-invocation token cost in mode-economics.md. Generated-by: Claude Opus 5

step-reconcile/case-2-stale-anchor and step-upgrade-stamp/case-1-mixed- checks assert on a `detail` field that the output schema defines as free text, but the runner's default prose-field set only recognises the plural `details`, so both were graded by verbatim string equality and could never pass a real, correct answer that adds context beyond the terse expected string. Extend each fixtures dir's grading-schema.json with `detail`, the same mechanism several other suites already use for their own custom free-text field names, so both are judged semantically instead. Neither the output schema nor the expected values change: case-2 still pins a re-anchor (not a config) proposal for a moved anchor, and case-1 still pins that upgrade stamps only the override whose requires_config resolved, excluding the one that did not. Generated-by: Claude Sonnet 4.5

…nciliation check case-1-at-floor's expected action of "silent" predates the shared pre-flight block's step 4 (the reconciliation-stamp check this plan added): the fixture has an adopted lock and overrides, all plugins already at floor, and no reconciled: entry for the invoked skill in either store, which is exactly step 4's "neither store names this skill" branch — propose the one-time /magpie-setup reconcile sweep, action=print-command. Went through every other preflight-floor case and every other evals/setup/ step against step 4; findings recorded in the task report rather than silently applied. Generated-by: Claude Sonnet 4.5

Rulings 11-12: the reconciliation check (step 4) sat lexically before step 5's stop-for-restart gate with no stated relationship to step 3's outcome, leaving an agent unable to tell whether to surface a reconciliation finding alongside a restart notice. Resolve it as one rule: step 4 runs unless there is nothing to reconcile, or step 3 is about to stop the run for a restart (install/update ran, commands were only printed for no CLI, or nothing ran because of an untrusted url) — skip it there, since the session is restarting anyway and the check costs nothing to repeat. An *unknown* step 3 result is not a stop (step 5 already says so) and tells the session nothing about the project's own configuration, so step 4 now runs through it explicitly instead of reading as a third, undefined case. Propagate via check-skill-preflight.py --fix (65 plugin SKILL.md copies). skill-surface-hash.py --fix confirms the hash does not move — the generated pre-flight region stays excluded from the fingerprint. Token counts did move (the block gained a paragraph), so skill-token-count --write's docs/mode-economics.md update rides along in this same commit rather than a separate one as planned: prek's token-count hook checks the staged snapshot against every plugins/*/SKILL.md and tools/dev/preflight-block.md file, so staging the propagation without the regenerated counts fails that hook deterministically (the same prek-stash interaction Tasks 1 and 9 hit and resolved the same way). No --no-verify or SKIP used. Also makes preflight-floor/case-8-unknown-plugin-list decidable: its report.md never stated whether .apache-magpie-overrides/ or .apache-magpie-local/ existed, so step 4's outcome could not be derived from its own stated facts. Added that .apache-magpie-overrides/ exists and no store names the invoked skill (mirroring case-1's grounding), and updated expected.json to the reconcile-sweep proposal Ruling 12 now requires. Re-ran the five step-3-blocking cases (2/3/4/5/7); none changed under Ruling 11, since they already expected no step 4 finding. Generated-by: Claude Sonnet 4.5

…l-absent-from-stamp Discovered on the required re-run after Ruling 11 (step 4 skips when step 3 stops the run): this fixture's lock stated a 0.9.0 floor for magpie-pr-management while its own plugin-list line reported that plugin at 0.2.0.dev202609180100 — below the floor, and not something anyone had reason to notice before Ruling 11 existed, since step 4 used to run regardless of step 3's outcome. With step 3 now genuinely below-floor, step 3 stops the run for a restart and step 4 (the thing this case exists to test — a skill absent from every stamp proposing the sweep) never runs at all, defeating the case's own point rather than exercising it. The floor value was incidental, not load-bearing for anything this case pins, so lower it to 0.1.0 (satisfied by the stated 0.2.0.dev202609180100) and say so explicitly, so step 3 passes silently and step 4 runs as the case always meant to test. expected.json is unchanged — the case still pins propose_sweep / ["no_stamp"]. Generated-by: Claude Sonnet 4.5

…ditional Rulings 11-12 shipped in tools/dev/preflight-block.md but were never stated in the durable record: the spec's pre-flight criterion and the design doc both still described the check as running unconditionally. Extend adoption-and-setup.md's acceptance criterion 17 with the skip rule (step 3 stopping the run for a restart skips it; an unreadable plugin manager does not), and add a seventh "Where the build departed from the design" item recording the same divergence and its reason — a reconciliation proposal stacked onto a restart notice is the prompt pile-up this design otherwise avoids, and the check costs nothing to repeat next session. Generated-by: Claude Sonnet 4.5

The shared pre-flight block proposed `/magpie-setup reconcile` whenever the running skill had no entry in the project's `reconciled:` stamp, but `reconcile` only ever creates entries for skills the project configures. Eleven shipped skills declare no `requires_config:` and carry no override, so nothing an adopter can do puts them in a stamp: every adopted project got a sweep proposal from each of them on the first invocation after every plugin update, forever, after already sweeping. The sweep is now proposed only when there is no `reconciled:` block in either store at all. A stamp that exists and omits this skill is silent — the project does not configure that skill, and the `requires_config` step later in the same block already covers the case where it does and a file is missing. Scope is stated broadly to match: a skill is in scope when an override names it, or when its `requires_config:` entries resolve from the project's own config directories. Also in the block: - the verify-overdue clock reads the most recent of `verified_at` and `verify_suggested_at`, so writing `verify_suggested_at` re-arms something; before, the nudge fired on every run; - a skill that cannot see its own `surface_hash` skips the check silently rather than guessing; - every write merges into `.apache-magpie-local/reconciled.json` and creates it, and its directory, when absent; - `acknowledged.sweep` takes the same version fallback `version` has, so it is defined on a snapshot install; - restated rationale comes out of step 4. A `skills` entry in both stores is documented as the expected transitional state it is — `config` on one machine, `adopt` on another — with `reconcile` offering to drop the redundant local entry, and `unadopt` migrates the stamp back into the local file before removing the lock rather than stranding it. `locks.md` no longer names `config` as a writer of the committed block. The fingerprint's anchor regex widens from `##`/`###` to include `####`: twelve skills use fourth-level headings for real structure, and an override anchors to those exactly as it does to a `##` step. Twelve digests move, which is free today and expensive after the first adopter holds a stamp. The `skill-surface-hash` hook now watches every `*.md` directly inside a skill directory, not just `SKILL.md`, because the fingerprint spans sibling detail files — a heading renamed in `locks.md` moves the `setup` digest and would otherwise pass green on the PR and fail on `main`. Measured cost, recomputed: the block grew from 1,679 to 3,271 tokens, +1,608 per skill including the `surface_hash:` line, +49.0% on the smallest skill (ci-runner-audit, 3,281 to 4,889). Generated-by: Claude Opus 4.5

- The accepted always-on cost was quoted as "+1,100 tokens / ~+34%". Recomputed from the generated table: the pre-flight block grew from 1,679 to 3,271 tokens, +1,608 per skill with the `surface_hash:` line counted, and +49.0% on the smallest skill (ci-runner-audit, 3,281 to 4,889) — the figure a maintainer is being asked to accept a permanent cost against. - Two new departed-from-design items (seven to nine): the sweep gate is the absence of the whole stamp rather than of an entry, and a `skills` entry in both stores is an expected transitional state with `unadopt` migrating the map back rather than stranding it. - `upgrade` does not record on decline — it writes stamp entries for what its walk reconciled and nothing else. Dropped from the decline sentence in both documents. - The stamp's scope is stated as the broad reading it always had: an override naming a skill, or its `requires_config:` entries resolving from the project's own config directories. "A handful, not the ~75" is replaced; the count follows from how much the project configures. - Spec criteria 17-19 and 21 restated to match, and criterion 21 gains `unadopt`'s migration. Generated-by: Claude Opus 4.5

The sweep proposal is gated on the absence of the whole `reconciled:` block, not on the absence of an entry for the running skill. Two cases pin it: - `case-7-no-config-surface` (new) — a skill with no `requires_config:` and no override, in a project that holds a stamp. Eleven shipped skills are shaped this way, so under the old reading each of them proposed the sweep on every adopted project after every plugin update, forever. - `case-6-skill-absent-from-stamp` — the same project state for a skill that does have a configuration surface; its expectation moves from `propose_sweep` to `silent`, and its fixture now states that both `requires_config:` entries resolve, so the outcome is decidable from the facts it gives. `case-4-no-stamp` remains the suite's only `propose_sweep` case. Generated-by: Claude Opus 4.5

`reconcile.md`'s opening still described itself as what the pre-flight names when neither store names the running skill. That is now the silent case; the sweep is named only when no `reconciled:` block exists in either store at all. Generated-by: Claude Opus 4.5

…ow needs Step 4 gained a rule this wave: a skill that cannot see its own `surface_hash` skips the reconciliation check silently rather than guessing. `case-1-at-floor` and `case-8-unknown-plugin-list` never stated one, so both became correctly silent and stopped matching their `print-command` expectations — the fixtures were under-specified for the rule, not wrong about the outcome. Both now state the invoked skill's `name:` and `surface_hash:`, and say explicitly that no `reconciled:` stamp exists in either store, which is what the sweep branch keys on after this wave. Expectations unchanged. Same "make the case decidable from its own facts" correction case-8 already took once. Generated-by: Claude Opus 4.5

`main` moved by one commit while this branch was in flight — apache#1323, which edited `setup-isolated-setup-update`'s own body. The rebase carried this branch's measured row for that skill, which predates the edit; regenerating restores the true figure (5,231 → 5,578). No hand edit: `skill-token-count --write` produced it, and the pre-flight block and every `surface_hash` were confirmed in sync beforehand. Generated-by: Claude Opus 5

check-skill-preflight.py solved one-source-many-copies for exactly one shared block. check-shared-blocks.py generalizes it: the pre-flight block keeps its exact historical auto-insert behavior (same source path, same delimiter text, same exemption rule — none of the 65 propagated copies change), and any number of additional "declared" blocks can now be sourced from tools/dev/blocks/<name>.md and filled into a region a target already carries, never inserted, and only within the skills/ tree. Generated-by: Claude Opus 5

…shared-blocks One mechanism, not two: the pre-commit hook that used to call check-skill-preflight.py now calls check-shared-blocks.py, and the retired script is deleted rather than kept as a second entry point. files: widens to cover every *.md directly inside a skills/<name>/ directory and tools/dev/blocks/*.md, so a future declared block's source and targets are both covered by the same hook. Generated-by: Claude Opus 5

potiuk · 2026-09-21T22:59:36Z

Needs a lot of deduplication :)

…reflight skill-surface-hash.py stripped the generated region before hashing by matching the literal preflight marker text alone. That was safe only while zero declared blocks existed: the moment one lands in a shared detail file, its own headings would join the carrying skill's surface_hash, and editing shared framework text would start reading as project-specific drift. skill-surface-hash.py now loads check-shared-blocks.py as a module and reuses its PREFLIGHT_RE, DECLARED_RE, and new strip_generated_regions() directly, so the two scripts' notion of "a generated region" cannot drift apart. The docstring states the resulting trade explicitly: an override anchored to a heading inside a shared block will not be detected as drift by this fingerprint, which is correct (shared text moving is a framework change) but was previously an accident of a regex rather than a stated contract. Also: fix declared_seen only counting declared-block files that had an error or a change, so a correctly-in-sync declared block was never counted; name the renamed check-shared-blocks hook in the skill-surface-hash pre-commit comment. Generated-by: Claude Sonnet 4.5

… files The five setup skill detail files (install/uninstall/upgrade/verify/ worktree-init) had duplicated procedures: the git-repo + main-checkout pre-check (install~uninstall), ASF detection (upgrade~verify), worktree enumeration and the sandbox-allowlist helper chain (install~upgrade). Extract the genuinely-identical wording into tools/dev/blocks/<name>.md, wired via check-shared-blocks.py's declared-block markers, so each is now written once and propagated rather than hand-copied. Also fixes a bug in check-shared-blocks.py's is_allowed_target(): it used path.resolve(), which follows this repo's own self-adoption symlinks (skills/<name>/ -> plugins/<family>/skills/<name>/) to their real location outside skills/, rejecting every legitimate target here. Switched to path.absolute(), which normalises without following symlinks, matching how skill-surface-hash.py already treats this tree. .lychee.toml now excludes tools/dev/blocks/: a block's relative links are written to resolve from the host skill directory they propagate into, not from the block source's own location. Generated-by: Claude Sonnet 4.5

…st nesting Fix round 1 on the shared-block extraction: - C1 (critical): install.md's Step 12 pass 3 had lost its only `dangerouslyDisableSandbox: true` instruction when the block replaced the lead-in that named it. Restored, mirroring upgrade.md's and worktree-init.md's own bolded call-site wording. - declared_block_text() now dedents the flat block source and re-indents it to the host's own marker indentation (DECLARED_RE gained an `^(?P<indent>[ \t]*)` group, `re.M`). Blank lines inside a region are never indented, so the trailing-whitespace hook's stripping and this generator's own output do not fight to a non-convergent --fix. Restored install.md's two regions (Step 0's main-checkout-precheck, Step 12's worktree-enumeration + sandbox-allowlist-helper) to their original nesting under the outer numbered list; verified via pandoc -f gfm that both render as one continuous <ol>, matching the pre-extraction shape. strip_generated_regions() (shared with skill-surface-hash.py) strips indented regions the same way, with its own test. - I2: is_allowed_target() now normalises with os.path.normpath on top of .absolute(), closing a `skills/../docs/notes.md`-style traversal that .absolute() alone does not catch. - I3: added a symlinked-skill-dir test exercising is_allowed_target() and process_declared() together, reproducing this repo's own self-adoption layout — the exact case the .resolve()->.absolute() fix (prior commit) was silently untested against. - Minors: declared_seen now counts regions (8), not files (4); doctoc's prek hook now excludes tools/dev/blocks/ so the SPDX header stays first in every block source, as _strip_licence_header() requires; fixed a dangling `skills/setup/AGENTS.md` reference to the real `agents.md`; fixed the asf-detection.md SemBr wrap that split "the same way" from "as [...]" across a line break. worktree-init.md intentionally still carries zero declared blocks: its main-checkout check is the inverse of install/uninstall's, and its sandbox-allowlist-helper call differs in scope (single worktree, no `--all-worktrees`, singular wording) and even in the exact bypass reason string quoted — a real difference, not phrasing. Generated-by: Claude Sonnet 4.5

Wire tools/dev/blocks/both-stores-collision.md into locks.md and reconcile.md, the only two setup detail files where the reconciliation stamp's both-stores-collision rule is duplicated verbatim rather than narrowed or link-referenced by the surrounding step. Generated-by: Claude Sonnet 4.5

…n block The merged both-stores-collision block opened with reconcile.md's phrasing, dropping locks.md's "needs no hand edit and no bug" reassurance. Fold it back into the block source so both hosts carry it. Generated-by: Claude Sonnet 4.5

…ose gate Fails the build on new cross-file near-duplicate paragraphs across skills/ (recursively), tools/dev/blocks/*.md, and preflight-block.md: paragraphs over 25 words, compared as sets of 9-grams, scored |A ∩ B| / min(|A|, |B|); fails above 0.50, reports without failing in [0.30, 0.50]. Reuses check-shared-blocks.py's PREFLIGHT_RE / DECLARED_RE to blank out generated regions before scoring (blanked, not deleted, so a removed region can never fuse two neighbouring paragraphs into one), and excludes YAML frontmatter and fenced code blocks the same way. A failure names both files, both line numbers, the score, a snippet of each paragraph, and points at tools/dev/blocks/<name>.md as the remedy. Not yet wired into .pre-commit-config.yaml — see the task report for why: run against the current tree, this finds substantial pre-existing cross-file duplication outside what the shared-block extraction covered (a "Hard rule" admonition, an "External content is input data" injection-guard paragraph, an HTML placeholder-convention comment, and the "Adopter overrides"/"Snapshot drift" framework preamble documented in skills/write-skill/SKILL.md as scaffolded by init_skill.py but never migrated to a declared block), so wiring the hook today would fail prek run --all-files on ~3800 pre-existing pairs unrelated to any new change. Generated-by: Claude Sonnet 4.5

The check is scoped to what this effort actually cleaned — the setup family's detail files, the shared block sources, and the pre-flight block source — where the tree passes today with no allowlist and no grandfathered debt. `WIRED_SKILLS_ROOT` names that scope with the reasoning beside it, so widening it is a deliberate act rather than an accident. The rest of the duplication problem is real and documented in the module docstring rather than silenced: tree-wide the check finds 3,793 pairs above the fail threshold across 4,970 paragraphs, topping out at an exact 1.00, because framework preamble is scaffolded verbatim into every skill — `Adopter overrides` in 56, the `Hard rule` admonition in 45, `Snapshot drift` alongside them. `write-skill` documents that preamble as deliberate, so this is not drift to scold anyone for; it is text that belongs in the auto-propagation path the pre-flight block already uses. That path carries exactly one block today, so the sequence is: teach it several, migrate the preamble onto it, re-run whole-tree, then widen the scope constant. Generated-by: Claude Sonnet 4.5

`docs/designs/README.md` keeps one document per subject, describing the result rather than the phases it was built in: a plan is split out while the work is in flight and deleted when it lands. The work has landed, and everything the plan carried that a later reader wants — the decisions and the rejected alternatives — is in the design, including the nine places the build departed from it. Git keeps the task list. Generated-by: Claude Opus 5

…d sidecar Every framework skill carries the pre-flight block always, so every token in it is paid on every invocation of every skill. Most of it was branch handling for outcomes that almost never occur: what to do when a plugin sits below the floor, when a fingerprint differs, when the verify interval has elapsed. That detail now lives in `tools/dev/preflight-detail.md`, propagated as a whole `preflight-detail.md` file beside each skill's `SKILL.md`, which the block points at and the agent reads only when a check actually fails. The block is 1,754 tokens instead of 3,271, so the reconciliation check costs each of the 65 skills +90-91 tokens over its pre-check size rather than +1,608 — +2.8% on the smallest skill in the catalogue instead of +49.0%. What stayed in the block is every rule that has to bind whether or not the sidecar was read: the prohibitions, the unknown-is-not-absent rule, and the two things `config` may not do. The hot text decides one thing — whether to stay silent — and says so at the top: never act on a non-silent outcome without reading the sidecar, and if it cannot be read, say so rather than improvising the branch. All seven graded reconciliation eval cases still pass, including the five that now route through the sidecar. The reason this was not done when the check shipped was wrong, and the correction is recorded in the design's Risks. The claim was that the detail could not move behind a pointer because the target sits in the framework snapshot or the plugin cache, which a sandboxed session cannot read. That conflated two policies: the Bash sandbox denies those paths, the agent's own file-read tool does not — verified by reading one plugin-cache file with each, one refused and one served. A second constraint was real and shapes the result: Agent Plugins 1.0 forbids a symlink escaping the plugin root, so a shared `skills/_shared/` include is unreachable. A sibling is not — `plugins/<family>/skills/<skill>/` is the real directory `skills/<skill>` symlinks into — so the sidecar lands physically inside the plugin and needs no path to reference it. One caveat survives, on Gemini's extension install alone, where the sidecar lands outside the workspace and a native read may fall back to an approved shell read that prompts; that lands only on the cold path. Mechanically this is the propagation tool learning a second shape: a whole generated file, not a region inside one. `skill-surface-hash.py` excludes the sidecar by name, without which every edit to the shared detail would move all 65 digests and tell every adopter their configuration went stale — no digest moves here. `check-duplication.py` skips the copies and scans the source in their place. The eval harness gains `also_include`, because a prompt built from the block alone would grade the routing and never the branch it routes to. Also fixed in passing: the block source carried a doctoc TOC, which rode into all 65 copies as a table of contents for a file none of them are. The source joins the doctoc hook's exclude list, and the propagator strips a leading doctoc region so a future source cannot repeat it. Generated-by: Claude Opus 5

…ecar too The first split left four steps whose entire body fires only on a branch that already routes to `preflight-detail.md`, so they were paying for themselves on every invocation of every skill and earning it on almost none: the snapshot-method remedies (step 2), the below-the-floor restart notice (step 5), the vetted-ops proposal (step 9) and the verify suggestion (step 10). Each keeps a stub that decides whether it is silent and nothing more. Step 4's prose is tightened in place — it is the step this branch introduced, and it was the largest remaining. Step 8 deliberately stays whole. "Never run `/magpie-setup adopt` unattended" is a prohibition whose value is that it binds whether or not anything else was read, and it guards the one action here that would commit a recommendation into every contributor's checkout. The block is 1,540 tokens, down from 3,271 un-split and — the part worth stating plainly — **139 below the 1,679 it was before the reconciliation check existed at all**. Every one of the 65 skills carrying it is 120-126 tokens cheaper than it is on `main` while gaining the whole check; `ci-runner-audit` goes 3,281 to 3,158. All seven graded reconciliation cases still pass, and no `surface_hash` digest moves. Generated-by: Claude Opus 5

Three steps still carried their reasoning inline although their bodies only fire on a branch. Step 9's framing ("not a pre-flight check") became a shared lead-in for the pair, its `never apply` prohibition moved next to the proposal it constrains, and step 10's config-resolution chain moved to where the value is explained. Step 8 keeps the prohibition itself — `adopt` commits a recommendation into every contributor's checkout, the block is the only place that rule is written, and it has to bind whether or not the sidecar was read — while the mention it asks for, which is conditional on step 7 having written something, moves out. The block is 1,448 tokens: 231 below the 1,679 it was before the reconciliation check existed, so each of the 65 skills carrying it is 212-218 tokens cheaper than on `main` while gaining the whole check. `ci-runner-audit` goes 3,281 to 3,066, -6.6%. This is where the split stops, and the reason is now written into the design. What remains inline is each step's own test for whether it is silent, and a step cannot know it has nothing to say without evaluating that test — so moving the triggers behind the pointer would mean reading the 2,335-token sidecar on every run to save 1,448. The same arithmetic rules out replacing the block with a bare pointer. Graded reconciliation evals 7/7; no `surface_hash` digest moves. Generated-by: Claude Opus 5

potiuk added 30 commits September 22, 2026 00:26

chore(skills): add the generated surface_hash field

21089ba

Generated-by: Claude Sonnet 4.5

feat(dev): generate a reconciliation fingerprint for every skill

89dad39

Generated-by: Claude Sonnet 4.5

feat(setup): verify sweeps reconciliation and reports newer plugin ve…

b644133

…rsions Generated-by: Claude Opus 5

potiuk added 3 commits September 22, 2026 00:29

potiuk added family:setup setup-* skills capability:platform Framework / agent substrate skills (install, verify, doctor, override, status, setup bootstrap) labels Sep 21, 2026

potiuk added 2 commits September 22, 2026 00:56

potiuk added 11 commits September 22, 2026 01:08

Navigation Menu

feat(setup): reconciliation tracking for marketplace installs - #1324

feat(setup): reconciliation tracking for marketplace installs#1324
potiuk wants to merge 46 commits into
apache:mainfrom
potiuk:design/marketplace-reconciliation

potiuk commented Sep 21, 2026

Uh oh!

potiuk commented Sep 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

potiuk commented Sep 21, 2026

Summary

Type of change

How it works

Test plan

RFC-AI-0004 compliance

Notes for reviewers

Uh oh!

potiuk commented Sep 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant