Agree what /architecture:improve should and should not do, and fix six frictions found running it end to end

[kyle-sexton]

Why this issue exists

/architecture:improve deepening was run end to end over this repository for the first time at
real scale: a six-subagent scan, verification of all 21 candidates it produced, an HTML report, a
durable candidate artifact, and then implementation of every candidate across 18 commits (draft PR
#4059).

The skill did its job. The scan found real duplication, and the verification phase earned its cost.
But running it at that scale surfaced six concrete frictions and four questions about scope that
are genuinely the maintainer's call, not mine. This issue records both so we are on the same page
about what this skill is for before anyone extends it.

Evidence for every claim below is one run, named where it matters.

Four decisions reserved for the maintainer

These change what the skill is, so I am not deciding them. My recommendation is stated for each.

Q1. Is there an implement-all-candidates mode, or is this one candidate per run?

Today the skill assumes one selected candidate: the interview loop, the agreed-shape field and
the handoff are all singular. This run was asked to implement all 21 in one pull request, and the
skill had nothing to say about it. Everything about running 21 candidates, ordering them by
leverage, fencing parallel workers, deciding commit granularity, was invented outside the skill and
is not written down anywhere.

My recommendation: keep the skill single-candidate, and add a short handoff section naming what
changes at bulk scale rather than absorbing the execution. The skill's own guidance is that an
interview settles decisions and never becomes the execution container, and a bulk mode would make
the corpus size the session's length. But the handoff currently goes silent exactly where a
maintainer needs it most.

Q2. Does the skill own a scan fan-out and cost budget?

Phase 1 dispatched six parallel exploration subagents with no stated cap and no cost guidance. That
is the single largest cost in the run by a wide margin, and the skill never says what a sensible
fan-out is or what it costs.

My recommendation: state a default fan-out, and a one-line cost expectation, in the skill body.
This repository already has the precedent: .claude/rules/hook-budget.md holds a fixed latency
budget for always-on hooks and says the budget never relaxes to absorb an overage. A scan budget is
the same idea applied to tokens. The number itself is yours to pick.

Q5. Does the HTML report cap or group cards above a threshold?

Twenty-one candidate cards is more than one page reads well. The report has no cap, no grouping and
no pagination, so a large repository produces a document the reader scrolls rather than scans.

My recommendation: group by recommendation badge and collapse everything below the top band by
default. The report already computes a badge per candidate and already has a Top recommendation
section, so the grouping key exists.

Q8. What fencing discipline do parallel implementation workers follow?

Downstream of Q1. When multiple workers implemented candidates concurrently in this run, two
collided on the same file before I imposed a rule that each worker owns a disjoint set of paths and
is told which paths another worker owns. That rule worked, and it is written down nowhere.

My recommendation: if Q1 keeps the skill single-candidate, capture this as one paragraph in the
handoff section rather than as machinery.

Six frictions, with the fix

These are not scope questions. Each is a defect or a gap with an evidenced cause.

1. The skill's own documented step is blocked by a sibling plugin in this marketplace

reference/topic-docs.md says the memory root's self-ignore guard applies on first write,
"verify-or-create .gitignore with *". It does not say how. The natural shell form is a redirect,
and the guardrails plugin in this same marketplace denies it:

BLOCKED: echo/printf > file write bypasses Write/Edit hooks

Reproduced in this run against printf '*\n' > .work/.gitignore. One marketplace plugin's
documented step is refused by its sibling, and the agent has to discover that by being denied.

Fix: name the mechanism. The guard is created with the Write tool, not a redirect. This is a
one-line change to the binding, and it applies to every skill that writes into the memory tier, not
only this one.

2. Nothing checks the branch is current before a long run

A deepening pass is long. This run's branch drifted eight commits behind its base while the work
was in flight, and one of those commits renamed a git config key inside a file that two candidates
had restructured. The result was a merge conflict that needed hand resolution at finalize time,
after every candidate was already committed, which is the most expensive moment to find it.

Fix: add a base-freshness check to the start of the deepening action, and recommend re-checking
before the handoff. Cheap, and it moves the conflict to the point where it costs least.

3. Phase 1.5 verification works and should say so in its output

The verification phase corrected two of 21 candidate claims that would otherwise have reached the
report as fact. That is a real hit rate on a gate people will be tempted to skip for speed.

Fix: keep the gate, and record the correction count in the candidate artifact. A reader can
then see the gate ran and what it caught, which is the argument for not skipping it next time.

4. Two artifact fields read as unfinished when the interview is skipped

Every one of the 21 candidate entries carries an empty agreed-shape and graft-record, because
the work went from verified candidate straight to implementation without the interview branch. An
empty field reads as an abandoned artifact rather than as a path deliberately not taken.

Fix: make both fields explicitly optional, with a recorded reason when the direct-to-
implementation path is used, so the artifact says which path it took.

5. Validation guidance stops before a worker needs it

Workers dispatched to implement candidates were told to validate, but the skill offers no guidance
on what a green result actually requires. In this run one worker reported a suite green that it had
run against a 41-byte stub binary which echoed its arguments and exited zero. The real binary was
installed later and the suite does pass, but the earlier green was worthless and nothing in the
process would have caught it.

Fix: one line in the scan briefing and any worker-facing guidance: a suite that skips for a
missing tool is not a pass, and a stub standing in for the tool is not the tool.

6. The skill does not say to sweep names it retires

One candidate retired seven library helpers. Every live call site was migrated correctly, but the
retired names survived in comments across eight files and in two test assertions that asserted
against a substring, so they would have passed against either spelling. Nothing failed; the repo
simply carried stale references and two weakened assertions.

Fix: add retiring a public name to the deepening checklist as a step that sweeps comments and
assertions, not only call sites. Note the exception found here: two plugin README passages record
what a past version dropped, and renaming those would falsify the record.

Acceptance criteria

• reference/topic-docs.md names the Write tool as the mechanism for the memory-root guard, and no
  documented step in this plugin is refused by guardrails.
• The deepening action performs a base-freshness check before the scan, and recommends one before
  the handoff.
• The candidate artifact records how many claims Phase 1.5 corrected.
• agreed-shape and graft-record are documented as optional, and a direct-to-implementation run
  records that as the reason rather than leaving them empty.
• Worker-facing guidance states that a skipped suite is not a pass and a stub is not the tool.
• Retiring a public name is a checklist step covering comments and assertions as well as call sites.
• The four reserved questions above are answered in this issue's comments before any of them is
  implemented.

Captured assumptions

• This was produced by /planning:interview in unattended mode: nobody was available to answer
  during the run. The four reserved questions are recorded as blocked rather than decided, per the
  skill's auto-guard.
• The acceptance-criteria coverage prompt was skipped for the same reason, so
  unwanted-behaviour and state-driven criteria coverage went unexamined.
• acceptance_criteria_format resolved to free-text (untagged bullets) because no
  convention-home pointer region is bound in the root instruction file, so the resolver exits 1
  and the ladder degrades to the documented default.
• Every observation comes from one run over one repository. A second repository might weight these
  differently, particularly the report-size question.

Out of scope

• Changing /architecture:map-landscape or /architecture:record-decision. Neither was exercised.
• The improvement:find boundary. The skill's existing routing note held up in this run.
• Any change to the Ousterhout vocabulary or the badge heuristics. Both worked as written.

Related

• Draft PR refactor: twenty-one module deepenings across the hook library, guardrails and the check scripts #4059, the 21 implemented candidates, which is where every claim here was observed.
• .claude/rules/hook-budget.md, the precedent cited for Q2.
• docs/conventions/topic-docs/, which owns the guard referenced in friction 1.

[kyle-sexton]

Resolutions for the four reserved questions

The maintainer's instruction was to resolve these "as long as we're backed by consensus
based research from official, authoritative, and trusted sources". That is conditional, so
each question was researched first by an agent that gathered live sources and was forbidden
to recommend anything, then decided by a separate agent that saw only the evidence packet
and was required to return UNSETTLED where the evidence did not support a resolution.

Every external page was fetched live on 2026-09-11. Nothing below is stated from recall.

Read the basis labels. Only one of these four resolves on research consensus. The
others resolve on repository rules, on a verified negative, or on judgment, and they are
labeled so nobody later cites them as something they are not.

Question	Verdict	Basis
Q1 bulk mode	Resolved, one residual open	repo-rule plus judgment, not research
Q2 scan fan-out	Rule resolved, number unsettled	research-backed rule, no defensible constant
Q5 report structure	Resolved, no threshold emitted	research-backed
Q8 worker fencing	Resolved	official-doc plus primary-literature plus repo-precedent

---

Q1. Does the skill gain an implement-all-candidates mode?

Resolved: no. The skill accepts many selected candidates in one run, interviews each to
its own agreed shape, and hands the ordered set off as one artifact. It never implements
them. One residual stays open.

The argument this issue proposed for that answer does not survive verification. The
original framing leaned on native-first: Claude Code ships /batch, and
docs/PLUGIN-PHILOSOPHY.md says never custom-build what a fitting native mechanism covers.
Checked against the /batch description, it does not fit on three counts. It takes an
instruction and does its own decomposition, while this case arrives with 21 candidates
already identified and designed. It produces independent units, while architecture
deepenings overlap files and carry ordering. It opens a pull request per unit, while the
request here was one pull request for all 21.

This repository had already run that exact gate and recorded the result, in
plugins/docs-hygiene/skills/extract-ssot/actions/batch.md, verified 2026-09-06 against
Claude Code 2.1.263: an in-skill batch action over an already-produced candidate list is a
distinct concern from bundled /batch. Native-first is therefore recorded here as a
verified negative. It decides nothing about this question and should not be cited as
though it did.

The tripwire in planning:interview does not fire either. Its scope clause is a settled
shape applied across a corpus, and its test is whether question count scales with source
units rather than genuine forks. The 21 candidates are 21 different shapes each carrying a
real interface-design fork, so each is a genuine fork. What does bind is that rule's first
sentence, that an interview settles work and never executes it, reinforced by this skill's
own line: "Does not write code. Discovery and design skill only."

The change is smaller than the issue implied. The candidate artifact is already plural:
status: proposed | selected | agreed-shape | rejected is a per-entry field. The
singularity lives only in Phase 3 prose and the handoff paragraph.

What a multi-candidate handoff needs that a single one does not, and what the skill should
record: the order candidates land in and why, the file overlaps between them, and which
candidates depend on another landing first. Commit granularity and pull-request shape stay
the execution lane's call and are carried into the handoff rather than decided here.

Still open for the maintainer: whether that becomes an invocable action or stays prose
in Phase 3. extract-ssot did the former, batch-simplify did the latter, and nothing
decides between them. Recommendation, labeled judgment: prose first, promote it if the
ordering logic ever grows a real algorithm.

Strongest objection, and it is real. Overexposure, in Ousterhout's sense: every
single-candidate run, the common case, now carries schema fields that stay empty and reads a
bulk paragraph it never uses. Scoping the paragraph by its opening condition reduces that
cost without zeroing it.

---

Q2. Does the skill own a scan fan-out and cost budget?

Split verdict. The rule resolves. The number does not.

Resolved, research-backed: the skill states how the scan width is derived. Anthropic
reports the exact failure and the exact remedy: "Agents struggle to judge appropriate effort
for different tasks, so we embedded scaling rules in the prompts." A skill body is that
prompt. The agent-teams documentation independently does the same thing in prose and names
the constraints that justify it: token costs scale linearly, coordination overhead
increases, diminishing returns.

The rule is a derivation, not a constant: one scan subagent per area the scope decomposed
into, because the right width is a property of the repository being scanned and not of the
method. Named out loud before dispatch, so the run's largest cost is a decision the operator
sees rather than a side effect.

Unsettled: the number. Every candidate constant fails its own justification test, and
the vendor guidance bars unjustified values by name.

• The agent-teams "3 to 5" is calibrated for teammates whose stated reason for staying low
  is coordination overhead. Scan subagents do not coordinate with each other, so carrying
  the number over would keep the constant and strip its justification.
• The research-system ladder is rung-assigned by research task shape, and the same source
  says coding parallelizes less. Assigning a rung to "codebase friction scan" is an
  assignment that source never made.
• No documented default fan-out width exists anywhere. The only vendor numbers for
  subagents are ceilings, 20 concurrent and depth 3, not defaults.

Also unsettled: a stated cost figure. The vendor's answer to cost visibility is
architectural rather than editorial, since /usage already attributes spend per skill and
per subagent. What is supported is the mechanism sentence, that cost rises with the count,
not a number.

What would settle the number: run this lens on one scope at widths 1, 3 and 6 and
compare distinct candidates surviving the Phase 1.5 gate against tokens spent. Three or four
runs would license a repo-internal default with honest provenance.

Noted inconsistency worth closing regardless: the same skill already states 3 to 4
subagents for its Design-It-Twice fan-out and says nothing for the more expensive scan, and
the Design-It-Twice number is asserted in evals.json while the scan's silence is not.

---

Q5. Does the HTML report cap or group cards above a threshold?

The principle resolves. No threshold is emitted, because none exists to emit.

Four independent source families converge on the remedy for a large findings set, and the
remedy is order and label, not remove: chunking into meaningful groups (NN/g), Miller's
own load-bearing finding, scoping plus a quality bar (Google's static analysis practice),
and severity and rank as first-class fields with a severity facet as shipped precedent
(SARIF and GitHub code scanning). None of the four is a cap. None is a collapse.

Why there is no threshold question left to answer. Capping, paginating and collapsing
are conditional mechanisms, and each needs a number nobody has. Ordering and banding are
unconditional: identical at 3 candidates and at 30, hiding nothing, needing no number.

Decision: order candidates by the recommendation badge, group each strength into its own
band, unconditionally and at every count, with no cap, no pagination, and no hidden or
collapsed card. The key already exists and is already verified upstream, since Phase 1.5
reproduces the shallow signal behind every Strong, so the bands differ in how far a reader
can trust the claim.

On "seven plus or minus two", since it is the rule most likely to be reached for here:
it is rejected for this use by NN/g across three authors and three dates, and by Miller
himself in the source paper, who calls treating the spans as one process "a fundamental
mistake" and the coincidence "a pernicious, Pythagorean coincidence". A scrollable report is
a recognition task with all options continuously displayed, which is not what that paper
measured. It licenses nothing here.

A no-JavaScript collapse was checked and rejected on evidence, not on feasibility.
<details> works within the inline-only constraint. It was declined because NN/g says
accordions are wrong "when your audience requires the majority or all the content on the
page to find answers", which is the condition someone who ran an architecture scan meets.

Two corrections to this issue's own claims, both verified against the file:

1. The report does not already lead with its best finding. #candidates renders before
   #top-recommendation, so the conclusion sits at the bottom and its anchor points upward.
2. There is no ordering rule anywhere in the pipeline, not in the report spec and not in the
   scan action.

Strongest objection: nothing in the research demands any change at 21 cards. The only
item-count figure in the sweep is roughly 100 items per page, five times the observed count,
and the badge is already on every card. Declining this change contradicts no source.

---

Q8. What fencing discipline do parallel implementation workers follow?

Resolved, and the issue's framing of this question was wrong.

This issue recorded the improvised rule as a possible workaround for something the platform
already solves. Verified against implement-dispatch/SKILL.md, it is not. Worktree
provisioning there happens "once per item, on the first dispatched phase", and the skill
caps "concurrent dispatch waves at 3-5 workers by default". So workers running concurrently
inside one item share one checkout by design. Worktree isolation separates items from
each other; it never separates two workers in the same wave.

The official routing rule is an if/else on surface capability: isolate with worktrees where
the surface supports it, and partition by file ownership where it does not. This repository's
own dispatch contract puts a wave on the partition branch. Path fencing was the correct
mechanism, not a workaround.

The precise gap in the current contract. The fence enumerates ALLOWED and FORBIDDEN, and
the divergence-escalation clause fires only on touching something FORBIDDEN. A path sitting
in a sibling's ALLOWED set and in nobody's FORBIDDEN list is outside that clause, so the
worker edits it and never stops. Two well-formed fences can overlap with nothing to catch it.

The discipline, each rule labeled:

1. Each item's workers edit in a worktree that is not the session's default checkout.
   Already owned, no change. official-doc, repo-precedent
2. Workers dispatched in the same wave hold mutually disjoint ALLOWED sets, composed
   together before any of the wave is dispatched. This is the missing clause.
   official-doc, primary-literature (write-write conflict is defined by the shared item)
3. Each worker's FORBIDDEN list covers the paths the other workers in its wave own, since
   that is what brings an overlap inside the escalation clause. repo-precedent
4. Disjointness is measured on the file written, not the checkout it is written in. Two
   workers in separate worktrees writing one path produce a merge conflict at integration
   rather than an overwrite at edit time. primary-literature
5. Work that will not split into disjoint fences does not go out as a wave. official-doc
6. The brief lists the paths, never the worker who owns them. This drops half the
   improvised rule. Attribution adds no enforcement the FORBIDDEN list does not already
   carry, and it goes stale as soon as a wave is recomposed. judgment; no source supports
   either form

Placement: implement-dispatch, which is the only surface that composes worker briefs
and already owns the fence vocabulary. The architecture skill carries none of this: its
only parallelism is a read-only design fan-out, and a write-write conflict needs at least one
write. That is an evidence-backed none, not a deferral. No new rules file either, since
.claude/rules/ governs contributors editing this repository while this discipline governs
the runtime behavior of a plugin that ships to consumers.

Strongest objection, and the best follow-on in this whole set: worktree isolation is
enforced by checks that cannot be bypassed, whereas this remedy is another instruction an
orchestrator can forget. A deterministic pre-dispatch check that intersects the wave's
ALLOWED sets and refuses on any non-empty result would be strictly stronger, in the same
shape as the existing comment-hygiene and exec-bit checks. It was not proposed because
nothing in the evidence establishes a pre-dispatch hook point to run it in. If the rule
should hold under pressure rather than under attention, that check is worth scoping.

---

Process notes

• Research and resolution were deliberately separated. Researchers were forbidden to
  recommend; resolvers saw only the evidence and were required to return UNSETTLED where it
  ran out. Two did.
• Resolvers ran on Opus. The Fable quota was exhausted partway through, and Opus was the
  named alternative.
• One researcher flagged that "one skill, one job" advice circulates attributed to Anthropic
  but could not be found on any official page. It is treated as absent from the vendor record
  and is not cited here.
• Three citations could not be fully verified and are marked in the working notes rather than
  relied on: a paywalled CACM paper, a truncated context-engineering page, and an X post
  behind a bundled playbook claim that returned 404.

---

Generated by Claude Code