Skip to content

claude-ops/audit-native-overlap: add the integration axis to the store and the wrap and suggest grammars to native-references #4049

Description

@kyle-sexton

Parent

Refs #4047. Unit 2 of 9 (policy unit, plan Phase 4). Predecessor: #4048 (tooling fix). Human-gated by class: this unit adds a required field to the schema-1 store and takes a major version of the native-references convention, which is a contract change (C4, work-class: structural), so it carries needs-human and a person reviews and merges. Every decision the unit needed from a human is now recorded below (2026-09-11), so a worker can execute it end to end; the PR body re-quotes the verdict table for the user's final confirmation.

What to build

The native-surfaces store gains a runtime-relationship axis beside verdict, the native-references convention gains the two grammars that axis needs, and every pre-flight the sweep depends on is answered here so no later unit reverts a verdict.

Pre-flights, answered 2026-09-11 and quoted in the PR body before any verdict is written:

  • Model-invocability per row, keyed per registration from the fixed extraction. On 2.1.263, doctor (disableModelInvocation:!0, survivesBundledKillSwitch:!0, terminalOriented:!0) and design-sync are model-disabled; simplify, run, code-review, and the design canvas skill carry no invocation-control field and are model-invocable. The name design carries two bundled registrations: the canvas skill (registerDesignCanvasSkill, the surface both design rows describe) and the claude.ai/design hub (model-disabled, no row describes it), plus a local access command. A model-disabled bundled skill cannot be invoked by any other skill, so its rows take suggest, never wrap and never route (a route phrase on a surface the model never lists is dead text).
  • simplify takes a [<target>] argument and its prompt prepends the review target, so the batch-simplify wrap scopes it per file set.
  • CI lane: allowed-tools grants and never restricts (official skills reference: "It does not restrict which tools are available: every tool remains callable"), so the review:security-review frontmatter proves nothing about reach, and the official skills reference lists /security-review among the built-ins the Skill tool can invoke. The row takes route because the lane runs a pinned reusable workflow in another repository whose tool allowlist and plugin roster this repository does not control, and because the native command is unusable there (it diffs against origin/HEAD, which the Actions checkout does not provide; the skill body records this).

Store schema (additive):

  • integration is required on every row, one of route, wrap, suggest. Class rules: builtin-command takes route or suggest; bundled-skill, plugin-backed-builtin, and marketplace-plugin take route or wrap; a bundled-skill row carrying the new marker model-invocation-disabled takes suggest only; session-skill takes route; a defer verdict takes route. Every wrap or suggest row carries an evidence line naming the observed invocation mode, because Skill-tool reach is a per-surface property (the class rules are a floor, not the determinant). NATIVE_MARKERS gains model-invocation-disabled, set from the extraction registration the row's evidence names, never from the bare name.
  • baked gains booleans native_step (true only with wrap) and suggest_sentence (true only with suggest).
  • Forward parity: native_step looks for the literal heading ## Native step: <name> (<class>) in the component body; suggest_sentence looks for the sentence shape If /<name> is available in your session ( in the body. Reverse parity gains a body scan for that sentence shape only, keyed on the shape and never the bare phrase (the claude-ops:changelog body already contains "available in your session" in unrelated prose and is a negative fixture).
  • The generated view renders an Integration column in the summary table and an Integration: line per row.
  • Two new rows: export against session-flow:handoff and export against session-flow:retro, both suggest, so every existing suggestion site traces to a row.
  • Re-derive observations before the verdict pass: run detect on the fixed extraction and refresh observation.detail and recheck.verified to 2.1.263 for every extraction-class row whose native surface the extraction resolves and whose evidence still holds; rows whose evidence moved keep their record and gain an evidence line saying what moved; every bundled-skill row gains or drops the model-invocation-disabled marker from the registration its evidence names. Reason-text corrections in the same pass: the export row's "built-ins are user-invoked only" becomes the per-surface reason (/export and /skill-doctor are non-prompt command types the Skill tool never lists; /init and /security-review are prompt-typed and reachable); the security-review row's reason rests on the CI-harness and origin/HEAD facts, not on allowed-tools; the code-review row's reason rests on cost and shape (the native surface fans out its own agents; the CI lane cannot be typed into a session), not reach; the two design rows name the canvas registration, the hub registration, and the local access command, with the recheck trigger widened to "which registrar wins isEnabled"; the morning row gains a second live-roster observation (2026-09-11, cloud session); the skill-doctor row notes the native report excludes bundled skills.

Convention (one major bump of native-references; the CHANGELOG entry cites the triggers):

  • wrap grammar: a body section ## Native step: <name> (<class>) carrying, in order: the gate token resolves in your session; the identity check by class (bundled: name in the listing, invoke by alias where the Skill tool resolves one, advisory description check; a description that reads as a different surface is a likely user or project shadow, skip with a warning, except where the bundled surface itself defers to a project skill of the same name, as run does; a name with no description, which name-only and budget overflow both produce, is invoked with a stated "identity confirmed by name alone" warning; plugin: namespaced form plus marketplace provenance when the CLI resolves); the mutation clause (a mutating surface may be wrapped only where the wrapping skill's own contract mutates the same thing, the invocation passes an explicit scope or report-only argument, the skill fingerprints what the surface may write before the step and diffs after, and an unexpected diff is reported as "mutation detected after a scoped invocation", with the run exiting degraded); the invocation form; what our part adds before or after; the skip-and-report contract for the states did not resolve in this session, invocation refused (the report names the reason and never retries: not in the session's skills allowlist, disabled for model invocation by disableBundledSkills or skillOverrides, or a permission deny, which is alias-aware), identity mismatch, mutation detected, and resolved but degraded (the surface ran a weaker procedure and said so, as simplify does without the Agent tool; the wrapper relays that disclosure and never restates the step as the full procedure), each naming the axis line settings or environment, plan, platform or provider, host surface and the enable path; a note that the wrapped body enters context once and stays there; and the bare unattended argument, declared by the caller, under which the skill records instead of asks and never invokes a mutating surface.
  • suggest grammar: the sentence If /<name> is available in your session (<basis>), run it for <job>., addressed to the person, where <basis> is a same-file four-part verification record per upstream-drift naming the surface's own gate (for /doctor: DISABLE_DOCTOR_COMMAND or a skillOverrides entry, since it alone survives disableBundledSkills; for every other bundled skill, disableBundledSkills as well); placed at the start of the run when the surface covers everything the skill does and at the end when coverage is partial; a model-disabled bundled skill is suggested as /<name> exactly as a built-in command is; the wording is "reserved for the person to run", never "cannot be invoked" as an absolute; unattended runs record it in output. The convention states that the suggest token differs from the route token by design (model-observed versus person-checked), is not the rejected assertion phrasing always available, and that parity keys on the sentence shape.
  • Class table with the marketplace-plugin row naming seam-phrasing as the owner of that class's wrap grammar and the model-disabled bundled-skill row (suggest only).
  • Amended one-owner rule: one owning description phrase per plugin per surface; a second skill in the same plugin carries a Native step or suggest section with a same-plugin pointer to the owner's Boundary, never a second phrase. The rule governs description phrases only, so sibling skills may each carry a suggest sentence for the same surface.
  • The Enforceability row for the gate-token check moves from "candidate check named, not built" to built (unit 1 built it). Major-bump triggers per the convention's own Versioning section: an enforceability verdict changes, and the canonical-token clause gains a second token; the one-owner amendment in the Boundary section contributes.

Verdict table, confirmed by the user 2026-09-11 and validated by two independent fresh-context research agents (Fable 5.1, Opus) against the official skills, commands, settings-reference, sub-agents, env-vars and errors references, the 2.1.263 binary, and this repository's conventions. The PR body re-quotes it for final confirmation.

Native Class Component Verdict Integration Basis
doctor bundled-skill, model-invocation-disabled claude-ops:audit-install-state complementary suggest model-disabled; owning sentence, end of run (partial coverage)
doctor bundled-skill, model-invocation-disabled claude-ops:audit-skill-visibility complementary suggest model-disabled; owning sentence, end of run
doctor bundled-skill, model-invocation-disabled claude-ops:audit-performance complementary suggest model-disabled; pointer-carrying sentence to audit-install-state's Boundary; route would be dead text and its sentence an orphan
skill-doctor builtin-command claude-ops:audit-skill-visibility complementary suggest local-jsx/local command type, never a prompt; not Skill-tool reachable; native report excludes bundled skills
simplify bundled-skill code-tidying:batch-simplify complementary wrap model-invocable; takes a target; batch-shaped host mutates the same thing
simplify bundled-skill code-tidying:tidy complementary route simplify works the current diff, tidy works a rotated lane of resting code with no diff; simplify applies fixes, tidy is structure-only
run bundled-skill testing:run-e2e complementary wrap model-invocable; starts processes, tracked tree is the fingerprint; a project run is a legitimate target
code-review bundled-skill review:code-review complementary route cost and shape: native fans out its own agents; CI lane cannot be typed into a session
security-review plugin-backed-builtin review:security-review complementary route Skill-tool reachable per docs, but the CI harness in another repo owns the tool roster and the native command is unusable there (origin/HEAD)
morning session-skill claude-ops:morning-brief defer route defer forces route; second live observation added
export builtin-command session-flow:clean-stop complementary suggest local-jsx command type, not Skill-tool reachable
export builtin-command session-flow:handoff (new row) complementary suggest same; body already carries the sentence
export builtin-command session-flow:retro (new row) complementary suggest same; body already carries the sentence
playground marketplace-plugin visualization:visualize complementary route seam-phrasing owns that wrap grammar; playgrounds:use is the wrapper
playground marketplace-plugin prototype:explore-directions complementary route same
design (canvas) bundled-skill visualization:visualize complementary route canvas registration is model-invocable, so wrap is admissible; route on job merits (component owns the artifact job, no sub-step to wrap); no marker; revisit in unit 7
design (canvas) bundled-skill prototype:explore-directions complementary route same; revisit in unit 8
design-sync bundled-skill, model-invocation-disabled visualization:visualize defer route defer forces route; hidden and model-disabled

Totals: 2 wrap, 7 suggest, 9 route, 18 rows. Each row's evidence gains the substrate observations from the 2026-09-08 report where they apply (skill-doctor gate basis; design name collision) and a budget_caveat note that audit-performance and audit-skill-visibility have under 60 characters of description headroom.

Apply step (audit-native-overlap skill body): preconditions add "the row's integration is not route when a Native step or suggest sentence is to be written" and "the row does not carry model-invocation-disabled when a Native step is to be written"; the emitted artefacts add the Native step section and the suggest sentence; the per-entry cap precheck is a hard precondition (description plus when_to_use after baking at most 1,536 characters, measured by the skill-quality check); every wrapped or suggesting skill declares unattended in its argument-hint.

Version: next minor of claude-ops at PR time.

Research record

Both validators fetched https://code.claude.com/docs/en/skills.md, commands.md, settings-reference.md (plus sub-agents.md, env-vars.md, errors.md) on 2026-09-11 and grepped the 2.1.263 binary bytes. Load-bearing quotes: skills reference, invocation table for disable-model-invocation: true: "You can invoke: Yes | Claude can invoke: No | Description not in context"; skills reference, Skill tool: "A few built-in commands are also available through the Skill tool, including /init and /security-review. Other built-in commands such as /compact are not."; skills reference, allowed-tools: "It does not restrict which tools are available: every tool remains callable"; settings reference, disableBundledSkills: "/doctor stays typable like the built-in commands; to hide it, set DISABLE_DOCTOR_COMMAND instead"; binary: eo({name:"doctor",...survivesBundledKillSwitch:!0,...userInvocable:!0,disableModelInvocation:!0,...}), registerDesignCanvasSkill registering name:r with r="design" and no invocation-control field, {type:"local-jsx",name:"export",...}, {type:"local-jsx",name:"skill-doctor",...}, the Skill tool listing filter e.type==="prompt"&&!e.disableModelInvocation, and the /init and /batch bundled prompts invoking update-config and code-review through the Skill tool (first-party precedent for wrap). Work-class basis: plugins/work-items/reference/work-class-labels.md ("work-class: structural (C4): Refactors, migrations, contract changes"; a C4 item "must never carry" the autonomous-eligible label) and plugins/autonomy/reference/guardrails/work-classes.md.

Acceptance criteria

  • python3 plugins/claude-ops/skills/audit-native-overlap/scripts/overlap.py self-check --upstream-sha ed404106fcd80ba98ecb7c851e531dcb626d13b7 exits 0 with no problems and no advisories.
  • jq '[.rows[] | select(.integration == null)] | length' docs/native-surfaces/records.json prints 0; jq '.rows | length' docs/native-surfaces/records.json prints 18; jq '[.rows[] | select(.integration=="wrap")] | length' … prints 2 and … select(.integration=="suggest") … prints 7; python3 … overlap.py generate --check exits 0.
  • jq '[.rows[] | select(.native.class=="builtin-command" and .integration=="wrap")] | length' docs/native-surfaces/records.json prints 0; jq '[.rows[] | select((.native.markers | index("model-invocation-disabled")) and .integration!="suggest")] | length' … prints 0; jq '[.rows[] | select(.native.name=="design" and (.native.markers | index("model-invocation-disabled")))] | length' … prints 0; jq '[.rows[] | select(.verdict=="defer" and .integration!="route")] | length' … prints 0.
  • grep -c "built-ins are user-invoked only" docs/native-surfaces/records.json prints 0.
  • docs/conventions/native-references/README.md contains ## Native step: <name> (<class>), is available in your session (, marketplace-plugin, model-invocation-disabled, and resolved but degraded at least once each, and its CHANGELOG records a major bump citing the enforceability row and the token set.
  • python3 plugins/claude-ops/skills/audit-native-overlap/scripts/test_overlap.py exits 0 with one test per class rule (including the model-disabled suggest-only rule and the evidence-line requirement on wrap and suggest rows), the two new baked flags, and the suggest-shape parity in both directions.
  • The PR body quotes the three pre-flight answers and the verdict table above; the user confirms the table in the PR; the PR is a draft until green; a human reviews and merges.

Blocked by

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.priority: mediumReal value, no hard deadline; normal backlog flow.work-class: structuralRefactors, migrations, contract changes; cross-cutting and hard to reverse.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions