feat(code-metrics): clone classes, explicit caps, registry cluster lines, and rollups for audit-duplication

[kyle-sexton]

Closes #4068

Summary

/code-metrics:audit-duplication reported detector pairs as groups, so eighteen byte-identical copies of hook-utils.sh arrived as seventeen rows and the fragment's lines counted seventeen times; a 1.4 MB minified bundle was silently left out of the scan by jscpd's own size guard; the root canonical of a vendored file could not be declared in the sanctioned-replication registry; and the markdown read like a size report (Files. Functions. Over reference.) with no way to see which lane or directory carried the duplication. This PR fixes all of that in the code-metrics plugin (0.2.3 to 0.2.4), adds the registry cluster-line grammar this repository needed, and records the five clusters.

Fix

• Explicit caps, reported instead of hidden. duplication.max_size (default 1mb, binary units) and duplication.max_lines (default null) are applied by the jscpd adapter before the tool runs, because jscpd 4.x and 5.x disagree on their --max-size/--max-lines defaults and on what 0 means, and neither names a skipped file. A skip makes the lane's run row partial (count and largest file), the document partial, and the summary carries a Partial: line. Both majors are translated; reference/collectors.md pins 5.2.0 and records 4.3.0.
• Clone classes. New cluster-clones.py merges pair rows that share an identical (file, start_line, end_line) and equal lines into one row per class (union-find, instances sorted by root-relative path, clustered label); lines count once. Overlap without identity stays separate, so a class is never wider than what the detector called identical.
• Registry cluster lines. <canonical> -> <member>... names a root-relative canonical copy and the plugin paths or gitignore-style globs that carry it. registry-filter.py matches root-relative (a run from a subdirectory matches the same lines), first matching line wins; the drift checker and replica-collapse.py skip such lines. Five annotated cluster lines are added for the sync-declared root lib/ canonicals (ADR 0033).
• Report. summary.by_lane and summary.by_directory (additive v1 fields, cumulative ancestors of each class's first instance, computed after exclusion); duplication rows listed largest first; a ## Rollup section to duplication.rollup_depth (default 2); Files with clones: N.; an empty exclusion list is stated with its reason; one headline with the install hint when no clone detector ran, and the skill offers the install without performing it; run rows gain an additive hint field.
• Docs. SKILL.md, README, CHANGELOG [0.2.4], reference/config.md, reference/report-schema.md, setup template and bundled defaults; the claude-config audit-pass exclusion-set reference describes both registry line shapes (claude-config 0.42.4, wording only).

Two plan corrections came from live runs and are recorded in the plan: jscpd emits a star for N copies (each later copy paired with the first, same range every time), so three full copies at different offsets are one class and the "offset" fixture is a partial third copy; and the surviving root lib/ groups after exclusion are all lib/*.test.sh per-suite harness boilerplate that no sync script declares, so they are a finding, not sanctioned replication.

Verification

• scripts/affected-tests.sh --run over the branch: every shell suite green except plugins/claude-ops/skills/plugins/scripts/cache-content-check.test.sh, whose process-budget PS4 trace probe fails identically on an untouched tree in this sandbox (the pid-stamped PS4 did not reach the traced shell); the NOT RUN Python and Node suites pass in their own lanes (932 pytest cases across the twenty selected suites; the autonomy fixtures .mjs, 544 checks) except plugins/session-flow/scripts/tests/test_save_point.py::test_new_origin_falls_back_to_directory_name, which fails identically on a detached origin/main worktree and touches no file this branch changes.
• code-metrics suites on the merged tree: dispatch.test.sh 79 cases, audit-duplication.test.sh 27 cases, test_report.py 42, test_registry_filter.py 20, test_cluster_clones.py 13, test_jscpd.py 15, test_replica_collapse.py 8, test_pathglob.py with the shared root_relative cases; check-cross-plugin-source-drift.sh --check exit 0 and its test 15 cases; check-code-metrics-config-reference.py 28 keys; check-changelog-parity.sh --check-bump origin/main, --check, and --check-order exit 0; check-changed-skills.sh origin/main and the catalog check pass.
• Lint: pinned ruff check and format, shellcheck, shfmt, markdownlint over every changed markdown file, the em-dash gate, and the ai-slop detector over the instruction surfaces, all clean. CI's lint lane on an earlier head found three hygiene defects (exec bits on two new scripts, final newlines on three regenerated captures, the skill description over 1024 codepoints), fixed in d91a5971; ci-lanes reported all lanes passed on that head.
• Live runs with jscpd 5.2.0 and 4.3.0 on this tree (--all --registry scripts/cross-plugin-source-registry.txt): before, 919 pair rows and 75267 duplicated lines with the miro bundle silently skipped; after, 722 classes and 11364 lines under 5.2.0, each of the five cluster lines excluding its class (18, 7, 4, 2, 2 instances), by_lane and by_directory identical from the repository root and from lib/, and the partial typescript row naming plugins/miro/dist/index.min.js when the default **/dist/** exclusion is lifted.
• Review and outcome verification: the section below the plan.

Related

• Refs code-metrics: fix six audit-duplication gaps a whole-tree run exposed #4068 for the interview and plan discussion.
• docs/adr/0033-declare-a-canonical-plus-copies-cluster-as-one-arrow-line.md records the registry grammar.
• Plugin CHANGELOG [0.2.4] carries the user-facing entries; main shipped [0.2.2] (fix(code-metrics): ground the principles skill in its primaries and add a skill-quality Next-section check #4098) and [0.2.3] (fix(playbooks,skills): keep first and second person out of description prose across 36 skills #4108) while this PR was open, and the branch is merged with both.

Approved plan, Brief and dogfood (the full PLAN.md with the five phases is in the branch history at `beb6169d:docs/topics/code-metrics-duplication-audit/PLAN.md`; the contract slice is pruned at close-out)

Brief

Scope-change note (2026-09-11, before approval, from the fresh-context plan review and the
stress-test, each finding verified against the tree): four criteria below were corrected. The
hook-utils.sh class has eighteen instances (root plus seventeen plugins), not seventeen. The
shipped 1mb cap skips one tracked file here, plugins/miro/dist/index.min.js (1.45mb), so the
typescript lane reads partial by design. The hook-telemetry-sink.sh pair differs at line 51
and has no sync script, so it is not sanctioned replication and its registry line is dropped
(five lines, not six). Pair-to-class merging joins byte-aligned copies; a copy embedded at a
different offset with different surrounding lines stays its own group. Cluster lines carry an
explicit -> marker because a registered path may contain a space.

TLDR

• audit-duplication reports skipped files instead of hiding them: the jscpd adapter pre-filters
  by byte size and line count, passes both caps explicitly on 4.x and 5.x, and marks the lane
  partial when anything was skipped; defaults are no line cap and a 1mb byte cap.
• jscpd's pair reports are merged into clone classes before summarizing, so byte-aligned copies
  count once.
• The sanctioned-replication registry gains cluster lines
  (<canonical path> -> <copy path or glob>...) so a canonical file outside any plugin can declare
  its copies; this repo's drift checker skips marked lines and five such lines are added here.
• The markdown report sorts clone rows by duplicated lines, adds per-lane and per-directory rollups
  (cumulative, counts beside share, computed after exclusion, listed to depth 2), and the JSON gains
  summary.by_lane / summary.by_directory as additive v1 fields with an explicit ignore-unknown
  rule in the schema reference.
• The duplication summary line drops "Functions" and "Over reference"; a run with no detector prints
  one consolidated install headline and the skill offers, never performs, the install. Version 0.1.9.

Goal

A whole-tree or change-scoped duplication audit on any repository, this one included, produces
numbers a reader can act on without re-aggregating: no file is silently dropped by a size cap,
byte-aligned copies of a fragment form one group counted once, replication the repository declares
about itself (including copies of a root-level canonical file) is excluded and shown as an
exclusion, and the report says where the surviving duplication sits by lane and by directory. On
this repository the audit reads clean apart from genuine duplication.

Constraints

• The plugin never installs, downloads, or npx-fetches a detector; SKILL.md may instruct Claude to
  offer the install command and run it only on the user's confirmation.
• The report emits no finding, severity, or exit-code gate; duplication has no reference value.
• The code-metrics/v1 schema string is unchanged; every JSON change is additive (new optional
  fields), and reference/report-schema.md states that readers ignore unknown keys.
• Single-token registry lines keep their exact meaning, including a path that contains a space;
  the drift checker (scripts/check-cross-plugin-source-drift.sh) keeps its behavior for them and
  skips only lines carrying the -> marker.
• Both jscpd 4.x (latest-4 = 4.3.0) and 5.x (5.2.0) stay supported by the adapter; the adapter
  always passes explicit --max-lines and --max-size on both majors, treats a configured 0 as
  null (jscpd reads 0 as "default" for 4.x lines and "skip all" everywhere else), and never
  emits a 0 cap.
• An unmeasured value is null, never 0; a run that measured nothing keeps "Measured nothing".
• Validate with scripts/affected-tests.sh --run; every changed file maps to at least one suite.
• No lib/hook-utils.sh or other cross-plugin synced source is edited; every cluster line mirrors
  the src= and copy list its scripts/sync-*.sh already declares.
• One issue, one draft PR whose body opens with Closes #<issue> and carries the four required
  sections; CHANGELOG entry under [0.1.9].

Acceptance criteria

• Running audit-duplication.sh --json --all on this repository with jscpd 4.3.0 and again with
  5.2.0 yields, for every byte-identical whole-file class, the same set of instance files per
  group; tokens, instance order, and the exact line range are excluded from the comparison
  because the two majors tokenize differently and hub on different copies (5.x drops a leading
  comment block from the clone, so a file that opens with one is reported from its first code line
  under 5.x and from line 1 under 4.x; the verifier found one such class,
  resolve-hook.mjs in two knowledge skills). The bash row reads ok under both; with the
  repository's default **/dist/** scope exclusion lifted, the typescript row reads partial
  under both, naming plugins/miro/dist/index.min.js as the one file over the 1mb cap; with the
  exclusion in force (this repository's shipped team file) the bundle never reaches the cap and the
  row reads ok.
• duplication.max_lines defaults to null (no cap) and duplication.max_size to 1mb; both are
  documented in reference/config.md (gated against config-defaults.json) and exported to the
  adapter, and the adapter's tests cover the explicit-cap argv on both majors, the 0-means-null
  rule, and that no 0 cap is ever passed.
• IF every file in a lane is skipped by the caps, THEN that lane's run row reads partial with the
  reason, no collector is invoked for it, and the script never exits 3 for that cause.
• The eighteen byte-identical copies of hook-utils.sh (root lib/ plus the seventeen plugin
  copies scripts/sync-hook-utils.sh --print-manifest lists) produce exactly one clone group with
  eighteen instances, and with the registry line
  lib/hook-utils.sh -> plugins/*/hooks/hook-utils.sh that group appears once under excluded[]
  with duplicated_lines counted once.
• Two groups whose instances overlap without identical line ranges (the hook-telemetry-sink.sh
  shape, and a third copy that carries only part of a fragment two full copies share, so the
  detector names the first copy with two different ranges) stay separate groups after the merge.
  Three full copies at different line offsets are one class: jscpd pairs each later copy with the
  first and names that first copy with the same range in every pair (verified against 5.2.0).
• A cluster line excludes a group only when every instance's root-relative path matches the
  canonical path or one of the members (literal or glob) and the instances' directories are all
  distinct; two copies inside one directory still count as duplication; a single-token line behaves
  exactly as before, a registered path containing a space included; when a single-token line and a
  cluster line both match, the first matching line in file order wins.
• scripts/check-cross-plugin-source-drift.sh --check passes on this repository with the five new
  cluster lines present, each under its own annotation block, and its tests cover a marked line
  being skipped.
• After this change, audit-duplication.sh --json --registry scripts/cross-plugin-source-registry.txt --all
  on this repository reports zero surviving groups whose instances include a root lib/ file that
  a sync script declares as its canonical copy (scripts/sync-*.sh --print-manifest src), from
  the repository root and from a subdirectory alike. Groups over the per-suite test-harness
  boilerplate that lib/*.test.sh files share with plugin test files survive: no sync script
  declares them and the shell-test-helpers convention keeps those helpers per file, so they are
  the audit's finding, not sanctioned replication.
• WHILE no registry is configured and none is passed, the report's excluded[] is empty and the
  summary states that no registry was configured.
• The markdown Measures table for a duplication document lists clone groups in descending order of
  duplicated lines, and the report carries a ## Rollup section with a per-lane table and a
  per-directory table (rows to depth 2 by default, duplication.rollup_depth configurable) whose
  numbers are cumulative up the tree and carry groups and duplicated_lines; the per-lane
  values sum to summary.duplicated_lines, and the root row of by_directory equals it.
• The JSON summary carries by_lane and by_directory with the same numbers (empty maps when a
  duplication collector ran and found nothing); schema is still code-metrics/v1;
  reference/report-schema.md documents both fields, the run row's additive hint field, and
  states that readers ignore unknown keys; verification:measure, the one marketplace consumer,
  reads only status and is unaffected.
• The duplication summary line reads Files with clones: N. followed by the duplicated-lines and
  exclusion lines, with no "Functions" or "Over reference" text; the sibling skills' summary lines are
  byte-identical to today's.
• When no duplication collector resolves, the markdown opens with one headline naming the install
  command and /code-metrics:setup, taken from the run row's hint field rather than parsed out of
  its reason, and SKILL.md instructs Claude to offer the install and run it only on confirmation.
• The jscpd adapter's docstring and reference/collectors.md state that 4.x and 5.x are both
  translated, pin 5.2.0, note the kind field, and record that the two majors tokenize differently;
  the schema reference names "intentional clones" beside "sanctioned replication".
• plugin.json carries the next patch version above the one on the default branch at merge time
  (0.2.4 over the 0.2.3 main shipped while the PR was open; the Brief was drafted against 0.1.8) and
  CHANGELOG.md carries the matching entry covering every item above; scripts/affected-tests.sh --run exits 0, or exits 3 with only Python and Node suites listed as not run, each of which then
  passes in its own lane, or exits 1 only for a suite that fails identically on a detached
  origin/main worktree and touches no file this change edits (named in the PR body).

Captured assumptions

• Version bumps to 0.1.9, not 0.2.0, because this plugin's changelog bumps patch for features
  (0.1.7 was a feat). Revisit if the marketplace's release convention says a new registry grammar
  or JSON fields require a minor bump.
• Once-per-class counting follows PMD CPD and SonarQube; the fetched literature is silent on how
  duplicated lines are totalled, and jscpd v5 is the one tool that sums per pair. Revisit if a
  standard sets a duplicated-lines definition.
• The merge keys on identical (file, start_line, end_line) instances and equal lines, so only
  byte-aligned copies join; jscpd 5.x hubs every later copy on the first and names it with the same
  range, so full copies at different offsets are one class, while 4.x hubs on the last input and
  extends into shared flanking lines, so under 4.x the same copies can stay two groups (a live
  probe of three offset copies gave one class under 5.2.0 and two groups under 4.3.0). Closure is exact only for
  type-1/type-2 clones, which is all the adapter receives because it passes no --max-gap-lines.
  Revisit if similar clones are ever enabled.
• Merged instances are sorted by root-relative path (cluster-clones.py --root) so the first
  instance, and therefore by_directory attribution, is the same under 4.x (which hubs on the last
  input) and 5.x (which hubs on the first), and the same whichever directory the run started from.
• rollup_depth default 2 is the plugin's choice; no upstream sets a depth (SonarQube and Codacy
  roll up every directory). Revisit if a consuming repository's layout makes depth 2 meaningless.
• The byte cap of 1mb aligns with jscpd 5.0.7's parser guard and SonarJS's 1000kb generated-code
  rule, and means 1,048,576 bytes, the value jscpd 5.2.0 reports for 1mb; no line cap by default
  aligns with jscpd 5, PMD CPD, SonarQube, and Linguist. Revisit if jscpd changes its default or
  multiplier in a later major.
• The cluster lines live in the existing registry file because the plugin's own documentation names
  that file as the shape; this is a repository-convention choice with no external authority.
  Revisit if the drift checker grows a second consumer of the file.
• The -> marker is the cluster-line signal because a registered single-token path may contain a
  space and the drift checker's tests protect that case. Revisit if a consuming repository has a
  path containing ->.
• The .claude/hooks/hook-telemetry-sink.sh and plugins/claude-ops/hooks/hook-telemetry-sink.sh
  pair differs at line 51 and has no sync script, so the audit keeps reporting its overlap as
  duplication; a future sync script earns it a cluster line. Revisit when that script exists.
• "Sanctioned replication" stays the plugin's term; the literature's term "intentional clones"
  (Cordy 2008) is named beside it. No upstream tool models the canonical-plus-copies relation, so
  no vocabulary conflict exists.
• The upstream doc drift found in jscpd (docs/rust.md "no limit" for --max-size, "per block"
  help text for --max-lines, stale jscpd.dev v5 defaults) is reported separately, not here.

Out-of-scope

• Installing a detector on the user's behalf, or an npx fallback.
• A pass/fail gate, threshold, or severity for duplication.
• Cross-language clone detection.
• Changing dupl or cpd adapters beyond passing their rows through the new merge and rollup
  steps unchanged.
• A second registry file or a manifest generated from the sync scripts.
• Migrating the registry's single-token lines to the new grammar.
• A repository config excluding plugins/miro/dist/ from this repo's own audits; the partial
  row is the designed reading and a config is the consuming repo's choice.

Deferred questions

• None. Every question registered in the interview was answered; no row was deferred or blocked.

Dogfood

Detector	Status	Surviving classes	Duplicated lines	Files with clones	Excluded	Partial lanes
jscpd 5.2.0	partial	742	11831	487	14 (the five cluster lines: 18, 7, 4, 2, 2 instances)	typescript (1 of 295 files over 1mb: plugins/miro/dist/index.min.js)
jscpd 4.3.0	partial	625	10926	429	14	typescript (same file)

Before this change the same 5.2.0 run reported 919 pair rows and 75267 duplicated lines with no
skipped file named. After the merge with the 0.2.1 line of the plugin (default scope.exclude
globs, the other lane, scope.registries in this repository's team file) the 5.2.0 run reads
complete with 722 classes, 11364 duplicated lines, 492 files with clones, and 16 exclusions: the
miro bundle now falls under the default **/dist/** exclusion before the cap is reached, so no
lane is partial; pass --all on a tree without that exclusion, or set scope.exclude: [], to
see the cap and the partial row. The two majors tokenize differently, so their counts are not comparable with
each other. The largest surviving classes are per-suite test-harness boilerplate (pass/fail
helpers shared by lib/*.test.sh and plugin test files), which no sync script declares and the
shell-test-helpers convention keeps per file: a finding for the operator, not sanctioned
replication.

Review and outcome verification

Outcome verification (/verification:confirm, a fresh-context verifier auditing the acceptance criteria against the diff with live jscpd 4.3.0 and 5.2.0 runs): thirteen of fifteen criteria passed on direct evidence. The two failures were criteria whose wording the tree contradicted, not the code: the plugin version (the Brief named 0.1.9; main was already at 0.2.1 and shipped 0.2.2 and 0.2.3 while this PR was open, so the change ships as 0.2.4), the typescript partial row being observable only with the repository's own default **/dist/** exclusion lifted, one byte-identical class that jscpd 5.x names from its first code line where 4.x names it from line 1, and the two pre-existing suite failures listed above. The criteria are amended to those facts in the plan. One observation outside the criteria was a real defect and is fixed on the branch: the class merge sorted instances by cwd-relative path, so summary.by_directory attributed a class differently from lib/ than from the root; the merge now takes --root, sorts root-relative, and the rollups are identical from both.

Code review (/review:quality-gate code, fresh-context reviewer over the merge-base diff): no CRITICAL finding. One IMPORTANT finding, fixed on the branch: with no line cap configured the jscpd adapter passed --max-lines 1000000, a number a generated file can reach, so such a file would have been skipped by jscpd's own gate with nothing named; the sentinel is now the signed 32-bit maximum, which both majors accept. Of two SUGGESTION items, one was taken (the root-relative path helper now lives in pathglob.py and is shared by the summarizer, the registry filter, and the class merge) and one left as it is (the skill script's integer reader falls back silently on a float or quoted number for rollup_depth).

🤖 Generated with Claude Code

https://claude.ai/code/session_01QShQwS5yDYYToBmofdEY5N