Skip to content

feat(clones): git-ast clones — structural clone detection (identity index, first slice) - #45

Merged
bdelanghe merged 3 commits into
mainfrom
feat/identity-index-clones
Jun 29, 2026
Merged

feat(clones): git-ast clones — structural clone detection (identity index, first slice)#45
bdelanghe merged 3 commits into
mainfrom
feat/identity-index-clones

Conversation

@bdelanghe

Copy link
Copy Markdown
Collaborator

Summary

  • Adds src/clones.rs with find_clones() — groups Defs by shape_hash across files, surfaces groups with >1 occurrence as structural clones
  • Adds git-ast clones FILE... verb wired into main.rs with extension-based dispatch (mirrors inspect)
  • First concrete step of prx-nf1f (identity index): the subtree_hashes / shape_hash computed per-file are now accumulated rather than discarded

Why this is the right first slice:
The shape_hash is already an exact equivalence-class key by construction — equal hash ⟹ same canonical body. No threshold, no heuristics. The identity-index research note calls the occurrence map "trivial"; this PR makes it real, in memory, over a provided file set. The fuzzy/near-clone layer (MinHash + LSH over subtree_hashes) is explicitly deferred.

Example output:

$ git-ast clones src/a.rs src/b.rs
d5e0ebfa8c0dfcfc  3×fn
  src/a.rs  helper
  src/a.rs  aux
  src/b.rs  helper

Groups are sorted by shape_hash (BTreeMap), occurrences in file-then-definition order.

Test plan

  • 11 new unit tests: empty input, single file, cross-file, exact-clone, 3-way group, multiple disjoint groups, HTML kinds preserved, sorted output, real Rust round-trip via printer::inspect
  • 108 total unit tests pass (was 97 before this PR; includes the 11 new + upgrade brings 0 regressions)
  • 5 cucumber integration tests pass
  • Smoke-tested against repo's own source (no false positives) and a synthetic known-clone pair (correct 3-occurrence group)

PR checklist

  1. Independent PR — new module + verb only; no changes to existing paths
  2. Changed codepaths verified — 11 targeted unit tests + smoke test
  3. Root cause — N/A (new feature)
  4. No duplicationfind_clones is pure post-processing; inspect dispatch is one pattern
  5. No unrelated changes — only src/clones.rs, src/lib.rs, src/main.rs

Depends on #44 (tree-sitter upgrade) landing first.

🤖 Generated with Claude Code

bdelanghe and others added 3 commits June 29, 2026 13:03
tree-sitter  0.22 → 0.26.10
tree-sitter-rust  0.21 → 0.24.2
tree-sitter-html  0.20 → 0.23.2

Both grammar call sites updated: language() fn → LANGUAGE.into(), which
is the canonical API since the 0.23 grammar series. All 97 unit tests
and 5 cucumber tests pass unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ntity index

First concrete step of prx-nf1f (identity-index). The subtree_hashes and
shape_hash produced by inspect are computed per-file and then discarded.
This commit accumulates them across a file set and surfaces equivalence
classes — definitions with identical bodies under the canonical form,
regardless of declared name.

Design:
- src/clones.rs: find_clones(&[(&str, &[Def])]) → Vec<CloneGroup>, keyed
  by shape_hash (BTreeMap for deterministic output). Groups with >1
  occurrence are structural clones; single occurrences are filtered out.
- Exact by construction: equal shape_hash ⟹ same canonical body. No
  threshold, no heuristics — that layer (MinHash+LSH over subtree_hashes)
  is explicitly deferred to the research note.
- Routes by file extension (html/htm → html::inspect, else printer::inspect)
  matching the existing inspect/match/blame dispatch.

CLI: git-ast clones FILE...
Output: shape_hash  N×kind\n  file  name\n per occurrence, one block per group.

11 new unit tests (108 total), all passing. Smoke-tested against the repo's
own source (no false clones) and against two files with a known shared body
(correctly groups 3 occurrences under one shape hash).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@bdelanghe
bdelanghe force-pushed the feat/identity-index-clones branch from e4fa22d to 15d5784 Compare June 29, 2026 17:12
@bdelanghe
bdelanghe merged commit 2e2f870 into main Jun 29, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant