Skip to content

Introduce overload-safe semantic identities#95

Merged
samchon merged 9 commits into
masterfrom
feat/semantic-identity-v2
Jul 20, 2026
Merged

Introduce overload-safe semantic identities#95
samchon merged 9 commits into
masterfrom
feat/semantic-identity-v2

Conversation

@samchon

@samchon samchon commented Jul 18, 2026

Copy link
Copy Markdown
Owner

Work in progress, opened for visibility. The identity contract is implemented and the deterministic suite is green on all three operating systems, but two things are not done — they are spelled out in "What is not working" below, with the exact cause of each. Full handoff context is in #119.

Closes #65.

Summary

The position-invariant id contract path#qualifiedName:kind is not a unique semantic identity across languages: it collides for non-TypeScript overloads, multiple constructors, generic arities, partial declarations, extension methods, and same-named locals, so generic indexing silently deduplicates distinct declarations and strict providers reject the second valid node.

This introduces an intrinsic semantic identity for non-TypeScript declarations while keeping TypeScript ids byte-identical for ttsc parity — ttsc never emits overloads or constructors, so no TypeScript id changes and the pinned application contract and its prose grammar stay literally true.

Design

  • semanticIdentity — a versioned identity record (language, native symbol, declaration role, module/target/translation-unit scope, and an overload discriminator folded into a sha256 digest) derived deterministically into an opaque id: @v2/<language>/<digest>#<name>:<role> when the provider can promise cross-edit stability, or @g2/… when only generation-scoped stability is honest. Legacy-id equivalents are retained for backward-compatible handle resolution.
  • assignSemanticIdentities — promotes non-TypeScript nodes to intrinsic ids before dedupe and remaps edge endpoints. Undecorated callables, locals, and anonymous/generated declarations are marked generation-scoped rather than pretending their source coordinate is stable.
  • Identity-aware dedupe/merge — semantic duplicates merge their declaration/implementation locations; a semantic-id collision is a producer defect; a legacy collision the generic path cannot disambiguate keeps the last declaration, while the strict provider boundary still rejects it.
  • Resolution & serializationSamchonGraphMemory indexes exact and legacy ids; resolveGraphHandle resolves an old undiscriminated handle to deterministic ranked candidates; wireEdges, reduce, and the benchmark viewer treat an opaque semantic id verbatim instead of splitting it as a path.

Because the id no longer carries the file for non-TypeScript nodes, anything that took the text before # now consults node.file. The #qualifiedName:kind suffix is preserved, so id.endsWith("#App.dispatch:method") still matches both legacy and semantic ids.

What is not working

1. Multi-location declarations crash the build (blocker). The swift, scala and ruby LSP experiment jobs fail with:

Error: @samchon/graph: semantic node has more than the declaration/implementation
location policy can preserve: @v2/swift/a06e3866…#Argument:namespace

mergeDuplicate in indexer/dedupeNodes.ts preserves only evidence + implementation and throws past that. A Swift type extended across several files, a reopened Ruby class, or a Scala companion legitimately has three or more declaration locations, so the throw kills the whole graph build. Issue #65's own Approach point 3 allows either "one semantic node with multiple provenance locations" or an explicit declaration/implementation policy — so this must become a deterministic, order-independent policy (preferably an ordered provenance list) rather than a crash, and it must not silently drop a location.

2. Coverage is 99.88% lines / 99.62% branches, not 100%. The Test step passes on all three operating systems; only the Coverage gate is red. Remaining gaps, each root-caused:

File Uncovered Missing input
indexer/dedupeNodes.ts 71-74 the fact-disagreement throw — two semantic nodes with the same id but a different name/kind
indexer/dedupeNodes.ts 82-85 the >2-locations throw — superseded by the policy fix above
SamchonGraphMemory.ts 205-208 fileOfEdgeSource is only called as evidence.file ?? …, so the edge needs no evidence.file and a semantic from absent from the dump
indexer/assignSemanticIdentities.ts 57 a parameter node whose qualifiedName has no (
indexer/assignSemanticIdentities.ts 159 the > ? 1 arm of the ambiguous-endpoint sort tiebreak
provider/semanticIdentity.ts 158-159 validateIdentity empty language/symbol/role throw
provider/semanticIdentity.ts 162-163 validateIdentity native.key === "" throw
provider/mergeGraphSlices.ts 124-125 edgeOrderKey's ?? "" / ?? 0 fallbacks — a strict edge with no evidence
provider/mergeGraphSlices.ts 131 the > ? 1 comparator arm
indexer/overrideEdges.ts 91 push's else arm — two members on the same semanticMemberKey group key

The three compareText helpers are left < right ? -1 : left > right ? 1 : 0; a sort over distinct keys can never return 0, so that arm is genuinely unreachable and wants a /* c8 ignore */ with a proof comment rather than a contrived test. Extend tests/test-graph/src/features/test_semantic_identity_edge_cases_stay_covered.ts, which already covers the rest of this surface and demonstrates the white-box idiom.

3. Deliberately deferred. Provider-native keys (Clang USRs, Roslyn ISymbol, JVM descriptors, Swift USRs) are not populated — the current discriminators come from lexical/static signatures. Those arrive with the language providers (#71#84), which is what this contract exists to unblock.

What is verified

  • pnpm --filter @samchon/graph build and @samchon/graph-test build — no errors
  • Parity gate green: test_application_contract_reproduces_the_pinned_ttsc_reference, …fails_closed_on_drift, test_readme_embeds_the_exact_application_contract — TypeScript ids and the ttsc contract are unchanged
  • The five semantic-identity regressions (distinguish provider symbols, keep overload ownership explicit, legacy handles remain resolvable, round-trip preserves references, edge cases stay covered)
  • Broad local suite: static extractors for every language, dedupe/merge, real-codebase static dump, provider (fake server), resolver, resident — all Success
  • Test step green on ubuntu, macOS and Windows
  • Coverage step at 100% — see "What is not working" §2
  • swift / scala / ruby LSP experiments — see "What is not working" §1

This is the P0 dependency of provider authority (#66), strict snapshot validation, and every language provider (#71#84).

🤖 Generated with Claude Code

@samchon

samchon commented Jul 19, 2026

Copy link
Copy Markdown
Owner Author

The local implementation is not mergeable yet, and the green GitHub checks cover only the empty reservation commit.

Reproduced blockers:

  • ambiguous overload endpoints are assigned to the lexicographically first semantic hash;
  • same-named non-callable locals are hashed before closure/final ownership and merge;
  • TypeScript overload sets still collide because TypeScript is skipped by semantic assignment;
  • valid symbols with 3+ partial locations are rejected, while generic identities include document scope and cannot merge cross-file partials;
  • prefix-only detection reserves valid legacy @v2/** and @g2/** paths;
  • display spelling participates in the final ID outside the identity digest;
  • variable-to-property refinement occurs after semantic validation, leaving kind/ID/alias disagreement;
  • semantic duplicate nodes/edges are normalized away before strict-provider validation;
  • opaque IDs remove the file coordinate assumed by public IDs and compressed reached/tour payloads.

Acceptance must preserve canonical coordinate-bearing application structure, finalize ownership/kind before identity, bind overload endpoints only with signature/native proof, support arbitrary valid partial locations, reject exact duplicate provider facts, and add order/line-insertion/LSP/static/resident/dump regressions. #96/#97 lands first.

@samchon

samchon commented Jul 20, 2026

Copy link
Copy Markdown
Owner Author

Handing this off. Full context for a fresh agent — the design, the CI state, the blocking >2-location bug, and every remaining coverage gap root-caused with the exact input to write — is in #119.

Current state: Test green on all 3 OSes; CI red on (1) the swift/scala/ruby experiments, which throw semantic node has more than the declaration/implementation location policy can preserve on real multi-location declarations, and (2) coverage at 99.88% lines / 99.62% branches. Both are fully diagnosed in #119 §3a and §3b.

@samchon
samchon marked this pull request as draft July 20, 2026 08:14
@samchon
samchon marked this pull request as ready for review July 20, 2026 09:19
samchon and others added 9 commits July 20, 2026 23:43
Replace the position-invariant `path#qualifiedName:kind` id — which collides for non-TypeScript overloads, constructors, generic arities, and same-named locals — with an intrinsic semantic identity for non-TypeScript declarations, while keeping TypeScript ids byte-identical for ttsc parity.

- `semanticIdentity`: a versioned identity record (language, native symbol, role, scope, overload discriminator folded into a sha256 digest) derived into an opaque `@v2/<lang>/<digest>#<name>:<role>` id (or `@g2/…` when only generation-scoped stability is honest), plus legacy-id equivalents for backward-compatible resolution.
- `assignSemanticIdentities`: promotes non-TypeScript nodes to intrinsic ids before dedupe and remaps edge endpoints; undecorated callables, locals, and anonymous declarations are marked generation-scoped rather than claiming source-coordinate stability.
- dedupe/merge become identity-aware: semantic duplicates merge their declaration/implementation locations, a semantic-id collision is a producer defect, and a legacy collision the generic path cannot disambiguate keeps the last declaration (the strict provider boundary still rejects it).
- `SamchonGraphMemory` indexes both exact and legacy ids; `resolveGraphHandle` resolves an old undiscriminated handle to ranked candidates; `wireEdges`/`reduce`/the benchmark viewer treat an opaque semantic id verbatim instead of splitting it as a path.

TypeScript ids, the pinned ttsc application contract, and its prose grammar are unchanged.

Closes #65

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G2eNarjGmcWmEjFTW2Qcz
A non-TypeScript node now carries a provider-native `@v2/…`/`@g2/…` id whose file and scope live on the node, not in the id string. Match the enclosing symbol by the preserved `#qualifiedName:kind` suffix (or resolve the node for its file) instead of an exact legacy `path#name:kind` string, and register the two cohesive identity modules (semanticIdentity, dedupeNodes) in the export-convention allowlist alongside the existing reduce/parseGraphArgs/view entries.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G2eNarjGmcWmEjFTW2Qcz
Add a white-box regression that exercises the collision guards, opaque-id file recovery, strict-slice normalization, and ambiguous-endpoint tiebreaks that no ordinary fixture reaches: fileOfNodeId on a bare vs symbol id, the validateIdentity refusals, empty-file and evidence-less generation scopes, the two-location semantic merge, mergeGraphSlices strict normalization, wireEdges over present/legacy/opaque sources, and the duplicate-id and absent-source dump guards.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G2eNarjGmcWmEjFTW2Qcz
@samchon
samchon force-pushed the feat/semantic-identity-v2 branch from e49c55f to 36131ba Compare July 20, 2026 14:46
@samchon
samchon merged commit 05242fd into master Jul 20, 2026
18 checks passed
@samchon
samchon deleted the feat/semantic-identity-v2 branch July 20, 2026 14:47
samchon added a commit that referenced this pull request Jul 21, 2026
Round 13. A `local N` symbol is an index-local counter scoped to its document,
and the identity derived from it says so — but the definition map was keyed by
the symbol string alone. Two files each holding a `local 4` therefore collided:
the second overwrote the first, and every reference in the earlier file then
resolved to a declaration in the later one, silently and with full confidence.

Definitions are now filed under a document-scoped key when the symbol is
generation-stable, and lookups from inside a document use the same key, so a
local reference can only ever reach the local in its own file. Global symbols
are unchanged — they mean the same declaration wherever named, which is what
makes them global.

A collision check on the derived id remains as a backstop: two documents can
still digest one identity, and publishing it twice would have mergeGraphSlices
reject the whole slice.

Refs #69 #95

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
samchon added a commit that referenced this pull request Jul 22, 2026
* chore: claim the bulk-index provider foundation cycle

Claims #66, #67, #68, #69, and #102 as one integrated campaign cycle before
any implementation lands, so no overlapping contributor work starts against
the same surface.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(provider): make strict provider discovery a registry

Replaces the `language === "typescript"` arm inside buildLspGraph's language
loop with a data-driven provider registry. A provider now states its own terms
— which languages it owns, what its facts are grounded in, which edge families
it can prove, why it cannot serve a build, where its executable is, and what
the project needs prepared — and the indexer only asks and reports.

- IGraphProvider / GRAPH_PROVIDERS / selectGraphProviders, with one-owner-per-
  language validated statically over the whole registry rather than over the
  providers that happened to resolve on this machine.
- IBulkGraphSession owns a language set, so one provider can replace C and C++
  atomically instead of publishing two slices that were never simultaneous.
- Snapshot provenance carries provider, authority, and provable fact families;
  assertGraphSnapshotContract holds every published slice to what the registry
  declared instead of trusting the payload.
- ISamchonGraphDump.provenance publishes one row per contributing provider,
  with manifest and content digests recomputed by the publisher. Computation
  mode stays out of the dump and rides on IIndexerResult, so a dump remains a
  pure function of its source.
- Resident refresh deduplicates multi-language sessions by identity, adds a
  commit fence that rejects a candidate whose consumed source moved while it
  was preparing, and retries a bounded three times rather than publishing two
  checkouts' halves.
- Strict SCIP ingestion: wire contract, strict parser, symbol parser, and an
  adapter that maps only what SCIP proves and refuses to infer calls,
  constructions, imports, or decorators from source punctuation.

Not yet verified: per the user's instruction this branch is validated by CI
rather than locally.

Refs #66 #67 #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci(release): gate publication on a verified two-package release

The v0.2.0 release built and published in two steps and checked nothing in
between, so the first thing that could notice a bad release was an install in
somebody else's project. npm still serves graph 0.1.0, graph-sitter never
reached the registry at all, and GitHub has no release.

- Both manifests declare publishConfig.access "public". A scoped package
  published without it is rejected as unauthorized, and saying it in the
  manifest means every publisher gets the same answer.
- release-verify proves the tag is an immutable vX.Y.Z, that every publishable
  manifest carries that version, and that the tagged commit was merged into the
  release branch. Recovery from a failed release is a new version, never a
  moved tag.
- The verify job runs frozen install, build, test, and coverage before anything
  is published, then packs both tarballs and proves each contains what its
  `files` advertises.
- graph-sitter publishes first and the registry is polled until it actually
  serves it; graph follows. npm accepts a dependent whose dependency does not
  exist, which is precisely how the reported break happened.
- A clean-install smoke test installs from the registry into an empty directory
  and exercises both the import and the CLI entry point — the only step that
  proves the published artifacts resolve each other, since every earlier step
  ran where workspace:* always resolves.
- The GitHub release is created only after both packages install.

Registry authority stays manual: publication runs in the protected `npm`
environment against NPM_TOKEN, and no credential is stored in the repository.

Refs #102

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(provider): add the strict semantic conformance kit

The experiment catalog this replaces proved that a server started and returned
some nodes: eight languages were allowed zero edges and the rest could pass on
one, so a provider that linked every pair of names it saw scored better than
one that resolved carefully and found few.

Conformance is now stated as golden facts, each with a negative twin one
property away — a name that appears only in a comment, a symbol that merely
shares a spelling, a function passed as a value rather than invoked. The twin
is the point: a provider cannot be shown to resolve calls by producing calls.
The kit is proved against a deliberately text-heuristic provider that satisfies
every positive case and fails exactly the twins.

Structural invariants are checked separately because they are not about any
language: dangling endpoints, duplicate ids, zero-based spans, manifest entries
without a checker digest, and edge families the provider never claimed. Slice
determinism is compared through the published digests, which are what the dump
and the resident commit fence actually compare.

Refs #68

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(scip): satisfy the lint contract in the session's process handling

CI reported three errors on one line: a promise executor whose arrow body
returned an assignment, a first parameter not named `resolve`, and `void` as
an expression-position type argument.

Rewriting rather than patching the line, because the shape that caused it was
also wrong in two ways CI had not reached: the exit promise was built inside
the surrounding executor, shadowing its `resolve`, and the child was only
registered as owned after that executor ran. A session that owned children
only once they had answered would leak precisely the ones that hang, which is
the case `close` exists for. The child is now registered before the first
await, and the deferred is its own named helper.

Refs #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(provider): cover the registry, snapshot contract, and SCIP ingestion

Coverage for the new provider surface, written against the behaviour each
piece exists to guarantee rather than against its lines:

- Provider selection: partial language ownership, a multi-language candidate,
  and one attributable sentence for every decline (refusal, absent tool,
  unprepared project). Registry overlap is proved to fail even when only one of
  the two providers could resolve on this machine, since the defect is static.
- Snapshot contract: each way a slice can exceed what its entry declared —
  unowned language, foreign node, unclaimed edge family, misattributed
  provenance, overstated authority, and fact sets that are over, under, or the
  same size but different.
- Digests: manifest order does not change the manifest digest, because two
  providers listing one program differently describe one program; publication
  order does change the content digest, because the dump's contract is
  byte-identity on an unchanged checkout.
- SCIP: every descriptor form including backtick escaping and doubled
  backticks, the local-symbol generation scope, each unreadable symbol shape,
  and every validation refusal. The mapping test asserts what is published and
  then asserts that no `calls` edge exists — the index never proved one.

Refs #66 #68 #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(provider): hold new sources to the one-export-per-file convention

CI's export-convention gate rejected four new files that exported several
symbols each. Rather than register them as exceptions, each now exports one
symbol named for its file and carries its companions in a merged namespace,
which is the shape IBulkGraphSession and ISamchonGraphDump already use:

- selectGraphProviders.ICandidate / .IResult
- adaptScipIndex.EDGE_KINDS / .IProps / .IResult
- scipSymbol.IParsed / .Descriptor / .nodeKind
- adaptTtscGraphDump.PROVIDER / .AUTHORITY / .EDGE_KINDS

Also repairs two real defects the same run exposed:

- The accumulated-session test still injected the removed resolveTtscGraphCommand
  and collectTtscGraph hooks in its generic-fallback case, so that case was
  asserting against a provider path that no longer existed.
- The doubled-backtick fixture was wrong, not the parser. SCIP escapes a literal
  backtick inside a quoted name as two backticks, so the name `tick``tock`
  carries one; the fixture had opened the quote twice and asserted the parser
  should recover a name from it.

Refs #66 #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(indexer): make the registry the seam, not the selection

Self-Review over the pushed diff found the dependency injection point wrong,
and CI's four ttscgraph failures were the symptom rather than the defect.

Injecting selectGraphProviders let a caller replace the whole selection, so
refusal, command resolution, project preparation, and the one-owner-per-
language check ran on no path any test exercised — the integration would have
been proved by a stub standing exactly where the logic under test belongs. The
seam is now the registry: selectGraphProviders always runs, and a test
substitutes one entry's resolve() so the shipped provider's refusal, authority,
fact families, and session construction are all still exercised.

Two warning sentences changed shape as a consequence, and their tests move with
them rather than the reverse: a fallback now names the provider *and* the
authority the build gave up, which is what #66 asks warnings to say and what a
reader needs to know whether the facts they hold came from a compiler or a text
scan. The library-resolver test now asserts that second clause explicitly.

Refs #66

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(provider): validate every strict refresh, not only the first

Two more defects from the same Self-Review round, both about a claim that was
checked once and then trusted forever.

A resident source held no reference to the registry entry behind its sessions,
so assertGraphSnapshotContract ran on the initial build alone. A provider whose
second generation published an unclaimed edge family, or nodes in a language
its slice does not own, was merged straight into the dump — and the second
generation onward is where a long-lived session spends its entire working life.
IIndexerResult now carries the provider behind each kept session, and every
refresh is held to the same contract as the first.

SCIP external symbols were attributed to the session's first language. That is
correct for a single-language indexer and a coin toss for a provider owning C
and C++ together: every dependency leaf in the program would take whichever
language the registry happened to list first. The language a reference appears
in is a fact the index does state, so an external leaf is now created when its
first occurrence names it and inherits that document's language.

Refs #66 #67 #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(parity): review the dump's per-provider provenance against the reference

Two rules deferred their decision to #66; both now describe what #66 actually
did rather than an omission.

The reference's single-Program provenance sits exactly where this product's
list does, so the rule replaces it in place instead of deleting it: one
TypeScript Program resolves the reference's whole dump, while this one can be
owned in part by a compiler, a Clang compilation universe, and a semantic index
at once. It is optional because the generic and static lanes make no such
claim.

Its helper types are replaced rather than dropped. The reference's universe,
root, and digest rows describe one Program's manifest; every language answers
'what decides the file set' with a different shape and a reader only ever asks
whether it moved, so the product publishes a fingerprint per provider. The
producer row keeps tool and version, renames typescript to compiler, and gains
the schema and protocol generations a multi-provider reader needs to tell two
producers apart.

Also completes the registry-seam migration in the two tests that still injected
the removed selectGraphProviders hook, so both now exercise real discovery.

Refs #66

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(scip): make session close idempotent and escalating

Self-Review round 2. The new session repeated exactly the defect #107 fixed in
the ttscgraph client, in a file written after it.

close() emptied the child set and awaited those children, without caching its
result. A second caller found the set already empty and returned *before* the
children had exited — so two callers disagreed about whether the session was
closed, and the one who was wrong is the one who asked second. IBulkGraphSession
requires close to be idempotent; it now caches the shutdown promise, as the
ttscgraph client does.

It also awaited termination unconditionally. An indexer that ignores its first
signal — mid-write, blocked on a lock, wedged in a shutdown hook — would hold
close open forever, and a resident server's shutdown with it. Termination now
escalates to SIGKILL after a short grace period, since nothing here needs a
clean exit: the artifact directory is discarded either way.

Refs #69 #107

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(provider): digest the whole fact, not a chosen few fields

Self-Review round 2. contentOf hashed a node's id, kind, language, name, file,
and span offsets. A slice whose modifiers, export flag, qualified name,
decorators, literals, or enum members changed therefore digested identically to
the one before it — and both the resident commit fence and the dump's published
provenance compare exactly this value to decide whether anything moved. A
digest over part of the facts is not a weaker proof of the whole; it is a proof
of something else.

It is now a canonical serialization of each row. Key order is sorted rather
than preserved, because two structurally identical nodes built by different
code paths describe one declaration and must agree; absent optional properties
and ones explicitly set to undefined collapse together, which is what the graph
means by them.

Also fixes the release preflight's reachability check, which asked whether the
tagged commit was merged into origin/master against a shallow checkout that has
no such ref — a gate that would have refused every release for the same reason
it refuses an unmerged one.

Refs #66 #67 #102

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(parity): review the provenance JSDoc in the prose layer

The structure gate passes; only the prose layer remained, which compares the
JSDoc a caller reads rather than the shape alone.

The dump field's own comment is condensed to one paragraph. Its longer form
restated what IBulkGraphSession.IProvenance already documents in full, and a
reviewed prose rule has to carry the whole comment verbatim — so the duplicate
prose was both redundant to a reader and a second place to keep in step.

Refs #66

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(scip): report a position encoding the graph cannot honour

Self-Review round 2. positionEncoding was declared on the document type and
then neither parsed nor consulted.

A SCIP range is an offset in code units, and which code unit is the document's
own declaration. Graph columns follow the LSP convention of UTF-16 code units,
so an indexer counting UTF-8 bytes disagrees on every line holding a non-ASCII
character — silently, and only there, which is the shape of bug that survives a
test suite written in English identifiers.

Spans are display evidence rather than identity, so this reports instead of
rejecting. What it must not do is say nothing. An absent or unspecified
encoding stays quiet: 'did not say' is not 'said something wrong', and warning
on it would mark every well-behaved ASCII project.

Refs #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(scip): cover the session lifecycle and the adapter's refusals

CI's deterministic suite now passes in full; this closes the coverage gaps it
exposed, and they were concentrated in exactly the code Self-Review had just
repaired — so the fixes now have the regressions that lock them.

The session is exercised against a real child process rather than a stub,
because the paths worth proving are the ones a stub would replace: an indexer
that exits non-zero, one that exits cleanly having written nothing, a decoder
that fails or answers with garbage, an index built for another checkout, and a
child that ignores its first termination signal. Each leaves the published
generation exactly where it was. Unchanged is proved by the indexer not running
a second time, not by a counter that did not move.

The adapter's refusals get their negative cases: a relationship carrying two
flags is two claims rather than a tagged union, a relationship or occurrence
naming an undeclared symbol emits nothing, a self-relationship is not an edge,
an unmodelled kind publishes no node, an unreadable symbol is reported rather
than guessed, and a dependency leaf nobody references is never materialized.

Refs #68 #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: close the remaining coverage gaps in the new provider surface

Covers the resident transaction's own paths — a candidate discarded because its
source moved under it, a project that will not hold still reported rather than
papered over, and a later generation refused for publishing outside its
provider's contract. These are the three fixes Self-Review made without
regressions to lock them.

The rest are small refusals: a SCIP descriptor suffix that closes something it
never opened, optional records the graph does not read but still validates, a
package declaring no dependencies at all.

One defensive branch is removed rather than covered. canonical() fell back to
the string "undefined" for a value JSON.stringify cannot render, which was
unreachable for every value that reaches it — the object walk already filters
undefined. An undefined inside an array is the one place it survives, and it
now becomes null there for the reason JSON does the same: an array's length is
part of its meaning, so the hole keeps its place.

Refs #66 #67 #69 #102

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(fixtures): derive a scripted refresh's mode from what it did

Self-Review round 3. The scripted session reported `initial` whenever its
generation was 1, so a session whose snapshot list had run out republished its
last snapshot with changed=false and mode=initial — a combination no real
provider may produce, handed to every test that used the fixture. The mode now
follows the refresh: initial, rebuild, or unchanged.

Also ignores the release tarball directory. The published artifact is the one
the registry serves; these are the copies the preflight opened to check it, and
a local run should not leave them staged.

Refs #68 #102

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(scip): make an indexer a registry entry through one description

#69 had built the ingestion, the validation, and the lifecycle, but nothing
connected them to the registry — the parts existed and no provider could be
made from them.

scipProvider is that connection, and it is a description rather than a class
because the fourteen language entries that will use it differ only in which
executable to run, what to pass it, and which files decide its output.
Everything else is identical by construction, so they cannot drift apart in the
parts that are supposed to be the same.

Every entry inherits the bare index's provable facts. That is the point rather
than a limitation: a SCIP index cannot prove a call, a construction, an import,
or a decorator, and an entry claiming one would be rejected by its own snapshot
contract. A language that can prove more does it through typed enrichment, not
by widening the list. A bounded build is refused for the same reason the
compiler-owned lane refuses one — honouring a cap would mean indexing
everything and then deleting facts.

Refs #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(scip): pick a kind the graph really does not model

The unmodelled-kind case used SCIP's TypeParameter, which this graph does map —
to `type` — so the case asserted the opposite of what it described and CI
caught the extra node. It now names a kind outside the mapping entirely, so the
descriptor fallback is what has to decline it.

Refs #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(indexer): make the commit fence provable

CI could not satisfy the fence test, and the test was not the defect. The fence
guards the moment between a lane reading its source and the refresh publishing
what it read. In production that moment belongs to whoever else is editing the
checkout; in a test it belonged to nobody, because the last read and the fence
are adjacent statements with no seam between them.

A safety net that cannot be made to catch anything is not one. The static parse
that closes the preparation phase is now a dependency, so a test can move the
project inside it — which is the only place a candidate can be overtaken.

Also corrects the unmodelled-kind case, which named SCIP's TypeParameter. This
graph does map that one, to `type`, so the case asserted the opposite of what
it described.

Refs #67 #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(resident): move the project before expecting a refresh

The fence cases loaded twice against an unmoved checkout, so the second load
found nothing stale and returned without ever reaching the refresh — let alone
the static parse the fence fires from. The first build publishes its generation
without touching the static lane, which is exactly why it cannot fire there.

Refs #67

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(readme): describe the provider registry and dump provenance

A public schema change needs its README contract excerpt to move with it. The
document still described a single inline TypeScript special case, which is what
#66 replaced: indexing now asks a registry which languages each strict provider
owns, holds a snapshot to the edge families that provider claims, and names the
authority a fallback gave up rather than letting one read as the strict result.

The new dump field is documented with the thing readers most need from it —
that computation mode is deliberately absent, because it belongs to a refresh
rather than to the facts, and recording it would make two dumps of the same
unedited checkout differ.

Refs #66

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(conformance): hold the harness to what its own doc claims

Self-Review round 4. The comment said declarations and relationships were both
matched by name and kind; only declarations were. Relationships match by name
alone, and that is the right trade — a provider's ids are its own — but it
costs something the harness was not collecting: a fixture reusing one display
name makes every edge expectation naming it ambiguous, and an ambiguous
assertion passes for the wrong reason. The fixture is now held to unique names
instead.

The structural pass also checked only declaration spans. An implementation span
is a second real location encoded the same way, so a provider could emit a
zero-based one wherever a declaration and its body are apart — every overridden
method, every header-and-source pair.

Refs #68

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(resident): stop writing the text the project already had

The busy-project case wrote `value = ${revision}`, whose first revision is
byte-identical to the original source. The next attempt then found the checkout
unchanged and returned quietly — correctly, because a project edited back to a
state it already had *is* unchanged — so the retry bound was never reached and
the case failed for a reason that had nothing to do with the fence.

The discard case had the opposite problem: both its assertions passed just as
well when the fence never fired, which is how it went green while proving
nothing. It now counts preparations and requires the second one.

Refs #67

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(resident): count the generation the drift actually lands on

The build that opens a session does not poll it, so the honest snapshot is what
the second load commits and the drifting one arrives on the third. The case
asked for the refusal a generation too early and got the honest slice instead.

It now asserts the honest generation publishes first, which is what makes the
refusal afterwards mean what it says: a session checked only when it was opened
would never have been asked again.

Refs #66 #67

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(indexer): report a language a provider owns but does not publish

A candidate may legitimately publish fewer languages than it owns — a Clang
provider asked for C and C++ can answer with only the translation units it
found. What it must not do is leave the rest to the generic lane in silence: a
caller who selected a compiler-owned provider for C would then be handed
navigation facts for it with nothing to tell them apart, which is the same
failure the inline TypeScript branch used to produce and the reason #66 exists.

Every unpublished language now gets one sentence naming the provider and the
authority it did not supply.

Refs #66

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: attribute fake snapshots to the entry that publishes them

Both failures were the same fixture error: a snapshot carrying the default
provider name while the registry entry was named something else. The contract
check rejected it — correctly — before either case reached the behaviour it
meant to exercise, so one reported a provider failure instead of an unpublished
language and the other published nothing instead of its first generation.

Refs #66 #67

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: close the last coverage gaps in the provider surface

Coverage reached 99.91%; these are the paths that were left.

Reachable ones get cases: a SCIP bracketed descriptor with nothing inside it,
a descriptor that is only a suffix, a second enclosing scope so reference
attribution actually has to rank them, a scope starting on its owner's line so
containment compares columns rather than stopping at the line number, an
indexer that cannot be spawned, an aborted refresh, a refresh queued across a
close, a two-language session merged once rather than once per key, and an
explicit JSON null in the digest walk.

Two are not reachable. Merging a namespace onto a function compiles to an
"X || (X = {})" initialiser whose falsy arm cannot run, because the function
declaration above it is always evaluated first; and the SIGKILL escalation is
dead on Windows, which terminates on the first signal. Both are ignored with
that reason rather than covered by a test that would only be pretending.

Refs #66 #67 #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci(release): check the publish jobs against the order the manifests require

release-verify computed a publication order and printed it; nothing read it.
The workflow publishes in fixed, named jobs — which two packages do not justify
generating — so that job order is a claim about the dependency graph, and a
claim nobody checks is one that silently stops being true. A third package, or
a dependency reversed between the two, would publish a dependent before its
dependency exactly as v0.2.0 did.

The workflow now declares the order it wires, and the preflight refuses the
release when the manifests disagree with it.

Refs #102

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(scip): stop fabricating the digest the facts were computed from

Self-Review round 8. The session hashed each document off disk *after* the
indexer had answered and published that as `checkerDigest` — the field
documented as "the bytes the checker resolved against, the ground truth for the
facts".

That is precisely the move IBulkGraphSession.ISnapshot.sources exists to
forbid, and precisely what #103 removed from the source reader: the bytes a
later read returns are not the bytes the indexer saw, a write that lands and
reverts in between is invisible to both reads, and the result would let a
reader prove byte-identity against text the facts were never computed from. An
absent proof is a fact about this indexer; a fabricated one is a lie about the
program.

A SCIP index can only state this when it carries Document.text, which the
parser was also dropping. When it does, the digest is real and the snapshot
claims the sourceDigests capability that licenses a reader to use it. When it
does not, checkerDigest stays empty, the capability is not claimed, the disk
digest is still reported for what it is worth, and the snapshot says why.

Refs #69 #103

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: cover the last two branches, and ignore the compiler's own

The namespace guards were ignored at the wrong line. TypeScript emits the
`X || (X = {})` initialiser at the namespace's *closing* brace, so a directive
placed before the opening one covered nothing. All three merged namespaces are
now bracketed; the constants inside run unconditionally, so the directive hides
nothing testable.

Two real branches remain and get cases: an `undefined` inside an array, which
is the one place the object filter cannot reach it and where dropping the hole
would make two different lists agree; and a reference sitting on the line its
enclosing scope opens on, so containment has to compare columns instead of
stopping once the line numbers match.

Refs #66 #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(scip): cover the optional shapes a real index actually takes

Three branches left, all of them things a real indexer does. toolInfo is
optional in the SCIP schema, so a snapshot has to name the provider that ran it
when the index names no tool. A project root can be a plain absolute path as
well as a file:// URI. And a failure that is not an Error still has to reach
the caller as one, because nobody can read .message off a string.

Refs #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(scip): put the column-comparison case outside the narrower scope

Moving the dependency-leaf reference onto its scope's opening line did make
containment compare columns, but it also put the reference inside the narrower
sibling scope that starts on the same line — so attribution landed on that
declaration instead of the method, which is correct behaviour and the wrong
assertion.

It now sits past the column where the narrow scope ends: the comparison still
has to look at columns, and the innermost scope that really contains it is the
method.

Refs #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(scip): name the authority in a refusal too

Round 10. The README promises every decline names the provider *and* the
authority the build gave up, and the resolve and prepare paths do. The SCIP
refusal named only the provider, so the one thing a reader actually loses —
which grade of fact they are now holding — was missing from the sentence that
exists to tell them.

Also drops a `predicate(..., true)` that read like a check while testing
nothing; the call above it proves admission by not throwing.

Refs #66 #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: keep one live run per branch under review

The queue reached ten runs and roughly eighty jobs, and the head that could
actually gate a merge waited nearly two hours behind results for code it had
already replaced. The experiment workflow fans out sixteen language jobs, so a
branch under active review holds most of the available runners in superseded
work.

A green check belonging to an older SHA proves nothing about the current one —
the campaign procedure says so itself — so those runs were occupying capacity
to produce evidence nobody may merge on. Both workflows now keep one live run
per branch.

Two exclusions, because cancellation is not always right: a push to master is
the verdict on what actually landed and each commit needs its own, and a manual
experiment dispatch is a run somebody specifically asked for.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(scip): give diagnostics their real position, and publish each edge once

Round 11, both defects that a real indexer would have exposed immediately.

The parser dropped Occurrence.diagnostics, so the adapter only ever saw
document-level findings and pinned every one to 1:1. A problem twelve lines
down reported at the top of the file is worse than saying nothing, because it
looks like an answer. A document-level finding still reports at the file's
first position — the file is what it is about — but an occurrence-level one now
keeps the range it was given.

Edges were published once per occurrence. A name referenced twice in one scope
is two occurrences of one relationship, and a symbol listing the same
relationship under two flags is one claim per flag: legitimate in an index,
duplicates in a graph. mergeGraphSlices rejects a strict slice carrying a
duplicate, so the first real scip-go snapshot would have been refused whole.
The first occurrence keeps its evidence, because the earliest is where a reader
is sent.

Refs #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(scip): map the import role SCIP actually states

Round 12. The import role was flattened into a bare reference, which threw away
one of the few facts every SCIP indexer records the same way — and the only one
that says a module boundary was crossed at this occurrence.

It is checked before the access roles because a mask carries more than one bit:
an imported symbol that is also read there is an import, and calling it an
access loses the boundary while keeping something a reader could have inferred
anyway.

The fidelity matrix in #69 lists imports as needing language mapping, and that
remains true of module *topology* — how the import resolved is not in the role.
What the role proves is that the crossing happened here, and the graph's own
`imports` edge means exactly that.

Refs #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(scip): scope a local symbol to the document that owns it

Round 13. A `local N` symbol is an index-local counter scoped to its document,
and the identity derived from it says so — but the definition map was keyed by
the symbol string alone. Two files each holding a `local 4` therefore collided:
the second overwrote the first, and every reference in the earlier file then
resolved to a declaration in the later one, silently and with full confidence.

Definitions are now filed under a document-scoped key when the symbol is
generation-stable, and lookups from inside a document use the same key, so a
local reference can only ever reach the local in its own file. Global symbols
are unchanged — they mean the same declaration wherever named, which is what
makes them global.

A collision check on the derived id remains as a backstop: two documents can
still digest one identity, and publishing it twice would have mergeGraphSlices
reject the whole slice.

Refs #69 #95

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(indexer): check the revision token the fence already named

Round 14. The fence's own comment says a bulk provider's manifest digest is its
revision token — and nothing read it. Bulk sessions are polled at the start of a
refresh and the generic lanes read afterwards, so a provider that rebuilt in
between left the transaction holding one program's facts beside another's. That
is the exact mixture the fence exists to refuse, arriving through the one lane
the fence was not watching.

The transaction now remembers which snapshot it took from each session and
refuses to commit if the session has moved on. Naming a token in a comment is
not the same as checking it.

Refs #67

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(provider): tell a provider's own duplicate from two providers' collision

Round 15. The strict arrays reaching mergeGraphSlices are now every provider's
slices concatenated, not one provider's output, and that made a duplicate node
ambiguous. One provider publishing an id twice is that provider's defect; two
providers publishing the same id means the registry let both own one
declaration — a collision no merge can resolve, because neither slice is wrong
alone and picking either silently discards facts the other proved.

Both are still refused. The message now distinguishes them, which is what makes
the second diagnosable: a reader told only that a node was duplicated looks for
the bug inside one provider and never finds it.

Endpoint closure stays a check over the strict facts as a whole, and the reason
is now written down. A provider owning C and C++ resolves calls that cross
between them, and publishing those is what a shared compilation universe is
for; requiring each slice to close over itself would reject exactly those.

Refs #66

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(scip): cover the shapes a real index has and the fixture did not

Branch coverage stopped at 87% in the adapter, and the gaps were all optional
records that every fixture happened to carry.

Two real shapes fill them: a document the indexer read and found nothing in —
an empty source, or one holding only comments, which belongs in the manifest
because it was read — and a *definition* occurrence whose symbol string cannot
be parsed, where there is no declaration to attach the span to and inventing
one would put a node in the graph named after a string nobody could read.

Refs #69

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(provider): reach the merge step by path, as an internal

CI rejected the import: mergeGraphSlices is not on the package surface, and the
bundled test could not name it. It is the indexer's own merge step rather than
something a consumer calls, so it is reached by path like every other internal
— the same convention isBulkGraphSession already follows, and for the same
reason.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(indexer): republish provenance instead of inheriting it

Round 16. The resident refresh built its dump by spreading the previous one,
so any field it did not overwrite came along — and provenance is where that is
worst. Its manifest and content digests are a proof about one generation, so an
inherited row states that a generation nobody produced is still current, and a
reader comparing digests concludes nothing moved.

This is the same hazard that already retired the carried-forward diagnostics
array in this very function, for the same reason: a dump is a function of its
source, not of the session's history. The dump is now written out, so only the
three facts a refresh genuinely cannot change survive — same project, and the
language set and indexing strategy that sameLanguages already gated on.

The sorting-and-omit rule moves to dumpProvenanceOf.fieldOf, shared by both
publishers. A one-shot build and a resident refresh disagreeing about ordering,
or about what an empty set means, would make one checkout publish two different
dumps.

Refs #66 #67

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(resident): make the mid-transaction rebuild observable when the fence looks

The case started a refresh from inside the static parse and expected the fence
to see it. It could not: the transaction runs from that parse to the fence
without awaiting anything, so the refresh was still a pending microtask when
the check read `current`. The case was passing or failing on scheduling rather
than on the behaviour it names.

The session is hand-rolled here so the rebuild lands synchronously, which is
what a provider that finished a build between the poll and the commit actually
looks like from the transaction's side.

Refs #67

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(resident): open the window the fence guards, and assert the retry

Two errors in one case, both mine.

The window never opened. A resident refresh only runs a static lane for a
language that has no session, and this fixture gave the one language a session
— so the parse the mutation hooked was never called and the fence had nothing
to see. The project is now mixed, TypeScript owned by the provider and Go left
to the static lane, which is the shape the fence exists for.

The assertion was also wrong about the contract. A superseded snapshot is a
stale candidate, and a stale candidate is retried rather than surfaced — that
is what the bounded retry is for. The case now asserts what should happen: the
candidate is discarded and prepared again, and what finally commits carries the
generation the provider actually holds.

Refs #67

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(resident): let the fixture name every file it consumed

The case still never reached the transaction. A resident source derives the
language set it guards from the keys of the sources map its build returned, so
naming only the TypeScript file made discovery see a language the state did
not — and the load rebuilt from scratch rather than refreshing, past the fence
this case is about.

Naming both consumed files is also just what a build result means.

Refs #67

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(resident): model the first strict generation faithfully

* fix(provider): refuse unsafe strict snapshots

* fix(scip): preserve empty source proofs

* fix(provider): refuse duplicate snapshot languages

* fix(test): align conformance with source provenance

* fix(scip): preserve provider input order

* fix(scip): honor pre-aborted refreshes

* fix(scip): canonicalize document paths

* fix(provider): require unique registry identities

* chore(indexer): remove stale provider imports

* fix(scip): reject drive-relative document paths

* fix(provider): retain refresh cleanup failures

* fix(provider): reject duplicate fact declarations

* fix(resident): rebuild on bulk slice language changes

* fix(release): publish the artifacts that passed verification

Close #102: [Release] Restore verified two-package publication

* fix(provider): fence strict generations before publication

* fix(scip): preserve only schema-proven semantics

* fix(provider): commit only stable owned input generations

Fence coordinator-owned inputs without reopening provider-owned source, retry changed-then-reverted consumed text, and apply the same bounded initial transaction to resident static builds.

* fix(scip): honor the current occurrence schema

Normalize typed ranges, retain valid symbol-less diagnostics and optional records, reject invalid int32 fields, report unsupported future semantics, and close the pipeline spawn race.

* test(scip): keep enclosing scopes schema-valid

Place the typed declaration on the line bounded by its deliberately narrow enclosing range so the fixture still exercises column ordering without violating SCIP containment.

* test(provider): prove the remaining strict boundaries

Exercise rejected-session cleanup, multi-provider ordering, resident build-input fencing, duplicate registry facts, and cancellation at the SCIP spawn handoff. Keep only the generated namespace initializer and cryptographic collision guard explicitly excluded from coverage.

* test(provider): exercise every branch boundary

Cover the canonical static fallback, deterministic SCIP cancellation races, omitted semantic names, malformed SCIP identities, ownerless occurrences, and multi-line spans left by the branch-only coverage report.

* test(scip): reject referenced nameless externals

Drive the final external-materialization branch with a referenced SCIP symbol that has neither an explicit nor parsed display name, and retain the assertion that no empty graph handle is published.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bulk index] Introduce overload-safe provider-native symbol identities

1 participant