Evaluate typed edges on existing chunk_links graph

[laynepenney]

Context

We already have a similarity graph: chunk_links stores pre-computed cosine similarity edges between chunks across sessions (top-3 neighbors per chunk, min similarity 0.35). This is traversed at query time via _expand_cross_session() with a 0.7 discount factor.

The current edges are untyped — they say "these are related" but not HOW (supports, contradicts, supersedes, extends, etc.). The question: does adding edge types improve retrieval quality enough to justify the cost?

What we already have (5 expansion mechanisms)

1. Cross-session chunk links — chunk_links table, cosine similarity, pre-computed at build
2. Same-session context — surrounding turns via turn_index lookup
3. Knowledge node source expansion — source_sessions/source_turns back-references
4. Topic clustering — Jaccard-based clusters in cluster table
5. Hybrid RRF — BM25 + embedding fusion with auto-boost

Proposed evaluation

Phase 1: Measure the gap (no code changes)

Add a "multi-hop" category to CodeMemo with 10-15 questions that require connecting information across sessions where the connection isn't obvious from keywords or embeddings alone. Examples:

• "We decided X in March, then reversed it in April. What's the current state?"
• "Agent A proposed a fix. Agent B found a problem with it. What was the problem?"
• Questions where following a typed edge (supersedes, contradicts) would help but generic similarity wouldn't

Run against current system. If flat retrieval scores >85%, typed edges may not be worth the cost.

Phase 2: Prototype typed edges (if Phase 1 shows a gap)

Option A — Classify at build time: During build_cross_session_links(), run a lightweight classifier on each (source, target) pair to assign a type. Could be rule-based (temporal ordering → supersedes, high similarity + different conclusion → contradicts) or a small model.

Option B — Classify at query time: Keep untyped edges, but when expanding, use query intent to filter. A "what changed" query only follows edges where timestamps differ significantly. A "what supports" query only follows high-similarity same-conclusion edges.

Option C — Add link_type column to chunk_links: Enrichment model outputs edge type alongside knowledge nodes in the same pass. No extra LLM call — just an extra field in the extraction prompt.

Speed constraints (local-first)

• Current build_cross_session_links() time: needs benchmarking
• Budget: typed edge classification should add <20% to build time
• Query-time expansion must stay <50ms for the full graph hop
• No external API calls for edge classification — must run on local models or rules
• CROSS_LINK_MAX_EXPAND=3 means we're only traversing 3 edges per result — typed filtering on 3 edges is essentially free

Decision criteria

Metric	Threshold to ship
Multi-hop accuracy improvement	>5% on new CodeMemo category
Build time increase	<20%
Query latency increase	<10ms
Cold start impact	<2s additional

If we can't hit all four, the current untyped graph is good enough.

References

• r/AIMemory thread on knowledge graph expectations (2026-04-06)
• Current expansion: core.py:1160-1321 (build), core.py:2068 (query-time expand)
• chunk_links schema: (source_id TEXT, target_id TEXT, similarity REAL)

[laynepenney]

Sprint 18 Evaluation: Typed Edges on chunk_links

Current State Analysis

The existing chunk_links system is well-scoped: 3 neighbors per chunk, cosine similarity >= 0.35, 0.7 discount on expansion. At query time, we expand from the top 5 results with at most 3 cross-session hops. The schema is minimal: (source_id TEXT, target_id TEXT, similarity REAL).

Five expansion mechanisms already cover a broad surface:

1. Cross-session chunk links (cosine similarity, what we're evaluating here)
2. Same-session context (turn_index proximity)
3. Knowledge node source expansion (source_sessions/source_turns back-refs)
4. Topic clustering (Jaccard clusters)
5. Hybrid RRF (BM25 + embedding fusion)

Cost/Benefit Assessment

Structural cost of typed edges: low. Adding a link_type TEXT column to chunk_links is a one-migration change. Storage cost is negligible (3 links per chunk, one extra string field per row). The query-time filtering on 3 edges is essentially free.

Classification cost: moderate. Three realistic options:

Approach	Build time impact	Accuracy	Complexity
Rule-based (temporal + similarity heuristics)	<5%	Low (heuristic)	Low
Enrichment model sidecar (extract type during existing enrich pass)	~0% (piggyback)	Medium	Medium
Dedicated classifier pass	20-50%	High	High

The enrichment piggyback (Option C from the issue) is the clear winner: we already run an LLM over every chunk during enrich(). Adding "relationship to prior chunks" as an extraction field costs no extra inference calls; just a prompt addition.

Gap Assessment

The critical question: does untyped similarity already handle the multi-hop cases well enough?

Looking at the 5 expansion mechanisms, the system already has implicit relationship signals:

• Temporal ordering is encoded in chunk IDs (session + turn index); a "supersedes" query can be resolved by retrieving the later chunk when both are surfaced
• Contradiction is partially handled by the contradictions table (separate from chunk_links)
• Extension/support is the default assumption when cosine similarity is high

The cases where typed edges would add signal beyond what we already have:

1. Two chunks with high similarity but opposite conclusions (cosine can't distinguish agreement from disagreement)
2. A chunk that supersedes another where the topic drifted enough to lower cosine similarity below 0.35
3. Multi-step reasoning chains where edge A -> B -> C requires knowing B extends A and C contradicts B

Recommendation

Defer implementation to Sprint 19+. The gap is narrow and the existing contradictions table partially covers the highest-value case.

Specific reasoning:

• The contradictions table already handles the "opposite conclusion" case, which is the most valuable typed-edge scenario
• Current LOCOMO J=72.4% and CodeMemo J=90.51% don't show a multi-hop retrieval gap
• The enrichment piggyback approach (Option C) is the right implementation path when we do this, but it requires careful prompt engineering to avoid regressing extraction quality
• Phase 1 of this issue (measuring the gap with new multi-hop CodeMemo questions) should be the gate before any implementation

Sprint 19 action items if we proceed:

1. Add 10-15 multi-hop questions to CodeMemo
2. Run baseline; if accuracy < 85%, proceed
3. Prototype Option C: add relationship_to_prior field to enrichment prompt
4. Benchmark build time delta
5. Measure retrieval improvement on multi-hop category

Premium boundary

OSS (recall infrastructure). Typed edges would be part of the core chunk_links system, not premium. The enrichment prompt change would be in the public synapt package.