research(nightly): diversity reranking for ANN results — MMR and MinCut-inhibition in Rust

[ruvnet]

Nightly Research: Diversity Reranking for ANN Candidate Sets

Adds ruvector-diversity-rerank, a Rust crate providing pluggable post-retrieval diversity reranking over any ANN candidate set. Connects RuVector's mincut philosophy to the well-studied diversity retrieval problem (MMR, graph-cut inhibition).

What's Included

1. Working Rust PoC — crates/ruvector-diversity-rerank/ with DiversityReranker trait and 3 variants
2. ADR — docs/adr/ADR-268-diversity-rerank.md
3. Research document — docs/research/nightly/2026-06-22-diversity-rerank/README.md
4. Real benchmark results — captured from cargo run --release -p ruvector-diversity-rerank --bin benchmark
5. Public gist — docs/research/nightly/2026-06-22-diversity-rerank/gist.md

Three Reranking Variants

Variant	Algorithm	Mean latency (N=100, d=64)	Diversity gain	Recall@10
BaselineReranker	Sort by ANN distance	11 µs	1× (reference)	1.000
MmrReranker	Maximal Marginal Relevance (λ=0.5)	430 µs	3.2×	0.300
MinCutReranker	Greedy threshold-graph inhibition (θ=0.85)	420 µs	6.2×	0.100

Acceptance Test

ACCEPTANCE (N=200, dim=64, k=20, seed=9876):
  baseline diversity:   0.1066
  mmr diversity:        0.2363  [+122% vs baseline]  PASS
  mincut diversity:     0.5577  [+423% vs baseline]  PASS
ACCEPTANCE: PASS

Tests

test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured

Why This Matters

• Agent memory: Prevents recall bias; surfaces breadth of experience instead of reinforcing recent memories
• RAG pipelines: Diverse chunks → higher ROUGE / BERTScore; reduces context window waste
• Graph RAG: Diverse retrieval covers multiple subgraph components per query
• Edge / MCP: MinCut at N≤100 runs in 420 µs — suitable for Cognitum Seed and MCP memory tools
• First in RuVector: No prior nightly covers diversity reranking; GNN reranking (2026-05-21) targeted accuracy, not diversity

Research doc: docs/research/nightly/2026-06-22-diversity-rerank/README.md
ADR: docs/adr/ADR-268-diversity-rerank.md

---

Generated by Claude Code