Benchmark and tune automatic memory recall

[sentry-junior]

Automatic memory recall needs lower and predictable latency without losing lexical-only or semantic-only memories. The current implementation on main already uses the right baseline: parallel bounded vector and PostgreSQL full-text retrieval, reciprocal rank fusion (RRF), a vector distance cutoff, and a relevance model that admits at most five memories.

PR #1244 explored skipping lexical retrieval whenever vector recall returned any in-threshold candidate. That is unsafe: one plausible vector distractor can prevent a stronger exact/token match from entering the candidate set, and vector misses make the serial fallback slower than the current parallel path.

Current behavior

• recallMemories retrieves vector and lexical candidates in parallel.
• Each leg receives a bounded candidate limit derived from the 20-memory recall window.
• Vector-only candidates are filtered by MEMORY_RECALL_MAX_VECTOR_DISTANCE (default 0.45).
• PostgreSQL full-text ranking is bounded by perf(memory): bound lexical ranking candidates #1242 after indexed filtering from perf(memory): optimize lexical search #1233.
• RRF combines ranks without comparing provider-specific raw scores.
• The relevance model performs final admission.

Proposed work

1. Keep both retrieval legs parallel and preserve RRF. Do not conditionally suppress lexical retrieval based on vector results.
2. Add a recall-specific quality corpus covering:
   • an exact/token lexical match competing with an in-threshold vector distractor;
   • a semantic-only match;
   • a memory supported by both legs;
   • no relevant memory.
3. Benchmark the recall path with representative memory counts and broad/narrow queries. Capture p50/p95 latency for each retrieval leg and end-to-end candidate retrieval, plus recall@20 / MRR.
4. Use that evidence to tune only the owning knobs:
   • per-leg candidate counts / overfetch;
   • lexical rank window;
   • vector distance threshold for the configured embedding model;
   • an ANN vector index only if EXPLAIN (ANALYZE, BUFFERS) shows vector scanning is material and filtered query plans use it.
5. Record the chosen limits and their benchmark rationale beside the constants.

Acceptance criteria

• Lexical-only and semantic-only expected memories remain in the 20-candidate recall window.
• An in-threshold vector distractor cannot suppress a stronger lexical result.
• No regression in the existing retrieval-quality suite or memory evals.
• The selected tuning has recorded before/after query plans and latency/quality results.
• Automatic recall remains optional and failure-safe.

This is separate from #1247 / #1246, which optimize passive extraction pre-search rather than turn-start recall.

Requested by David Cramer.

--

View Junior Session [Sentry]

[sentry-junior]

Implemented the recall MVP in #1250.

Shipped:

• keep parallel hybrid RRF; no lexical skip on vector hits
• HNSW cosine index on embeddings
• SQL-side recall distance cutoff
• hard caps: recall overfetch 2x, max 40/leg, lexical rank window 200, query text 1500 chars
• slight lexical RRF preference on automatic recall
• hybrid contract + weight tests

Still open after merge:

• production EXPLAIN / p50-p95 per-leg latency
• optional further knob tuning from those measurements
• query rewrite for multi-turn pronouns remains deferred