feat(hnsw): batch neighbor vector reads via badger Txn.MultiGet

[shaunpatterson]

Draft / dependent PR. Requires badger Txn.MultiGet — dgraph-io/badger#2297. CI will not compile until a badger release with MultiGet is available and go.mod is bumped. To review/test locally:

go mod edit -replace github.com/dgraph-io/badger/v4=/path/to/badger   # checked out at sp/badger_multiget

What

HNSW vector search reads each candidate's neighbor vectors one key at a time (getVecFromUid → CacheType.Get), and every Get becomes a full single-key NewKeyIterator(AllVersions) in badger. For a fixed candidate those sibling reads are independent — this batches them into a single read.

Changes

• tok/index: add MultiGet(keys) (vals, errs) to CacheType / Txn / LocalCache (batched counterpart of Get).
• posting:
  • ReadPostingListFromVersions — folds a key's version chain from a badger []ItemVersion exactly as ReadPostingList does from an iterator (delta-on-complete, newest-first, stop on complete/empty/deleted).
  • MemoryLayer.ReadManyData — resolves many keys in one badger.Txn.MultiGet; warm keys served from the global cache; two-phase read mirroring ReadData.
  • LocalCache.MultiGet — adds the per-txn cache/delta layer; viLocalCache/viTxn.MultiGet expose resolved values.
• tok/hnsw: getVecsFromUids batch-fetches a frontier's vectors; searchPersistentLayer collects a candidate's unvisited neighbors and reads their vectors in one MultiGet. Traversal/heap logic is unchanged — only the vector fetch is batched.
• TxnCache/QueryCache and the test mocks implement MultiGet.

Testing (with the local badger replace)

• Differential test (posting): ReadManyData and viLocalCache.MultiGet return identical results to the per-key ReadPostingList/Get path over an on-disk round-trip of complete/delta/empty/absent posting shapes.
• tok/hnsw and posting suites pass; full dgraph build clean.

Benchmarks (cold cache, real badger; benchstat n=6)

• badger MultiGet vs per-key NewKeyIterator: −5…−16% time, −35…−40% allocs (grows with frontier size).
• posting frontier read (Get loop vs MultiGet, BenchmarkHNSWFrontierRead): −12…−18% time, −15…−21% allocs for K=16/64/256.

This speeds up the neighbor-vector read path; whole-query latency improves by that fraction of time spent reading vectors.

🤖 Generated with Claude Code