feat(turbovec): multi-bit TurboQuant FastScan ANN index (ADR-194 M1) by shaal · Pull Request #521 · ruvnet/RuVector

shaal · 2026-05-30T03:28:32Z

What & why

Adds crates/ruvector-turbovec — a multi-bit TurboQuant FastScan-style ANN index — and ADR-194 documenting it. This closes the one mainstream ANN regime ruvector lacks: 2–4-bit scalar-quantized search (FAISS IndexPQFastScan / Qdrant / Milvus IVF-SQ territory).

Companion issue with the full rationale: #520.

The gap this fills

Crate	Quantization	Note
`ruvector-rabitq`	1-bit	great when memory dominates, but caps recall unless you re-inflate memory with an f32 rerank
`ruvllm` `turbo_quant.rs`	2.5–4 bit	a KV-cache value codec, not an `AnnIndex` (no top-k / LUT kernel / IDs)
`ruvector-turbovec` (new)	2/4-bit	the missing multi-bit search index

Techniques adapted from RyanCodrai/turbovec (Google Research TurboQuant, arXiv:2504.19874). Clean-room crate, not a port — it reuses ruvector's own primitives rather than duplicating them.

Why it's a good fit for this repo

Zero new plumbing — implements the existing ruvector_rabitq::AnnIndex trait and reuses ruvector_rabitq::RandomRotation (the randomized Hadamard, already cited to the same arXiv paper in rotation.rs). Drops into the same consumers as RaBitQ.
Recall without a mandatory f32 rerank — a per-vector length-renormalization scalar c_x makes the inner-product estimator unbiased at zero query-time cost (it folds into the final multiply), so the memory win is real.
Codebook-free, online ingest, no training — Lloyd–Max centroids are constants of the post-rotation Gaussian marginal; TQ+ per-coordinate calibration is frozen after a warm-up batch.
Composable — the same codes can later back IVF-SQ-FastScan, a natural follow-up to ADR-193 (rairs-ivf).

Pipeline (ADR-194 §T1–T4, T6)

norm = ‖x‖ ; û = x/norm                  strip & store length        (§T1)
r = P·û   (randomized Hadamard, reused)  → coords ≈ N(0,1)           (§T1)
z_i = (r_i − shift_i)/scale_i            TQ+ per-coord calibration    (§T3)
q_i = argmin |z_i − centroid|            Lloyd–Max 2/4-bit SQ         (§T2)
c_x = ⟨r,r̂⟩/⟨r̂,r̂⟩                       per-vector unbiased scale     (§T4)
score = ‖q‖² + ‖x‖² − 2·‖q‖‖x‖·c_x·⟨r̂,s⟩ ≈ ‖q − x‖²

Plus IdMapIndex: external u64 ids, O(1) delete (tombstone), allowlist-filtered search (§T6).

Measured proof — `cargo run --release -p ruvector-turbovec`

n = 5,000 uniform-random vectors (the worst case for ANN — no cluster structure), dim = 256, k = 10, no f32 rerank, scored vs exact brute-force L2:

Width	recall@10	bytes/vec (raw 1024)	compression	mean cosine bias
1-bit	0.308	48	25.6×	+0.0005
2-bit	0.561	80	14.2×	+0.0001
3-bit	0.767	112	9.8×	−0.0000
4-bit	0.879	144	7.5×	−0.0000

Recall rises monotonically with bit-width — precisely the regime 1-bit RaBitQ can't reach without re-inflating memory.
Mean cosine bias ≈ 0 at every width — empirical confirmation that the c_x renormalization yields an unbiased estimator (the core TurboQuant claim).
On real clustered embeddings (OpenAI/Cohere), recall at a given width is materially higher than this uniform stress test.
Determinism (same seed → bit-identical) and IdMap delete + allowlist filter both PASS.

Tests / quality

17 unit tests + 1 doc-test pass; covers pack/unpack round-trip (all widths), calibration fit, nearest-centroid quant, recall threshold, unbiasedness, determinism, exact-fallback for tiny indexes, IdMap delete/dup-reject/filter, wrong-dim rejection, zero-vector calibration exclusion, and a distortion-bound oracle (per-coordinate MSE under the paper's D_mse ≤ (√3·π/2)·4^(−b) and within 5% of the Max-1960 optimum, every width).
cargo build --release -p ruvector-turbovec green; clippy clean; cargo fmt --check clean.
Files kept under 500 lines; no unsafe in M1.

Post-review hardening (commits 5–9)

After the initial push the branch was tightened — each as its own focused commit:

Review fixes — input validation, release-build error propagation, zero-vector calibration exclusion (addresses CodeRabbit); dropped an unused dep.
3-bit width (ADR-194 §D3) — BitWidth::Three (8-level Max-1960 levels) fills the 2↔4-bit recall cliff: 0.767 @ 9.8×, measured. The bit-packer was already width-generic, so this is a centroid table + enum arms.
Distortion-bound oracle (§D4) — the theory-grounded MSE test above.
cargo fmt — isolated, normalizes pre-existing >100-col lines.

ADR-194 gained a "Divergences from the TurboQuant paper (arXiv:2504.19874)" section (D1–D5): where M1 departs from the paper and why. The notable one (D1): M1 scores with a per-vector c_x length-renormalization — empirically near-unbiased — rather than the paper's provably unbiased two-stage MSE + 1-bit-QJL-residual estimator. That's a deliberate cost/accuracy trade (zero extra bits vs m bits/vector); the QJL stage is scoped as a follow-up (M5), designed in #520.

Scope / follow-ups

This is M1 (scalar reference), now shipping 1 / 2 / 3 / 4-bit. The scalar scorer is intentionally the determinism oracle for later SIMD work:

M2 — FastScan nibble-LUT SIMD kernel (AVX2 + NEON), fuzzed bit-identical.
M3 — .tv persistence.
M4 — AVX-512BW kernel + ruvector-rulake dispatcher registration.
M5 — paper-grade unbiased scoring via the QJL residual stage (§D1; design in #520).
M6 — d-aware Beta-optimal codebooks (§D2).

Files

docs/adr/ADR-194-ruvector-turbovec-fastscan-index.md — full design, reuse boundaries, alternatives, acceptance criteria, measured results.
crates/ruvector-turbovec/ — quantize.rs, calibrate.rs, index.rs, idmap.rs, error.rs, lib.rs, main.rs (proof demo).
Cargo.toml / Cargo.lock — workspace member registration.

Note: benchmark figures attributed to the upstream RyanCodrai/turbovec project in ADR-194 are clearly marked as that project's numbers, not this crate's; this crate's own measured results are the uniform-random table above.

…N index Research and scope a new crate adapting TurboQuant techniques from RyanCodrai/turbovec: 2/4-bit Lloyd-Max scalar quantization, TQ+ per-coordinate calibration, length-renormalized unbiased scoring, and a nibble-LUT FastScan SIMD kernel (AVX-512BW/AVX2/NEON). Reuses ruvector-rabitq's Hadamard rotation + AnnIndex/VectorKernel traits and borrows ruvllm's MSE quantizer math, closing the missing 2-4-bit FastScan ANN regime. https://claude.ai/code/session_012AzArCzBwxrJp8mUngUcH5

New crate crates/ruvector-turbovec implementing the scalar-reference milestone of ADR-194 — the 2/4-bit FastScan-style ANN regime ruvector lacked (rabitq is 1-bit; ruvllm's TurboQuant is a KV-cache codec, not a search index). Pipeline: normalize -> randomized Hadamard rotation (reused from ruvector-rabitq) -> TQ+ per-coordinate calibration -> Lloyd-Max 2/4-bit scalar quantization -> per-vector length-renormalized unbiased scoring. Implements the shared ruvector_rabitq::AnnIndex trait. Adds IdMapIndex with O(1) delete and allowlist-filtered search. Proof (cargo run --release -p ruvector-turbovec), n=5000 uniform-random vectors, dim=256, k=10, no f32 rerank, vs exact brute-force L2: 1-bit: recall@10 0.308, 25.6x compression, bias +0.0005 2-bit: recall@10 0.561, 14.2x compression, bias +0.0001 4-bit: recall@10 0.879, 7.5x compression, bias -0.0000 Recall rises monotonically with bit-width; mean cosine bias ~0 confirms the unbiased estimator. Determinism + IdMap delete/filter verified. 12 unit tests + 1 doc-test pass; build green; clippy clean. M2-M4 (FastScan nibble-LUT SIMD kernel, AVX-512, dispatcher) are future milestones; the scalar scorer here is their determinism oracle. https://claude.ai/code/session_012AzArCzBwxrJp8mUngUcH5

https://claude.ai/code/session_012AzArCzBwxrJp8mUngUcH5

… errors - index: TurboVecIndex::add/search now return RabitqError::DimensionMismatch in release builds instead of silently accepting/masking wrong-length vectors (was debug_assert + unwrap_or_default). - index: finalize() excludes zero vectors from calibration fit so they don't bias shift/scale toward zero. - idmap: add_with_id validates dim up front and reports the real length (was hardcoded got: 0); add_with_ids rejects vectors/ids length mismatch with new TurboVecError::BatchLenMismatch instead of zip-truncating. - quantize: pack/unpack document preconditions and debug_assert code-range and slice-length (proportionate to internal helpers; no Result churn). - calibrate: fit debug_asserts every row has length dim. - ADR-194 frontmatter status proposed -> accepted to match body. Adds 4 tests (wrong-dim reject on add/search, zero-vector calibration exclusion via self-retrieval, batch-len mismatch, idmap wrong-dim). 16 unit tests + 1 doc-test pass; clippy clean; demo unchanged. https://claude.ai/code/session_012AzArCzBwxrJp8mUngUcH5

- Cargo.toml: remove unused rand_distr dependency and the redundant rand dev-dependency (rand is a normal dep for the demo bin + tests). - Cargo.lock: drop rand_distr from ruvector-turbovec. - ADR-194: attribute the FAISS-competitive figures to the upstream RyanCodrai/turbovec project rather than presenting them as this crate's measured results; point readers to the reproducible uniform-random Validation table instead. No code changes; 16 unit + 1 doc-test still pass, clippy clean.

Add an explicit 'Divergences from the TurboQuant paper (arXiv:2504.19874)' section mapping where M1 departs from the paper, so the gaps are reviewable and the follow-ups are paper-grounded: - D1: M1 uses a heuristic per-vector c_x scale, not the paper's provably- unbiased two-stage MSE + 1-bit-QJL-residual estimator. Soften the T4 and Validation wording accordingly (empirically near-unbiased, not proven). - D2: M1 quantizes against the N(0,1) limit + empirical TQ+ calibration, not the paper's exact d-aware Beta-optimal codebooks. - D3: M1 ships 1/2/4-bit; paper highlights ~2.5/3.5 bpc sweet spots — add 3-bit. - D4: assert measured distortion under the paper's closed-form bounds as a stronger test oracle than recall > 0.5. - D5: estimator variance deferred. Add milestones M5 (paper-grade QJL-residual estimator) and M6 (Beta-optimal codebooks); note what M1 already matches (norm-based L2, online ingest, full-precision query). No code change.

…ll cliff Adds BitWidth::Three (8-level Max-1960 optimal N(0,1) reconstruction levels). pack/unpack, calibration, scoring, and IdMap are width-generic, so only the centroid table + the enum arms change. Measured (cargo run --release -p ruvector-turbovec, n=5000 uniform-random, dim=256, k=10, no rerank, vs exact L2): 3-bit: recall@10 0.767, 112 B/vec, 9.8x compression, bias -0.0000 landing squarely between 2-bit (0.561) and 4-bit (0.879) — a useful memory/recall midpoint (~22% smaller than 4-bit for ~0.11 recall). Also refresh ADR-194: add the 3-bit Validation row, mark D3 done, widen T2 to {2,3,4}, correct the test count to 16, and scope the provenance note so the measured recall/compression/bias figures are called measured while the FAISS-competitive claims stay attributed targets. 16 unit + 1 doc-test pass; clippy clean; new code is rustfmt-clean.

Pure rustfmt normalization (rustfmt 1.8.0-stable) of lines that predated this branch; no semantic change. Isolated from the feature commits so the crate passes 'cargo fmt --check' cleanly for upstream CI.

Add quantizer_mse_within_paper_bound: draw 400k N(0,1) samples (Box–Muller, no new deps), quantize via the real quantize_coord path, and assert the per-coordinate MSE for every width stays under TurboQuant's distortion bound D_mse ≤ (√3·π/2)·4^(−b) (arXiv:2504.19874) AND within 5% of the Max-1960 Lloyd–Max optimum. A corrupted centroid level trips this far more precisely than the existing recall>0.5 threshold. Marks D4 done in ADR-194; updates test count to 17. The full-pipeline inner-product bound D_prod remains future work (tracked with D5).

Canonical ADR for the 2-4-bit scalar-quantized FastScan search index proposed in #520 / PR #521. Numbered 254 because the PR drafted it as ADR-194, which collides with the merged ADR-194 (ONNX embedder). Captures the gap, the T1-T6 design, reuse boundary, milestones M1-M5, measured M1 validation, and honest divergences from the TurboQuant paper. Co-Authored-By: claude-flow <ruv@ruv.net>

ruvnet · 2026-06-16T16:11:36Z

ADR number collision — needs renumber. This PR adds docs/adr/ADR-194-ruvector-turbovec-fastscan-index.md, but ADR-194 is already taken on main (ADR-194-ruvector-onnx-embedder-api-and-throughput.md, merged). Two ADR-194s.

I've reviewed #520 and authored the canonical record at the next free number — ADR-254 (a7028efc on main) — faithful to this PR's T1–T6 design, reuse boundary, milestones, measured M1 validation, and the honest divergences-from-paper caveats.

To land this PR cleanly, please git mv the ADR-194 file → ADR-254 (and update its adr:/heading to 254), or drop it in favor of the on-main ADR-254. The crate code (crates/ruvector-turbovec) is unaffected. Everything else in the M1 review looks strong: clean reuse of rabitq's rotation + traits, matter-of-fact measured numbers, and the scalar oracle as the determinism reference for the SIMD milestones.

ADR-194 is already taken on main (ruvector ONNX embedder API & throughput). Renumber this PR's turbovec ADR to the next free number (254), matching the canonical record on main. Keeps the fuller PR version (D1–D5 divergences table, D3/D4 measured-milestone markers) and adds a numbering note. Updates the 13 in-crate ADR-194 references and two stray ADR-193 'future work' pointers so they no longer resolve to the unrelated ONNX ADR. Refs ruvnet#520, ruvnet#521

…vec-optimization-FhaDh # Conflicts: # docs/adr/ADR-254-ruvector-turbovec-fastscan-index.md

shaal · 2026-06-17T17:58:01Z

Addressed the ADR-194 collision flagged above — took option 1 (renumber, keep this PR's fuller draft).

Done (commits 939adc9e + merge 5359b2c2):

git mv ADR-194-…turbovec… → ADR-254-…turbovec…; bumped adr:/heading/related and added a numbering note pointing at the canonical 254.
Updated the 13 in-crate ADR-194 references (7 files) + two stray ADR-194 'future work' pointers in ADR-193-rairs-ivf.md so none resolve to the unrelated ONNX ADR.
Merged origin/main and resolved the resulting ADR-254 add/add conflict. Build green; 17 unit tests + 1 doc-test pass on the merged tree.
Issue Missing ANN regime: 2–4-bit scalar-quantized FastScan search index #520 body + a comment updated to reference ADR-254.

One note for reconciliation: this branch's ADR-254 is the fuller version — it keeps your numbering note and adds the D1–D5 divergences table plus the D3 (3-bit) and D4 (distortion-bound oracle) measured-milestone markers that shipped in this PR. I resolved the merge in favor of it (superset of the condensed on-main draft), so keep this branch's copy at merge. Happy to graft anything from the on-main version (e.g. the explicit Links: block) if you'd prefer.

@indexed

…35) (#3) * fix(ruvector-wasm): correct adapter for WASM build's flat-index, distance-score, and metadata gaps (ruvnet#568) The published @ruvector/wasm build behaves differently from its generated .d.ts in three ways that bite consumers: 1. HNSW is not active — the wasm32 target compiles without the `hnsw` feature and falls back to a flat (brute-force) index, so search is O(n). The O(log n) win is latent until the WASM HNSW lands. 2. `result.score` is a cosine distance (lower is better), not the "higher is better" similarity the .d.ts advertises (ordering is correct: a, b before c). 3. Metadata does not round-trip — search/get return {}. Add RuvectorWasmAdapter (@ruvector/wasm/adapter) which wraps VectorDB with: - a metadata sidecar so inserted metadata round-trips - similarity = 1 - distance (generalised per metric) with `.score` aliased to similarity, plus the raw `distance` preserved - indexType/usesHnsw + WASM_HNSW_AVAILABLE so callers don't assume HNSW - client-side metadata filtering with over-fetch Includes TS declarations with corrected doc comments, a node:test suite covering all three findings, README guidance, and package exports. Co-authored-by: Claude <noreply@anthropic.com> * chore: Update NAPI-RS binaries for all platforms Built from commit 08c0d74 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore(ruvector-wasm): publish @ruvector/wasm 0.1.31 with corrected adapter Ships the RuvectorWasmAdapter (ruvnet#568) and restores a functional package — 0.1.30 published with only package.json (empty pkg/). 0.1.31 includes the built web pkg/ plus the adapter that corrects similarity score, metadata round-trip, and flat-index reporting. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 054d815 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(tiny-dancer): lock-step platform versioning + drop dead release profile Root cause of version drift: build-tiny-dancer.yml hard-coded VERSION="0.1.15", so every publish shipped stale platform binaries while the main package advanced (npm 0.1.18 was loading 0.1.15-era .node files). - workflow: derive VERSION from package.json; rewrite main optionalDependencies to pin that exact version before publishing, so binaries and JS can never skew. - package.json: bump to 0.1.19, pin all 5 optionalDependencies to 0.1.19, remove dead `publish:platforms` script (scripts/publish-platforms.js absent). - crate Cargo.toml: remove [profile.release] (Cargo ignores non-root profiles; release opt is already opt-level=3/lto=fat/codegen-units=1/strip at root). Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 57187b0 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * feat(tiny-dancer): full platform matrix — add musl (Alpine) + Windows ARM64 Extends the build/publish workflow from 5 to 8 environments: - linux-x64-musl, linux-arm64-musl (Alpine/Docker) via napi --use-napi-cross - win32-arm64-msvc (Windows on ARM) via rustup target cross-compile index.js now detects musl vs glibc at load time (process.report glibc header) and routes to the correct binary; adds win32-arm64. package.json bumped to 0.1.20 with all 8 optionalDependencies lock-stepped. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 1ef4eb8 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(tiny-dancer): use --zig + setup-zig for musl cross-builds (napi 2.18 has no --use-napi-cross) Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 4c448f0 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * feat(tiny-dancer): real FastGRNN training pipeline (ADR-252) Closes the three gaps that made tiny-dancer inference-only: 1. Real gradients: FastGRNN::forward_cached + backward implement single-step analytic backprop (h0=0); gradient-checked vs central finite differences. 2. Real Adam step: train_batch accumulates mean batch gradients; apply_gradients does L2 + global-norm clip + bias-corrected Adam update on the existing optimizer state. Model now actually learns (test: loss down, acc>0.9). 3. safetensors persistence: model.rs save/load serialize every tensor (f32 LE) with config in __metadata__; round-trip is bit-exact. 4. DRACO adapter: TrainingDataset::from_draco consumes the {embedding, scores} + prices shape (same as @metaharness/router) so one dataset trains both. Runnable example train_from_draco demonstrates DRACO -> train -> save -> load -> route end to end. 31 core tests green (gradient check, convergence, round-trip, adapter). Co-Authored-By: claude-flow <ruv@ruv.net> * feat(tiny-dancer-node): napi trainRouter export (ADR-252 B) Exposes the FastGRNN training pipeline to JS: trainRouter(rows, prices, opts) consumes DRACO {embedding, scores} rows, trains, and writes a .safetensors the existing `new Router({ modelPath })` loads. Type-checks; reaches JS on the next tiny-dancer platform rebuild/republish. Co-Authored-By: claude-flow <ruv@ruv.net> * release(tiny-dancer): 0.1.21 — expose trainRouter to JS (types + bump) Adds trainRouter/DracoRowJs/TrainRouterOptions/TrainRouterResult to the hand-written index.d.ts and bumps to 0.1.21 (8 optionalDependencies lock-stepped) so the native train_router export ships typed across all environments. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit e709718 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore: Update NAPI-RS binaries for all platforms Built from commit 1aa132b Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore: Update NAPI-RS binaries for all platforms Built from commit c520576 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * feat(ruvector-cli): integrate tiny-dancer router (train/route/info) npx ruvector tiny-dancer train <draco.json> --out model.safetensors trains a FastGRNN cost-optimal router from a DRACO {embedding,scores} dataset; `route` loads + routes; `info` reports availability. @ruvector/tiny-dancer added as an optionalDependency (native binary, lazy-required with install hint). Bump 0.2.31. Co-Authored-By: claude-flow <ruv@ruv.net> * feat(tiny-dancer): add score() raw-forward inference + 0.1.22 score(modelPath, embedding) loads a trainRouter .safetensors and runs the FastGRNN directly on the raw embedding (the inference path that matches how trainRouter trains — Router.route's 5-feature engineering is a different model contract). Validated easy=1.0/hard=0.0 on a freshly built binary. Co-Authored-By: claude-flow <ruv@ruv.net> * fix(ruvector-cli): tiny-dancer score command (raw forward) + dep ^0.1.22 Replaces the route subcommand (which used Router.route's 5-feature engineering — a different model contract) with 'score', backed by tiny-dancer's score() raw forward pass that matches trainRouter models. Also fixes the earlier Float32Array issue by removing the Router.route path entirely. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 39fb398 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore: Update NAPI-RS binaries for all platforms Built from commit 5173ce7 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore: Update NAPI-RS binaries for all platforms Built from commit 3c1f701 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * docs(adr): ADR-252 HelixDB vs RuVector comparison and improvement opportunities (ruvnet#570) * docs(adr): ADR-252 HelixDB vs RuVector comparison and improvement opportunities Compares HelixDB (LMDB/heed, compiled type-safe HelixQL, graph-vector thesis, graph-vector-bench) against RuVector's redb/Cypher/hybrid stack and proposes 7 prioritized, opt-in improvements: optional schema layer with load-time validation, first-class typed graph-vector binding and a unified search-then-traverse operator, in-query embed(), unified ANN+BM25+graph RRF hybrid, a reproducible benchmark harness, schema-driven typed SDK codegen, and an object-storage tier research spike. https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * feat(ruvector-graph): native schema layer + typed search-then-traverse (ADR-252 P1/P2/P4) Implements the HelixDB-inspired improvements natively in ruvector-graph: - schema.rs: opt-in GraphSchema (N::/E::/V:: equivalents) with load-time validation (self-consistency, node required/typed props + strict mode, edge from/to label constraints, vector dimension checks), higher-is-better distance metrics (cosine/dot/euclidean), and reciprocal_rank_fusion (P4). - typed_graph.rs: TypedGraph wrapper validating mutations pre-storage, plus a fused typed search_then_traverse operator (HelixQL SearchV<T>(q,k)::In/Out<E>) with optimized bounded-heap top-k selection (O(n log k)). Pure-Rust, no new deps, WASM-safe. 13 new tests, 148/148 lib tests green, clippy clean. Schemaless mode remains the default (opt-in coexistence). https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * perf(ruvector-graph): optimize search_then_traverse + add criterion bench (ADR-252) Hot-path optimizations for the typed search-then-traverse operator: - GraphDB::with_node / node_ids_by_label: zero-copy borrow scoring, eliminating per-candidate Node + embedding clones (get_nodes_by_label cloned everything). - Fused single-pass cosine (q.c and c.c in one read of the candidate) + hoisted query norm out of the per-candidate loop. - Bounded top-k min-heap (O(n log k)); clone id only for heap winners. - Rayon parallel scan over DashMap for >=4096 candidates (per-thread heaps, bounded merge); serial path below threshold. Adds benches/typed_graph_bench.rs (criterion). Measured vs first cut (128-dim, k=10): 10k 7.2ms->3.08ms (2.34x), 50k 74.3ms->28.5ms (2.61x), 1k 539us->432us. New parallel-vs-reference correctness test. 149/149 lib tests green, clippy clean. https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * feat(ruvector-graph): HNSW push-down for search_then_traverse (ADR-252 P2) Adds an opt-in ANN path to the typed search-then-traverse operator, removing the O(n) full-label scan for indexed vector types: - TypedGraph::build_vector_index(vector_type) builds a per-vector-type HybridIndex (HNSW under hnsw_rs, exact FlatIndex otherwise), holding only the bound label's nodes so searches stay label-scoped. Kept current incrementally via create_node -> index_node. - search_then_traverse routes through the index when present: ~O(log n) approximate search, over-fetch (max(4k, k+32)), then exact rescore with the schema metric so ANN results carry identical higher-is-better score semantics to the brute-force path. Brute force remains the default. - Parallel brute-force path refactored to capture &GraphDB (not &self) so it stays Send+Sync independent of the index's thread-safety bounds. Bench (50k nodes, 128-dim, k=10): brute-force parallel scan 27.6ms -> HNSW push-down 1.05ms (~26x; ~70x vs first cut). 151/151 lib tests green (3 new HNSW tests), clippy clean. https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * feat(ruvector-graph): inline embed() + tri-modal BM25/ANN/graph hybrid (ADR-252 P3/P4) P3 - inline embedding (HelixQL Embed()): - embed.rs: Embedder trait + dependency-free deterministic HashEmbedder (feature-hashing, explicit opt-in, never a silent fallback per ADR-194). - TypedGraph::with_embedder / embed / create_node_from_text (embed-at-insert, dimension-validated) / search_text (embed-at-query). P4 - tri-modal hybrid query: - bm25.rs: self-contained Okapi-BM25 inverted index. - TypedGraph::build_text_index + hybrid_search_text fusing ANN vector + BM25 keyword + graph traversal via reciprocal rank fusion in one typed call. - Refactored search_then_traverse into shared rank_seeds/expand helpers. Bench: hash_embed_256 717ns; tri_modal_hybrid over 10k docs (embed+HNSW+BM25+ RRF+traverse) 1.63ms end-to-end. 164/164 lib tests green (+13), clippy clean. https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * feat(ruvector-graph): schema-driven typed SDK codegen (ADR-252 P6) codegen.rs generates typed client stubs from a GraphSchema: - generate_typescript: interfaces with typed/optional properties (@indexed hints), edge from->to constraints, and a VectorTypes manifest + VectorTypeName. - generate_python: TypedDict classes + VECTOR_TYPES manifest. - generate_rust: serde-ready structs. Deterministic (schema elements sorted) for check-in/diff. Adds *_schemas_sorted accessors to GraphSchema. Closes HelixDB's schema->typed-SDK DX advantage. 168/168 lib tests green (+4), clippy clean. https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * docs(adr): renumber ADR-252 -> ADR-253 (252 taken by FastGRNN training pipeline) ADR-252 was already merged to main as the tiny-dancer FastGRNN training pipeline. Renumber this HelixDB comparison to ADR-253 to resolve the collision. Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: ruv <ruvnet@users.noreply.github.com> * chore: Update NAPI-RS binaries for all platforms Built from commit 1e1740a Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(mcp-brain-server): use native-tls-vendored so it cross-compiles to aarch64 (ruvnet#573) mcp-brain-server pins reqwest with the `native-tls` feature (for TLS close_notify compatibility). On aarch64 cross-builds, native-tls → openssl-sys fails because the target's OpenSSL dev headers aren't present, so the binary can't be built for Raspberry Pi 5. Switch to `native-tls-vendored`, which builds OpenSSL from source for the target — identical TLS behavior (still native-tls, close_notify compat preserved), just statically linked, no host/target OpenSSL headers required. Context: the Cognitum v0 appliance image (cognitum-one/v0-appliance) builds this binary as `ruview-mcp-brain-mini` (the :9876 vector-memory store) from this repo as a submodule. This is the only blocker preventing it from shipping on the Pi 5 (Hailo-8/10H) appliance. * chore: Update NAPI-RS binaries for all platforms Built from commit e346476 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(ruvector-core): data-loss in update_q_value (ruvnet#562) + silent quantization no-op (ruvnet#563) ruvnet#562 (Critical): PolicyMemoryStore::update_q_value deleted the policy and returned Ok(()) without re-inserting — silent data loss for any RL loop. Now it fetches the entry, updates q_value in metadata, and re-inserts (preserving the embedding); returns VectorNotFound on a missing id instead of false success. Adds a regression test. ruvnet#563 (High): DbOptions.quantization was accepted/persisted but never threaded into the index/storage, while the default was Some(Scalar) and docs promised 4x/32x compression. Default flipped to None; VectorDB::new now warns when a non-None quantization is set; enum/field docs state compression is a target "not yet applied", not a guarantee. 228 ruvector-core lib tests pass. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 135a030 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * docs(adr): ADR-254 ruvector-turbovec multi-bit FastScan ANN index (ruvnet#520) Canonical ADR for the 2-4-bit scalar-quantized FastScan search index proposed in ruvnet#520 / PR ruvnet#521. Numbered 254 because the PR drafted it as ADR-194, which collides with the merged ADR-194 (ONNX embedder). Captures the gap, the T1-T6 design, reuse boundary, milestones M1-M5, measured M1 validation, and honest divergences from the TurboQuant paper. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit a7028ef Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * docs(adr): ADR-255 ruvector <-> OIA Model integration (alignment profile) Grounded in a deep-research brief over agenticsorg/OIA-Model v0.1: maps OIA's 10 layers (L0-L9) + 6 spans to ruvector components, decides a non-binding alignment profile (ruvector as an L3 + L5-L8 provider), designates the RVF cognitive container as the L8 artifact and the witness chain as the SPAN-AUD/PRV primitive, and explicitly scopes out L0/L1/L9/L4-pretraining + the GCP-portability gap. Stays doc/tag-level — no OIA dependency, no API rename — because OIA is pre-1.0 with no machine-readable conformance. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 183ed4a Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(security): SECURITY.md disclosure policy (ruvnet#320) + CORS allowlist (ruvnet#560) (ruvnet#577) - Add SECURITY.md: private disclosure via GitHub PVR or ruv@ruv.net, scope, and response SLAs. Closes the responsible-disclosure gap raised in ruvnet#320 (gives reporters a channel without enabling beg-bounty noise). - mcp-brain-server CORS: add https://app.conceptmapping.org and https://conceptmapping.org to the default allowlist so pi.ruv.io/v1/* returns Access-Control-Allow-Origin for those browser origins (ruvnet#560). Kept an explicit per-origin allowlist (not `*`) since callers authenticate with Bearer tokens. cargo check -p mcp-brain-server: clean. Refs ruvnet#320 ruvnet#560 Co-authored-by: ruv <ruvnet@users.noreply.github.com> * ADR-256: harness router surface (borrow metaharness concepts) (ruvnet#575) * feat(ruvector): ADR-256 harness router surface + tracking (ruvnet#574) Borrow metaharness concepts using primitives ruvector already ships. - Add `ruvector harness status [--json]` — unified read-only view of the routing surface (Tiny Dancer cost router + semantic router + hooks routing + MCP + witness + memory), degrading gracefully when optional deps are absent. Implements ADR-256 rollout step 0. - Add ADR-256 (borrow-concepts decision, concept→primitive mapping). - Add CLI tests (Section 24): harness --help, status --json structure, bare-command behavior. Full suite: 72 passed, 0 failed. Refs ruvnet#574 Co-Authored-By: claude-flow <ruv@ruv.net> * feat(ruvector): ADR-256 default-deny MCP tool-access policy (ruvnet#574) Borrow metaharness's default-deny allowlist concept with our own machinery. - New pure, testable bin/mcp-policy.js: RUVECTOR_MCP_ALLOW / RUVECTOR_MCP_DENY / RUVECTOR_MCP_PROFILE=readonly. Precedence DENY > ALLOW/PROFILE > allow-all. No policy set = backward-compatible allow-all (policy.configured=false). - Wire into mcp-server.js: ListTools now returns only permitted tools; CallTool gates denied tools with an isError response before dispatch. - harness status --json now reports mcp.policy + accessControl posture. - Tests: test/mcp-policy.js (8 unit tests) wired into npm test; verified end-to-end over MCP stdio (readonly profile exposes 10 safe tools, filters hooks_force_learn). CLI suite still 72/0. Refs ruvnet#574 Co-Authored-By: claude-flow <ruv@ruv.net> * test(ruvector): ADR-256 startup-budget guard + harness/MCP-policy docs (ruvnet#574) - New test/startup-budget.js wired into npm test: absolute ceiling on `--help` cold start + relative delta guard ensuring `harness status` adds < 120ms over baseline (catches a heavy module leaking into the startup path). Measured here: --help 127ms, harness +3ms. Env-overridable. - README: document the default-deny MCP policy env vars (RUVECTOR_MCP_ALLOW/DENY/PROFILE) and the `harness` router command. Refs ruvnet#574 Co-Authored-By: claude-flow <ruv@ruv.net> * feat(ruvector): ADR-256 memory namespace + full verification, ADR status (ruvnet#574) - harness surface reports a stable memory namespace (RUVECTOR_MEMORY_NAMESPACE, default `ruvector`); CLI tests assert the default + override and the MCP accessControl/policy fields. - README documents the memory namespace. - ADR-256: add "Implementation status (as shipped)" — items 0/1/3/4 done, benchmarked + full npm test green; item 2 as a documented convention; item 5 deferred. No @metaharness/* runtime dep. Full suite: cli 73/0, mcp-policy 8/0, startup-budget 2/0, db-workflow/integration/sigterm green. Refs ruvnet#574 Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: ruv <ruvnet@users.noreply.github.com> * chore(ruvector): bump to 0.2.32 (ADR-256 release prep) (ruvnet#578) * chore(ruvector): bump to 0.2.32 (harness router + default-deny MCP policy) Release prep for the ADR-256 work merged in ruvnet#575: - `ruvector harness status [--json]` unified routing surface - default-deny MCP tool policy (RUVECTOR_MCP_ALLOW/DENY/PROFILE) - stable memory namespace (RUVECTOR_MEMORY_NAMESPACE) - startup-budget CI guard Additive / non-breaking → patch bump. Tests green (73/0, 8/0, 2/0). Does NOT publish — `npm publish` left to a maintainer with npm auth. Co-Authored-By: claude-flow <ruv@ruv.net> * fix(ruvector): cross-platform build script (node fs.cpSync) The build copied ONNX assets with `mkdir -p`/`cp -r`, which fail on Windows cmd.exe and broke `npm publish` (prepack/prepublishOnly) off-Linux. Replace with node fs.cpSync(recursive) — works everywhere. Verified: build + verify-dist green on Windows (14 dist paths present). Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: ruv <ruvnet@users.noreply.github.com> * chore: Update NAPI-RS binaries for all platforms Built from commit 716dbed Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore: Update NAPI-RS binaries for all platforms Built from commit a2c5ce4 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(wasm): unblock wasm bridge builds (getrandom/uuid js, wasm-opt=false) Apply RuVector-canonical wasm fixes (verified vs passing -wasm crates): - wasm-opt = false for bulk-memory validator failures (delta, fpga-transformer, graph-condense, mincut-gated-transformer, solver, sparse-inference) - getrandom { features=[js] } + uuid { features=[js] } for wasm32 (router, tiny-dancer) Result: 24 -> 29 of 35 -wasm crates build. Remaining 6 have distinct deeper causes (sqlite-C not wasm-able, arithmetic_overflow lint, non-member crates). Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: rUv <ruv@ruv.net> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: ruv <ruvnet@users.noreply.github.com> Co-authored-by: Dragan Spiridonov <spiridonovdragan@gmail.com>

@indexed

…ing (33/35) (#4) * fix(ruvector-wasm): correct adapter for WASM build's flat-index, distance-score, and metadata gaps (ruvnet#568) The published @ruvector/wasm build behaves differently from its generated .d.ts in three ways that bite consumers: 1. HNSW is not active — the wasm32 target compiles without the `hnsw` feature and falls back to a flat (brute-force) index, so search is O(n). The O(log n) win is latent until the WASM HNSW lands. 2. `result.score` is a cosine distance (lower is better), not the "higher is better" similarity the .d.ts advertises (ordering is correct: a, b before c). 3. Metadata does not round-trip — search/get return {}. Add RuvectorWasmAdapter (@ruvector/wasm/adapter) which wraps VectorDB with: - a metadata sidecar so inserted metadata round-trips - similarity = 1 - distance (generalised per metric) with `.score` aliased to similarity, plus the raw `distance` preserved - indexType/usesHnsw + WASM_HNSW_AVAILABLE so callers don't assume HNSW - client-side metadata filtering with over-fetch Includes TS declarations with corrected doc comments, a node:test suite covering all three findings, README guidance, and package exports. Co-authored-by: Claude <noreply@anthropic.com> * chore: Update NAPI-RS binaries for all platforms Built from commit 08c0d74 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore(ruvector-wasm): publish @ruvector/wasm 0.1.31 with corrected adapter Ships the RuvectorWasmAdapter (ruvnet#568) and restores a functional package — 0.1.30 published with only package.json (empty pkg/). 0.1.31 includes the built web pkg/ plus the adapter that corrects similarity score, metadata round-trip, and flat-index reporting. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 054d815 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(tiny-dancer): lock-step platform versioning + drop dead release profile Root cause of version drift: build-tiny-dancer.yml hard-coded VERSION="0.1.15", so every publish shipped stale platform binaries while the main package advanced (npm 0.1.18 was loading 0.1.15-era .node files). - workflow: derive VERSION from package.json; rewrite main optionalDependencies to pin that exact version before publishing, so binaries and JS can never skew. - package.json: bump to 0.1.19, pin all 5 optionalDependencies to 0.1.19, remove dead `publish:platforms` script (scripts/publish-platforms.js absent). - crate Cargo.toml: remove [profile.release] (Cargo ignores non-root profiles; release opt is already opt-level=3/lto=fat/codegen-units=1/strip at root). Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 57187b0 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * feat(tiny-dancer): full platform matrix — add musl (Alpine) + Windows ARM64 Extends the build/publish workflow from 5 to 8 environments: - linux-x64-musl, linux-arm64-musl (Alpine/Docker) via napi --use-napi-cross - win32-arm64-msvc (Windows on ARM) via rustup target cross-compile index.js now detects musl vs glibc at load time (process.report glibc header) and routes to the correct binary; adds win32-arm64. package.json bumped to 0.1.20 with all 8 optionalDependencies lock-stepped. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 1ef4eb8 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(tiny-dancer): use --zig + setup-zig for musl cross-builds (napi 2.18 has no --use-napi-cross) Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 4c448f0 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * feat(tiny-dancer): real FastGRNN training pipeline (ADR-252) Closes the three gaps that made tiny-dancer inference-only: 1. Real gradients: FastGRNN::forward_cached + backward implement single-step analytic backprop (h0=0); gradient-checked vs central finite differences. 2. Real Adam step: train_batch accumulates mean batch gradients; apply_gradients does L2 + global-norm clip + bias-corrected Adam update on the existing optimizer state. Model now actually learns (test: loss down, acc>0.9). 3. safetensors persistence: model.rs save/load serialize every tensor (f32 LE) with config in __metadata__; round-trip is bit-exact. 4. DRACO adapter: TrainingDataset::from_draco consumes the {embedding, scores} + prices shape (same as @metaharness/router) so one dataset trains both. Runnable example train_from_draco demonstrates DRACO -> train -> save -> load -> route end to end. 31 core tests green (gradient check, convergence, round-trip, adapter). Co-Authored-By: claude-flow <ruv@ruv.net> * feat(tiny-dancer-node): napi trainRouter export (ADR-252 B) Exposes the FastGRNN training pipeline to JS: trainRouter(rows, prices, opts) consumes DRACO {embedding, scores} rows, trains, and writes a .safetensors the existing `new Router({ modelPath })` loads. Type-checks; reaches JS on the next tiny-dancer platform rebuild/republish. Co-Authored-By: claude-flow <ruv@ruv.net> * release(tiny-dancer): 0.1.21 — expose trainRouter to JS (types + bump) Adds trainRouter/DracoRowJs/TrainRouterOptions/TrainRouterResult to the hand-written index.d.ts and bumps to 0.1.21 (8 optionalDependencies lock-stepped) so the native train_router export ships typed across all environments. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit e709718 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore: Update NAPI-RS binaries for all platforms Built from commit 1aa132b Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore: Update NAPI-RS binaries for all platforms Built from commit c520576 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * feat(ruvector-cli): integrate tiny-dancer router (train/route/info) npx ruvector tiny-dancer train <draco.json> --out model.safetensors trains a FastGRNN cost-optimal router from a DRACO {embedding,scores} dataset; `route` loads + routes; `info` reports availability. @ruvector/tiny-dancer added as an optionalDependency (native binary, lazy-required with install hint). Bump 0.2.31. Co-Authored-By: claude-flow <ruv@ruv.net> * feat(tiny-dancer): add score() raw-forward inference + 0.1.22 score(modelPath, embedding) loads a trainRouter .safetensors and runs the FastGRNN directly on the raw embedding (the inference path that matches how trainRouter trains — Router.route's 5-feature engineering is a different model contract). Validated easy=1.0/hard=0.0 on a freshly built binary. Co-Authored-By: claude-flow <ruv@ruv.net> * fix(ruvector-cli): tiny-dancer score command (raw forward) + dep ^0.1.22 Replaces the route subcommand (which used Router.route's 5-feature engineering — a different model contract) with 'score', backed by tiny-dancer's score() raw forward pass that matches trainRouter models. Also fixes the earlier Float32Array issue by removing the Router.route path entirely. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 39fb398 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore: Update NAPI-RS binaries for all platforms Built from commit 5173ce7 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore: Update NAPI-RS binaries for all platforms Built from commit 3c1f701 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * docs(adr): ADR-252 HelixDB vs RuVector comparison and improvement opportunities (ruvnet#570) * docs(adr): ADR-252 HelixDB vs RuVector comparison and improvement opportunities Compares HelixDB (LMDB/heed, compiled type-safe HelixQL, graph-vector thesis, graph-vector-bench) against RuVector's redb/Cypher/hybrid stack and proposes 7 prioritized, opt-in improvements: optional schema layer with load-time validation, first-class typed graph-vector binding and a unified search-then-traverse operator, in-query embed(), unified ANN+BM25+graph RRF hybrid, a reproducible benchmark harness, schema-driven typed SDK codegen, and an object-storage tier research spike. https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * feat(ruvector-graph): native schema layer + typed search-then-traverse (ADR-252 P1/P2/P4) Implements the HelixDB-inspired improvements natively in ruvector-graph: - schema.rs: opt-in GraphSchema (N::/E::/V:: equivalents) with load-time validation (self-consistency, node required/typed props + strict mode, edge from/to label constraints, vector dimension checks), higher-is-better distance metrics (cosine/dot/euclidean), and reciprocal_rank_fusion (P4). - typed_graph.rs: TypedGraph wrapper validating mutations pre-storage, plus a fused typed search_then_traverse operator (HelixQL SearchV<T>(q,k)::In/Out<E>) with optimized bounded-heap top-k selection (O(n log k)). Pure-Rust, no new deps, WASM-safe. 13 new tests, 148/148 lib tests green, clippy clean. Schemaless mode remains the default (opt-in coexistence). https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * perf(ruvector-graph): optimize search_then_traverse + add criterion bench (ADR-252) Hot-path optimizations for the typed search-then-traverse operator: - GraphDB::with_node / node_ids_by_label: zero-copy borrow scoring, eliminating per-candidate Node + embedding clones (get_nodes_by_label cloned everything). - Fused single-pass cosine (q.c and c.c in one read of the candidate) + hoisted query norm out of the per-candidate loop. - Bounded top-k min-heap (O(n log k)); clone id only for heap winners. - Rayon parallel scan over DashMap for >=4096 candidates (per-thread heaps, bounded merge); serial path below threshold. Adds benches/typed_graph_bench.rs (criterion). Measured vs first cut (128-dim, k=10): 10k 7.2ms->3.08ms (2.34x), 50k 74.3ms->28.5ms (2.61x), 1k 539us->432us. New parallel-vs-reference correctness test. 149/149 lib tests green, clippy clean. https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * feat(ruvector-graph): HNSW push-down for search_then_traverse (ADR-252 P2) Adds an opt-in ANN path to the typed search-then-traverse operator, removing the O(n) full-label scan for indexed vector types: - TypedGraph::build_vector_index(vector_type) builds a per-vector-type HybridIndex (HNSW under hnsw_rs, exact FlatIndex otherwise), holding only the bound label's nodes so searches stay label-scoped. Kept current incrementally via create_node -> index_node. - search_then_traverse routes through the index when present: ~O(log n) approximate search, over-fetch (max(4k, k+32)), then exact rescore with the schema metric so ANN results carry identical higher-is-better score semantics to the brute-force path. Brute force remains the default. - Parallel brute-force path refactored to capture &GraphDB (not &self) so it stays Send+Sync independent of the index's thread-safety bounds. Bench (50k nodes, 128-dim, k=10): brute-force parallel scan 27.6ms -> HNSW push-down 1.05ms (~26x; ~70x vs first cut). 151/151 lib tests green (3 new HNSW tests), clippy clean. https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * feat(ruvector-graph): inline embed() + tri-modal BM25/ANN/graph hybrid (ADR-252 P3/P4) P3 - inline embedding (HelixQL Embed()): - embed.rs: Embedder trait + dependency-free deterministic HashEmbedder (feature-hashing, explicit opt-in, never a silent fallback per ADR-194). - TypedGraph::with_embedder / embed / create_node_from_text (embed-at-insert, dimension-validated) / search_text (embed-at-query). P4 - tri-modal hybrid query: - bm25.rs: self-contained Okapi-BM25 inverted index. - TypedGraph::build_text_index + hybrid_search_text fusing ANN vector + BM25 keyword + graph traversal via reciprocal rank fusion in one typed call. - Refactored search_then_traverse into shared rank_seeds/expand helpers. Bench: hash_embed_256 717ns; tri_modal_hybrid over 10k docs (embed+HNSW+BM25+ RRF+traverse) 1.63ms end-to-end. 164/164 lib tests green (+13), clippy clean. https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * feat(ruvector-graph): schema-driven typed SDK codegen (ADR-252 P6) codegen.rs generates typed client stubs from a GraphSchema: - generate_typescript: interfaces with typed/optional properties (@indexed hints), edge from->to constraints, and a VectorTypes manifest + VectorTypeName. - generate_python: TypedDict classes + VECTOR_TYPES manifest. - generate_rust: serde-ready structs. Deterministic (schema elements sorted) for check-in/diff. Adds *_schemas_sorted accessors to GraphSchema. Closes HelixDB's schema->typed-SDK DX advantage. 168/168 lib tests green (+4), clippy clean. https://claude.ai/code/session_01BrEtcS3KZykinsv9RoBGrF * docs(adr): renumber ADR-252 -> ADR-253 (252 taken by FastGRNN training pipeline) ADR-252 was already merged to main as the tiny-dancer FastGRNN training pipeline. Renumber this HelixDB comparison to ADR-253 to resolve the collision. Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: ruv <ruvnet@users.noreply.github.com> * chore: Update NAPI-RS binaries for all platforms Built from commit 1e1740a Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(mcp-brain-server): use native-tls-vendored so it cross-compiles to aarch64 (ruvnet#573) mcp-brain-server pins reqwest with the `native-tls` feature (for TLS close_notify compatibility). On aarch64 cross-builds, native-tls → openssl-sys fails because the target's OpenSSL dev headers aren't present, so the binary can't be built for Raspberry Pi 5. Switch to `native-tls-vendored`, which builds OpenSSL from source for the target — identical TLS behavior (still native-tls, close_notify compat preserved), just statically linked, no host/target OpenSSL headers required. Context: the Cognitum v0 appliance image (cognitum-one/v0-appliance) builds this binary as `ruview-mcp-brain-mini` (the :9876 vector-memory store) from this repo as a submodule. This is the only blocker preventing it from shipping on the Pi 5 (Hailo-8/10H) appliance. * chore: Update NAPI-RS binaries for all platforms Built from commit e346476 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(ruvector-core): data-loss in update_q_value (ruvnet#562) + silent quantization no-op (ruvnet#563) ruvnet#562 (Critical): PolicyMemoryStore::update_q_value deleted the policy and returned Ok(()) without re-inserting — silent data loss for any RL loop. Now it fetches the entry, updates q_value in metadata, and re-inserts (preserving the embedding); returns VectorNotFound on a missing id instead of false success. Adds a regression test. ruvnet#563 (High): DbOptions.quantization was accepted/persisted but never threaded into the index/storage, while the default was Some(Scalar) and docs promised 4x/32x compression. Default flipped to None; VectorDB::new now warns when a non-None quantization is set; enum/field docs state compression is a target "not yet applied", not a guarantee. 228 ruvector-core lib tests pass. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 135a030 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * docs(adr): ADR-254 ruvector-turbovec multi-bit FastScan ANN index (ruvnet#520) Canonical ADR for the 2-4-bit scalar-quantized FastScan search index proposed in ruvnet#520 / PR ruvnet#521. Numbered 254 because the PR drafted it as ADR-194, which collides with the merged ADR-194 (ONNX embedder). Captures the gap, the T1-T6 design, reuse boundary, milestones M1-M5, measured M1 validation, and honest divergences from the TurboQuant paper. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit a7028ef Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * docs(adr): ADR-255 ruvector <-> OIA Model integration (alignment profile) Grounded in a deep-research brief over agenticsorg/OIA-Model v0.1: maps OIA's 10 layers (L0-L9) + 6 spans to ruvector components, decides a non-binding alignment profile (ruvector as an L3 + L5-L8 provider), designates the RVF cognitive container as the L8 artifact and the witness chain as the SPAN-AUD/PRV primitive, and explicitly scopes out L0/L1/L9/L4-pretraining + the GCP-portability gap. Stays doc/tag-level — no OIA dependency, no API rename — because OIA is pre-1.0 with no machine-readable conformance. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: Update NAPI-RS binaries for all platforms Built from commit 183ed4a Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(security): SECURITY.md disclosure policy (ruvnet#320) + CORS allowlist (ruvnet#560) (ruvnet#577) - Add SECURITY.md: private disclosure via GitHub PVR or ruv@ruv.net, scope, and response SLAs. Closes the responsible-disclosure gap raised in ruvnet#320 (gives reporters a channel without enabling beg-bounty noise). - mcp-brain-server CORS: add https://app.conceptmapping.org and https://conceptmapping.org to the default allowlist so pi.ruv.io/v1/* returns Access-Control-Allow-Origin for those browser origins (ruvnet#560). Kept an explicit per-origin allowlist (not `*`) since callers authenticate with Bearer tokens. cargo check -p mcp-brain-server: clean. Refs ruvnet#320 ruvnet#560 Co-authored-by: ruv <ruvnet@users.noreply.github.com> * ADR-256: harness router surface (borrow metaharness concepts) (ruvnet#575) * feat(ruvector): ADR-256 harness router surface + tracking (ruvnet#574) Borrow metaharness concepts using primitives ruvector already ships. - Add `ruvector harness status [--json]` — unified read-only view of the routing surface (Tiny Dancer cost router + semantic router + hooks routing + MCP + witness + memory), degrading gracefully when optional deps are absent. Implements ADR-256 rollout step 0. - Add ADR-256 (borrow-concepts decision, concept→primitive mapping). - Add CLI tests (Section 24): harness --help, status --json structure, bare-command behavior. Full suite: 72 passed, 0 failed. Refs ruvnet#574 Co-Authored-By: claude-flow <ruv@ruv.net> * feat(ruvector): ADR-256 default-deny MCP tool-access policy (ruvnet#574) Borrow metaharness's default-deny allowlist concept with our own machinery. - New pure, testable bin/mcp-policy.js: RUVECTOR_MCP_ALLOW / RUVECTOR_MCP_DENY / RUVECTOR_MCP_PROFILE=readonly. Precedence DENY > ALLOW/PROFILE > allow-all. No policy set = backward-compatible allow-all (policy.configured=false). - Wire into mcp-server.js: ListTools now returns only permitted tools; CallTool gates denied tools with an isError response before dispatch. - harness status --json now reports mcp.policy + accessControl posture. - Tests: test/mcp-policy.js (8 unit tests) wired into npm test; verified end-to-end over MCP stdio (readonly profile exposes 10 safe tools, filters hooks_force_learn). CLI suite still 72/0. Refs ruvnet#574 Co-Authored-By: claude-flow <ruv@ruv.net> * test(ruvector): ADR-256 startup-budget guard + harness/MCP-policy docs (ruvnet#574) - New test/startup-budget.js wired into npm test: absolute ceiling on `--help` cold start + relative delta guard ensuring `harness status` adds < 120ms over baseline (catches a heavy module leaking into the startup path). Measured here: --help 127ms, harness +3ms. Env-overridable. - README: document the default-deny MCP policy env vars (RUVECTOR_MCP_ALLOW/DENY/PROFILE) and the `harness` router command. Refs ruvnet#574 Co-Authored-By: claude-flow <ruv@ruv.net> * feat(ruvector): ADR-256 memory namespace + full verification, ADR status (ruvnet#574) - harness surface reports a stable memory namespace (RUVECTOR_MEMORY_NAMESPACE, default `ruvector`); CLI tests assert the default + override and the MCP accessControl/policy fields. - README documents the memory namespace. - ADR-256: add "Implementation status (as shipped)" — items 0/1/3/4 done, benchmarked + full npm test green; item 2 as a documented convention; item 5 deferred. No @metaharness/* runtime dep. Full suite: cli 73/0, mcp-policy 8/0, startup-budget 2/0, db-workflow/integration/sigterm green. Refs ruvnet#574 Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: ruv <ruvnet@users.noreply.github.com> * chore(ruvector): bump to 0.2.32 (ADR-256 release prep) (ruvnet#578) * chore(ruvector): bump to 0.2.32 (harness router + default-deny MCP policy) Release prep for the ADR-256 work merged in ruvnet#575: - `ruvector harness status [--json]` unified routing surface - default-deny MCP tool policy (RUVECTOR_MCP_ALLOW/DENY/PROFILE) - stable memory namespace (RUVECTOR_MEMORY_NAMESPACE) - startup-budget CI guard Additive / non-breaking → patch bump. Tests green (73/0, 8/0, 2/0). Does NOT publish — `npm publish` left to a maintainer with npm auth. Co-Authored-By: claude-flow <ruv@ruv.net> * fix(ruvector): cross-platform build script (node fs.cpSync) The build copied ONNX assets with `mkdir -p`/`cp -r`, which fail on Windows cmd.exe and broke `npm publish` (prepack/prepublishOnly) off-Linux. Replace with node fs.cpSync(recursive) — works everywhere. Verified: build + verify-dist green on Windows (14 dist paths present). Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: ruv <ruvnet@users.noreply.github.com> * chore: Update NAPI-RS binaries for all platforms Built from commit 716dbed Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * chore: Update NAPI-RS binaries for all platforms Built from commit a2c5ce4 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions * fix(wasm): unblock wasm bridge builds (getrandom/uuid js, wasm-opt=false) Apply RuVector-canonical wasm fixes (verified vs passing -wasm crates): - wasm-opt = false for bulk-memory validator failures (delta, fpga-transformer, graph-condense, mincut-gated-transformer, solver, sparse-inference) - getrandom { features=[js] } + uuid { features=[js] } for wasm32 (router, tiny-dancer) Result: 24 -> 29 of 35 -wasm crates build. Remaining 6 have distinct deeper causes (sqlite-C not wasm-able, arithmetic_overflow lint, non-member crates). Co-Authored-By: claude-flow <ruv@ruv.net> * fix(wasm): round 2 — mincut overflow, sqlite/simsimd gating, more js/wasm-opt (29→33/35) - mincut-gated-transformer: 16GB usize literal overflowed wasm32 32-bit usize -> saturating_mul (16GB on 64-bit, caps at usize::MAX on wasm32). PRIMARY crate. - domain-expansion-wasm: getrandom js feature - temporal-tensor-wasm: add to workspace.members + wasm-opt=false - tiny-dancer-core: cfg(not(wasm32))-gate native-only C deps — rusqlite (bundled SQLite) + simsimd (C SIMD); add scalar cosine fallback for wasm32. Native unchanged. Result: 33/35 -wasm crates build. Remaining 2: sparse-inference-wasm (lib.rs API drift vs SparseModel — needs wrapper update), micro-hnsw-wasm (excluded stub). Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: rUv <ruv@ruv.net> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: ruv <ruvnet@users.noreply.github.com> Co-authored-by: Dragan Spiridonov <spiridonovdragan@gmail.com>

claude and others added 5 commits May 29, 2026 23:35

chore: update Cargo.lock for ruvector-turbovec

6b4c7ff

https://claude.ai/code/session_012AzArCzBwxrJp8mUngUcH5

This was referenced May 30, 2026

Missing ANN regime: 2–4-bit scalar-quantized FastScan search index #520

Open

feat(turbovec): multi-bit TurboQuant FastScan ANN index (ADR-194 M1) shaal/ruvector#2

Closed

shaal added 4 commits May 29, 2026 23:44

style(turbovec): cargo fmt — wrap pre-existing >100col lines

2935381

Pure rustfmt normalization (rustfmt 1.8.0-stable) of lines that predated this branch; no semantic change. Isolated from the feature commits so the crate passes 'cargo fmt --check' cleanly for upstream CI.

shaal added 2 commits June 17, 2026 12:12

Merge remote-tracking branch 'origin/main' into claude/ruvector-turbo…

5359b2c

…vec-optimization-FhaDh # Conflicts: # docs/adr/ADR-254-ruvector-turbovec-fastscan-index.md

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(turbovec): multi-bit TurboQuant FastScan ANN index (ADR-194 M1)#521

feat(turbovec): multi-bit TurboQuant FastScan ANN index (ADR-194 M1)#521
shaal wants to merge 11 commits into
ruvnet:mainfrom
shaal:claude/ruvector-turbovec-optimization-FhaDh

shaal commented May 30, 2026 •

edited

Loading

Uh oh!

ruvnet commented Jun 16, 2026

Uh oh!

shaal commented Jun 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

shaal commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What & why

The gap this fills

Why it's a good fit for this repo

Pipeline (ADR-194 §T1–T4, T6)

Measured proof — cargo run --release -p ruvector-turbovec

Tests / quality

Post-review hardening (commits 5–9)

Scope / follow-ups

Files

Uh oh!

ruvnet commented Jun 16, 2026

Uh oh!

shaal commented Jun 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shaal commented May 30, 2026 •

edited

Loading

Measured proof — `cargo run --release -p ruvector-turbovec`