Productionize #506 proof-gated writes (Merkle WAL, tamper-evident) — implement/test/bench/secure/optimize/publish

[ruvnet]

Productionize #506 — proof-gated writes (ruvector-proof-gate)

Tracking issue for productionizing the #506 novel finding: a Merkle-accumulating write-ahead log for tamper-evident vector writes — cryptographic evidence of what was stored, when, and by whom. Motivated by the MemoryGraft attack (arxiv 2512.16962) and the Mnemonic Sovereignty survey: every major vector DB (Qdrant/Milvus/Weaviate/LanceDB/FAISS) accepts writes with zero integrity evidence. Fits ruvector's witness-chain DNA — a potential differentiator.

Driven by the 5-minute loop; one step per iteration, each with a progress reply + gist.

Steps

• Implement / integrate — crate ruvector-proof-gate brought into the workspace (branch feat/productionize-proof-gate). 16/16 tests pass.
• Test — strengthen tamper-evidence tests: verify the Merkle root detects any mutation/insertion/deletion/reorder; round-trip proof verification.
• Benchmark — write throughput + proof-gen/verify latency vs unguarded baseline (the integrity tax).
• Secure — adversarial: forged proofs rejected, second-preimage/length-extension resistance, no panics on malformed input.
• Optimize — Merkle path / hashing hot path.
• Publish — public API + release PR.

Status

Iteration 1 done: crate integrated, 16/16 tests green, branch pushed.

Related: #506 (source PR), ADR-194-proof-gated-writes. Loop job 05bbd4fb.

[ruvnet]

🔁 Loop iteration 2 — Test step ✅ (tamper-evidence proven)

• Test — tests/tamper_evidence.rs (5 tests) proves the core property end-to-end via the public API.

The chain root is a cryptographic commitment to the entire ordered write log:

• any mutation of a write → different root ✓
• insertion / deletion / reorder → different root ✓
• genuine receipts verify; forged commitment, out-of-range, and foreign-chain receipts are rejected (no panic) ✓
• deterministic: same log → same root ✓

Finding for the Secure step: verify_integrity() is currently only a structural check (commitments non-zero + differ from neighbours) — it does not re-derive commitments from stored payload hashes, so it can't catch a tamper that preserves structural validity. The secure step will strengthen it to a full cryptographic re-derivation.

Next (iteration 3): benchmark the integrity tax (write throughput + proof gen/verify vs NullGate baseline).

[ruvnet]

🔁 Loop iteration 3 — Benchmark step ✅ (the integrity tax)

• Benchmark — tests/perf_benchmark.rs (release, #[ignore]) measures the cost of tamper-evidence.

op	latency	throughput
NullGate.admit (no integrity)	35.8 ns	27.9 M/s
HashChainGate.admit (tamper-evident)	~1026 ns	~1.0 M/s
verify_receipt	~6.4 ns	157 M/s

Integrity tax ≈ 991 ns/write (~2 SHA-256 over the payload), and verification is effectively free (~6 ns, O(1)). At ~1 M tamper-evident writes/sec on one core, the tax is negligible next to the HNSW insertion a real write already does — a strong case for shipping it on by default. Regression budget set at 5000 ns/write.

Next (iteration 4): secure — close the verify_integrity() gap (re-derive commitments from stored payload hashes) + adversarial hardening.

[ruvnet]

🔁 Loop iteration 4 — Secure step ✅ (full re-derivation)

• Secure — closed the verify_integrity() gap flagged in the test step.

It was only a structural scan (commitments non-zero + differ from neighbours). Now the gate stores per-entry payload hashes and verify_integrity() re-derives every commitment from the genesis seed, comparing against the stored chain. A tamper that mutates a commitment, a payload hash, reorders entries, or desyncs lengths is now caught — not just degenerate chains.

+5 private-field tamper unit tests (commitment flip, payload-hash flip, reorder, length-mismatch, clean re-verify). All proof-gate tests green (20 unit + 5 tamper-evidence).

📓 All step writeups now live in one consolidated gist (per request): https://gist.github.com/ruvnet/7876f135f2032f97213b1a5e4988c620

Next (iteration 5): optimize the hashing hot path, then publish.

[ruvnet]

🔁 Loop iteration 5 — Optimize step ✅ (allocation-free hashing)

• Optimize — admit() built canonical_bytes() (a Vec + per-element extend) before hashing. Added WritePayload::payload_hash() that streams the same fields straight into SHA-256 — identical digest, no intermediate allocation.

	admit latency	throughput	integrity tax
before	~1026 ns/write	0.97 M/s	~991 ns
after	~703 ns/write (~31% faster)	1.42 M/s	~675 ns

Digests unchanged → all 20 unit + 5 tamper-evidence tests stay green (backward-compatible). Also applied to the MerkleGate path.

Next (iteration 6): publish — release PR for ruvector-proof-gate. (Single gist updated.)

[ruvnet]

🔁 Loop iteration 6 — Publish step ✅ — #506 productionization COMPLETE

• Publish — PR feat(proof-gate): productionize #506 — tamper-evident vector writes (Merkle/hash-chain WAL) #584 opens ruvector-proof-gate onto main.

All 6 steps done:

• Implement · [x] Test (tamper-evidence) · [x] Benchmark (~675 ns/write tax) · [x] Secure (full re-derivation) · [x] Optimize (~31% faster, digests identical) · [x] Publish (PR feat(proof-gate): productionize #506 — tamper-evident vector writes (Merkle/hash-chain WAL) #584)

cargo test green throughout (20 unit + 5 tamper-evidence). Single live-log gist: https://gist.github.com/ruvnet/7876f135f2032f97213b1a5e4988c620

Both priority targets are now productionized: #479 → PR #582, #506 → PR #584. The loop advances to #537 (reuse-under-drift) and #542 (PQ/IVFADC pruning).