Add projection-v1 backend with batch proof schema v3#59
Open
BidhanRoy wants to merge 4 commits into
Open
Conversation
prove_bytes/verify_bytes re-ran Params::new + keygen on every call; they now share a process-wide cache keyed by everything the circuit structure depends on (k, dims, fixed-point widths, scaling). Legacy verification also rejects oversized statements (dim/rank/k caps) before any keygen work so a hostile artifact cannot stall the verifier. https://claude.ai/code/session_01AY5wjgZXzcPpxgNC7UcVR6
One artifact set now covers a contiguous batch of invocation rows under the new pedersen-projection-v1 proof kind. Statements are digest-only (row digests + batch transcript digest; full rows come from the verifier-recorded transcript), bound to a pinned schema-3 manifest carrying hiding adapter commitments with per-write nonces and a one-time A/B range proof. - proof_v3.py: batching (contiguity, uniform-adapter, short final chunks, nonzero starts), statement construction with per-row delta self-checks, verifier-side digest/bound recomputation against proved range bounds, transcript-first coverage engine, expand_statement_rows, and mixed v2/v3 dispatch with no cross-version verification - generate_proofs defaults to the projection backend (env-gated ZKLORA_PROVER_BACKEND=legacy-halo2 rollback hatch); batch_verify_proofs signature unchanged, now routed through the mixed engine - contributor secret seed lives outside the artifact dir by default with guards on both the generation and verification paths - docs/protocol-projection-v1.md: normative protocol note (relation, FS schedule, LFO construction, range linkage lemma, padding soundness, integer-vs-field argument, accounting, comparison appendix); Rust prover work is gated on this note - tests/test_v3_artifacts.py: 15 mock-native tests covering the artifact contract, coverage failures, tamper/replay/reorder, secret guards, the legacy hatch, and bound composition https://claude.ai/code/session_01AY5wjgZXzcPpxgNC7UcVR6
…fixes Review blocking items: - Keep legacy-halo2 as the default prover backend until the native projection prover (prove_v3/verify_v3) ships in M2. projection-v1 is now opt-in via ZKLORA_PROVER_BACKEND; on installs whose native module predates v3 the default path keeps working instead of failing at generation time. Error messages now point at unsetting the variable. Rust (M0 hardening): - Bound the legacy key cache with a small hand-rolled LRU (cap 4): the cache key spans attacker-influenceable statement fields, so the previous unbounded map was a slow memory DoS. Params now cached per-k in a separate bounded LRU and shared across shapes instead of duplicated per entry. - verify_bytes builds vk-only cache entries; keygen_pk runs only when a shape is actually proven (entries upgrade in place on first prove). - Mutex poisoning recovers instead of panicking through PyO3. Python: - Bump requires-python to >=3.10: lora_contributor_mpi already uses PEP 604 unions evaluated at import time (predates this branch) and the new code uses Path.is_relative_to (3.9+); CI runs 3.11. - statement_from_batch rejects per-row input/output shapes that do not match the statement's flat [in_dim]/[out_dim] form, so a drifting capture path fails at generation time instead of at verification. - Hostile-artifact robustness: structural surprises in schema-3 statements and expected-adapter entries (missing keys, wrong JSON types) surface as ProofContractError instead of raw KeyError/TypeError; genuine contract errors pass through unwrapped. - Contributor-secret creation race (O_EXCL) now returns the winner's seed instead of leaking FileExistsError. - CoverageClaim.source is used in duplicate-coverage diagnostics; lora_contributor_mpi imports the now-public prover_backend(). Docs: fix MAX_V3_ROWS -> MAX_BATCH_ROWS naming drift in the protocol note. https://claude.ai/code/session_01JN4dDgBGjVsuh31NHCPV4P
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR lands the schema v3 artifact layer and protocol specification for the upcoming
pedersen-projection-v1backend, plus hardening of the legacy Halo2 backend (M0/M1 of the v3 plan). Statements move from one-proof-per-row to batched proofs over contiguous invocation rows.What is NOT in this PR: the native projection prover itself. The four native functions the Python layer dispatches to (
prove_v3,verify_v3,adapter_commit_v3,verify_adapter_manifest_v3) are not implemented in the Rust crate on this branch — they ship in the next milestone (M2). Until then the projection backend is opt-in and fails with a clear error on real installs; tests exercise the artifact layer through a deterministic fake (FakeNativeBothintests/test_v3_artifacts.py).Key Changes
New Schema v3 Artifact Layer (
src/zklora/proof_v3.py)Backend Selection (
src/zklora/zk_proof_generator.py)ZKLORA_PROVER_BACKENDselects betweenlegacy-halo2(default) andprojection-v1(opt-in)batch_verify_proofs()routes through the mixed verification engine and handles v2 and v3 artifacts side by sideLegacy Rust Hardening — M0 only (
src/src/lib.rs)MAX_LEGACY_K=24,MAX_LEGACY_DIM=16384,MAX_LEGACY_RANK=1024Params::new+keygen_*on every prove/verify; the cache is a small bounded LRU (shapes are attacker-influenceable, so an unbounded map is a slow memory DoS),Paramsare shared per-k, and verification builds vk-only entries sokeygen_pkruns only when a shape is actually provenMPI Contributor Integration (
src/zklora/lora_contributor_mpi/__init__.py)Protocol Documentation (
docs/protocol-projection-v1.md)Test Coverage (
tests/test_v3_artifacts.py)Review status
All feedback from the first review round is addressed in
e9eef0d(now on this branch): description accuracy, legacy-halo2 restored as the default backend until M2, bounded LRU key cache with per-kParamssharing and vk-only verify entries,requires-pythonbumped to>=3.10(the package already used PEP 604 unions evaluated at import time onmain), prover-side per-row shape validation, hostile-artifact structural errors surfaced as contract errors, contributor-secret creation race handled,CoverageClaim.sourceused in diagnostics,prover_backend()made public, and theMAX_V3_ROWS/MAX_BATCH_ROWSdoc drift fixed.Original work: https://claude.ai/code/session_01AY5wjgZXzcPpxgNC7UcVR6
https://claude.ai/code/session_01JN4dDgBGjVsuh31NHCPV4P