Define durable data and future provenance safety gates

README.md

@@ -16,7 +16,7 @@ Current application stack:
 
 - Frontend: Next.js + TypeScript + Tailwind
 - Backend: FastAPI + SQLModel
-- Data: SQLite
+- Data: SQLite for local development and tests; PostgreSQL is required for live user data
 - External food data: Open Food Facts
 - Identity/authentication: server-side identity flow with session cookies
 
@@ -53,7 +53,8 @@ CalorieApp V1 is intentionally centralized and scope-restricted.
 
 1. Next.js frontend provides UI and user interaction flows.
 2. FastAPI backend provides API behavior and business/data logic.
-3. SQLite persists current application data.
+3. SQLite persists local development and test data. Public user onboarding is
+   blocked until the PostgreSQL durable-data release gates pass.
 4. Open Food Facts is used as the external food data source.
 5. Identity/authentication is handled through backend-managed session flow.
 
@@ -71,6 +72,15 @@ CalorieApp V1 is not:
 
 The V1 scope is food and nutrition tracking only.
 
+Infrastructure follows a one-provider-per-role policy. Optional provenance is
+designed for the same PostgreSQL primary store and does not add a graph database,
+blockchain database or IPFS dependency to the core release.
+
+Repeatable tests, schema checks, staging restore drills and future scoped ledger
+verification are automation-ready. Production schema changes, privacy-purpose
+expansion, XRPL enablement, deployment and publication retain explicit approval
+gates.
+
 No wallet custody or financial transaction layer is claimed in V1. See
 [REGULATORY.md](REGULATORY.md) for the MiCA and financial-services boundary.
 
@@ -185,10 +195,15 @@ gate can additionally run the local developer health check.
 ## Documentation
 
 - Versioned Identity Bridge contracts: contracts/identity-bridge/v1/
+- XRPL-linked provenance contract: contracts/provenance/v1/
 - Historical image localization contract: contracts/localization/v1/
 - Public architecture: docs/public/architecture.md
 - Public roadmap: docs/public/roadmap.md
 - Public deployment guide: docs/public/deployment.md
+- Durable data and privacy foundation: docs/DURABLE_DATA_FOUNDATION.md
+- Voluntary XRPL transaction-linking architecture: docs/XRPL_TRANSACTION_LINKING.md
+- Public data-safety direction: docs/public/data-safety.md
+- Public XRPL reference direction: docs/public/xrpl-linking.md
 - Public release readiness checklist: docs/public/release-readiness.md
 - Public identity overview: docs/public/identity.md
 

backend/README.md

@@ -56,5 +56,7 @@ If you see inconsistent API responses:
 
 ## Notes
 
-- Data storage uses local SQLite via SQLModel for MVP persistence.
+- Data storage uses local SQLite via SQLModel for development and tests only.
+- Public user onboarding remains blocked until the durable PostgreSQL,
+  migration, persistence, export, erasure and recovery gates pass.
 - Open Food Facts is consumed only by backend service endpoints.

backend/tests/test_data_safety_contract.py

@@ -0,0 +1,179 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+
+ROOT = Path(__file__).resolve().parents[2]
+CONTRACT_DIR = ROOT / "contracts" / "data-safety" / "v1"
+
+
+def _load_json(name: str) -> dict:
+    return json.loads((CONTRACT_DIR / name).read_text(encoding="utf-8"))
+
+
+def test_data_safety_contract_keeps_live_history_off_sqlite() -> None:
+    contract = _load_json("data-safety.json")
+
+    assert contract["contract_id"] == "calorieapp.durable-data-safety"
+    assert contract["release_state"] == "blocked"
+    assert contract["architecture"]["primary_live_store"] == "postgresql"
+    assert contract["architecture"]["sqlite_allowed_environments"] == ["local", "test"]
+    assert contract["architecture"]["sqlite_allowed_for_public_live_history"] is False
+    assert contract["architecture"]["provider_driven_history_expiry_allowed"] is False
+    assert contract["architecture"]["formal_schema_migrations_required"] is True
+
+
+def test_data_safety_contract_forbids_public_personal_data_replication() -> None:
+    boundary = _load_json("data-safety.json")["decentralized_boundary"]
+
+    assert boundary["personal_data_on_public_blockchain_allowed"] is False
+    assert boundary["personal_data_on_public_ipfs_allowed"] is False
+    assert boundary["public_cid_treated_as_private"] is False
+    assert boundary["current_release_dependency"] is False
+    assert "non-reversible" in boundary["allowed_chain_record"]
+
+
+def test_data_classes_cover_current_and_planned_personal_flows() -> None:
+    data_classes = {
+        item["id"]: item for item in _load_json("data-safety.json")["data_classes"]
+    }
+
+    assert set(data_classes) == {
+        "food_history",
+        "identity_link",
+        "authentication_transient",
+        "food_search_query",
+        "voluntary_profile",
+        "donation_contact",
+        "xrpl_transaction_reference",
+        "calorie_record_fingerprint",
+    }
+    assert data_classes["food_search_query"]["persistent_storage_allowed"] is False
+    assert data_classes["food_search_query"]["calorieapp_identity_forwarded"] is False
+    assert data_classes["voluntary_profile"]["explicit_choice_required"] is True
+    assert data_classes["donation_contact"]["purpose_separation_required"] is True
+    assert data_classes["xrpl_transaction_reference"]["primary_key"] == [
+        "network",
+        "transaction_hash",
+    ]
+    assert (
+        data_classes["calorie_record_fingerprint"][
+            "plain_record_hash_allowed_on_public_ledger"
+        ]
+        is False
+    )
+
+
+def test_xrpl_linking_is_optional_off_chain_and_privacy_preserving() -> None:
+    contract = _load_json("data-safety.json")
+    linking = contract["xrpl_transaction_linking"]
+    memo = linking["memo"]
+
+    assert linking["status"] == "planned-disabled-by-default"
+    assert linking["current_release_dependency"] is False
+    assert linking["initial_user_facing_feature"] is False
+    assert linking["canonical_anchor"] == ["network", "transaction_hash"]
+    assert linking["accepted_ledger_state"] == "validated"
+    assert linking["database_link_is_off_chain"] is True
+    assert linking["link_requires_explicit_user_action"] is True
+    assert linking["automatic_wallet_history_profiling_allowed"] is False
+    assert linking["wallet_key_or_custody_access_allowed"] is False
+    assert memo["personal_data_allowed"] is False
+    assert memo["stable_user_identifier_allowed"] is False
+    assert memo["raw_database_identifier_allowed"] is False
+    assert memo["plain_record_hash_allowed"] is False
+    relation = linking["hash_relation_model"]
+    assert relation["public_anchor"] == ["network", "transaction_hash"]
+    assert relation["paired_caloriedb_anchor"] == "calorie_anchor_hash"
+    assert relation["top_pair_cardinality"] == "one-to-one"
+    assert relation["private_record_anchor"] == "calorie_record_fingerprint_id"
+    assert relation["join_entity"] == "ledger_record_link"
+    assert relation["transaction_hash_equals_record_hash"] is False
+    assert relation["anchor_to_record_many_to_many_edges_allowed"] is True
+    assert relation["explicit_authorization_required_per_edge"] is True
+    assert relation["cross_purpose_auto_linking_allowed"] is False
+    assert relation["public_hash_resolver_may_return_private_data"] is False
+    assert linking["unlinking"]["off_chain_association_deletable"] is True
+    assert linking["unlinking"]["on_chain_transaction_deletable"] is False
+
+
+def test_xrpl_linking_cannot_claim_automatic_worldwide_compliance() -> None:
+    boundary = _load_json("data-safety.json")["global_compliance_boundary"]
+
+    assert boundary["worldwide_compliance_claim_allowed"] is False
+    assert boundary["jurisdiction_feature_gates_required"] is True
+    assert boundary["data_protection_impact_assessment_required_before_enablement"] is True
+    assert boundary["independent_financial_regulatory_review_required_before_enablement"] is True
+    assert boundary["transaction_execution_or_routing_enabled"] is False
+
+
+def test_platform_budget_prevents_duplicate_core_services() -> None:
+    platforms = _load_json("data-safety.json")["platform_minimization"]
+
+    assert platforms["duplicate_identity_platform_allowed"] is False
+    assert platforms["second_primary_database_allowed"] is False
+    assert platforms["separate_graph_database_required"] is False
+    assert platforms["blockchain_database_required"] is False
+    assert platforms["ipfs_or_filecoin_required_for_core_release"] is False
+    assert platforms["new_provider_requires_architecture_record"] is True
+    assert platforms["roles"]["optional_ledger_reference"] == "XRPL only"
+
+
+def test_responsible_automation_keeps_human_release_and_privacy_gates() -> None:
+    automation = _load_json("data-safety.json")["responsible_automation"]
+
+    assert "test-and-build-checks" in automation["automated_by_default"]
+    assert "scheduled-staging-restore-drills" in automation["automated_by_default"]
+    assert "localization-completeness-checks" in automation["automated_by_default"]
+    assert "identity-purpose-expansion" in automation["approval_required"]
+    assert "xrpl-feature-enablement" in automation["approval_required"]
+    assert "public-content-publication" in automation["approval_required"]
+    assert automation["production_automation_runs_only_after_approval"] is True
+    assert automation["idempotent_and_retry_safe_required"] is True
+    assert automation["automatic_publication_allowed"] is False
+    assert automation["automatic_financial_action_allowed"] is False
+
+
+def test_all_required_durable_data_release_gates_are_explicit_and_blocking() -> None:
+    matrix = _load_json("release-test-matrix.json")
+    gates = {gate["id"]: gate for gate in matrix["gates"]}
+    expected = {
+        "provider_neutral_postgresql_configuration",
+        "production_sqlite_fail_closed",
+        "formal_schema_migrations",
+        "owner_isolation",
+        "restart_persistence",
+        "redeploy_persistence",
+        "backup_restore_drill",
+        "user_data_export",
+        "user_erasure",
+        "retention_policy",
+        "privacy_notice_alignment",
+        "no_personal_data_in_decentralized_public_storage",
+    }
+
+    assert matrix["contract_id"] == "calorieapp.durable-data-release-gates"
+    assert set(gates) == expected
+    assert all(gate["release_blocking"] is True for gate in gates.values())
+    assert all(gate["status"] in matrix["statuses"] for gate in gates.values())
+    assert gates["owner_isolation"]["status"] == "verified"
+    assert gates["production_sqlite_fail_closed"]["status"] == "not_started"
+    assert gates["retention_policy"]["status"] == "decision_required"
+    assert matrix["release_state"] == "blocked"
+
+
+def test_contract_release_order_ends_with_review_and_explicit_publication_go() -> None:
+    contract = _load_json("data-safety.json")
+    release_order = contract["release_order"]
+
+    assert release_order[-1] == "showcase-preview-review-explicit-go-scheduled-publish"
+    assert release_order.index("automation-and-observability-foundation") < release_order.index(
+        "formal-migrations"
+    )
+    assert release_order.index("privacy-review") < release_order.index(
+        "identity-feature-expansion"
+    )
+    optional_order = contract["optional_future_order"]
+    assert optional_order[0] == "xrpl-schema-compatibility-review"
+    assert optional_order[-1] == "adoption-led-scaling"

backend/tests/test_provenance_contract.py

@@ -0,0 +1,109 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+
+ROOT = Path(__file__).resolve().parents[2]
+CONTRACT = ROOT / "contracts" / "provenance" / "v1" / "traceability.json"
+
+
+def _contract() -> dict:
+    return json.loads(CONTRACT.read_text(encoding="utf-8"))
+
+
+def test_provenance_is_future_ready_without_becoming_a_launch_feature() -> None:
+    contract = _contract()
+    rollout = contract["rollout"]
+
+    assert contract["contract_id"] == "caloriedb.xrpl-linked-provenance"
+    assert rollout["core_public_release_dependency"] is False
+    assert rollout["initial_user_facing_feature"] is False
+    assert rollout["initial_wallet_or_ledger_scan"] is False
+    assert rollout["initial_transaction_or_memo_ui"] is False
+    assert "disabled feature flag" in rollout["phase_1"]
+    assert "testnet or synthetic-data" in rollout["phase_2"]
+
+    platform = contract["platform_boundary"]
+    assert platform["stored_in_primary_postgresql"] is True
+    assert platform["separate_graph_database_required"] is False
+    assert platform["additional_blockchain_required"] is False
+    assert platform["ipfs_required"] is False
+
+
+def test_provenance_automation_is_scoped_idempotent_and_disabled_by_default() -> None:
+    automation = _contract()["automation_boundary"]
+
+    assert automation["feature_flag_default"] == "disabled"
+    assert automation["explicit_purpose_scoped_link_request_required"] is True
+    assert automation["automatic_complete_wallet_scan_allowed"] is False
+    assert automation["automatic_cross_purpose_linking_allowed"] is False
+    assert automation["single_requested_transaction_verification_may_be_automated"] is True
+    assert automation["idempotent_ingestion_key"] == ["network", "transaction_hash"]
+    assert automation["retry_safe_processing_required"] is True
+    assert automation["production_enablement_requires_human_approval"] is True
+    assert automation["jurisdiction_gate_enforced_before_processing"] is True
+
+
+def test_top_anchor_is_one_to_one_from_xrpl_hash_to_caloriedb_hash() -> None:
+    anchor = _contract()["top_anchor"]
+
+    assert anchor["xrpl_key"] == ["network", "transaction_hash"]
+    assert anchor["caloriedb_key"] == "calorie_anchor_hash"
+    assert anchor["cardinality"] == "one-to-one"
+    assert anchor["hashes_are_equal"] is False
+    assert ["network", "transaction_hash"] in anchor["unique_constraints"]
+    assert ["calorie_anchor_hash"] in anchor["unique_constraints"]
+    assert "HMAC-SHA-256" in anchor["calorie_anchor_derivation"]
+
+
+def test_calorie_transaction_match_uses_exact_asset_identity_not_label_or_memo() -> None:
+    scope = _contract()["calorie_asset_scope"]
+
+    assert scope["exact_asset_registry_required"] is True
+    assert scope["registry_key"] == ["network", "issuer", "currency_code"]
+    assert scope["symbol_or_memo_only_match_allowed"] is False
+    assert scope["validated_transaction_and_metadata_required"] is True
+    assert scope["anchor_requires_calorie_relevance_evidence"] is True
+    assert scope["initial_pilot_transaction_types"] == ["Payment"]
+    assert scope["rule_version_stored_per_anchor"] is True
+
+
+def test_traceability_layers_start_at_hash_pair_before_events_and_lots() -> None:
+    layers = _contract()["layers_top_down"]
+
+    assert [layer["level"] for layer in layers] == [0, 1, 2, 3, 4]
+    assert [layer["entity"] for layer in layers] == [
+        "validated_xrpl_transaction",
+        "calorie_transaction_anchor",
+        "provenance_event",
+        "product_lot_or_batch",
+        "trace_view",
+    ]
+    assert layers[1]["relation_to_parent"] == "exactly-one"
+
+
+def test_supply_trace_is_a_gap_preserving_dag() -> None:
+    graph = _contract()["graph_rules"]
+
+    assert graph["shape"] == "directed-acyclic-graph"
+    assert graph["simple_linear_chain_assumed"] is False
+    assert graph["splits_supported"] is True
+    assert graph["merges_supported"] is True
+    assert graph["cycles_allowed"] is False
+    assert graph["missing_edges_reported_as_gaps"] is True
+    assert graph["missing_edges_inferred_or_fabricated"] is False
+
+
+def test_ledger_hash_does_not_claim_to_prove_physical_food_truth() -> None:
+    contract = _contract()
+    truth = contract["truth_boundary"]
+    privacy = contract["privacy_and_visibility"]
+
+    assert truth["physical_claim_requires_separate_evidence"] is True
+    assert "does not prove" not in truth["transaction_proves"]
+    assert "physical food product" in truth["transaction_does_not_prove"]
+    assert privacy["consumer_events_private_by_default"] is True
+    assert privacy["personal_food_history_public_by_default"] is False
+    assert privacy["public_transaction_lookup_may_return_private_record"] is False
+    assert privacy["cross_purpose_auto_linking_allowed"] is False

contracts/data-safety/v1/README.md

@@ -0,0 +1,24 @@
+# Durable data safety contract v1
+
+This directory defines the release-blocking data-safety boundary for CalorieApp.
+It separates durable private application data from optional decentralized
+research and records which controls are already verified, partial, planned or
+still require an explicit decision.
+
+The contract does not contain credentials, provider-specific endpoints or live
+user data. Passing its automated checks is necessary but is not by itself a
+legal, privacy or security certification.
+
+Files:
+
+- `data-safety.json`: architecture, data classes, retention boundaries and
+  prohibited storage patterns.
+- `release-test-matrix.json`: auditable status of every release-blocking gate.
+
+Status vocabulary:
+
+- `verified`: implemented and covered by the cited automated evidence.
+- `partial`: some foundations exist, but the release gate is not complete.
+- `not_started`: required implementation and verification remain outstanding.
+- `decision_required`: implementation must wait for an explicit policy choice.
+- `research_only`: not part of the current production architecture.