diff --git a/docs/runbook/README.md b/docs/runbook/README.md index 53c8db67738..cd20ea8b3dc 100644 --- a/docs/runbook/README.md +++ b/docs/runbook/README.md @@ -29,3 +29,4 @@ This page gathers the available guides to operate a Mithril network. | **Prepare Cardano node artifacts** | [prepare-cardano-node-artifacts](./prepare-cardano-node-artifacts/README.md) | Prepare and publish artifacts for an unreleased Cardano node version. | | **Cardano Docker bundle** | [cardano-docker-bundle](./cardano-docker-bundle/README.md) | Build and publish a Docker image bundling Cardano node with Mithril. | | **Update circuits verification keys** | [update-circuit-keys](./update-circuit-keys/README.md) | Update the circuits verification keys after an intentional circuit modification. | +| **Circuit verification key registry** | [circuit-key-registry](./circuit-key-registry/README.md) | Publish, rotate and revoke circuit verification keys in the genesis-signed registry. | diff --git a/docs/runbook/circuit-key-registry/README.md b/docs/runbook/circuit-key-registry/README.md new file mode 100644 index 00000000000..e3ef9283ce8 --- /dev/null +++ b/docs/runbook/circuit-key-registry/README.md @@ -0,0 +1,121 @@ +# Manage the circuit verification key registry + +## When to use this guide + +The circuit verification key registry is a genesis-signed whitelist of the circuit verification keys trusted for SNARK certificates (`Snark` and `IvcSnark` aggregate signature types). Clients resolve the registry of their network at runtime through the published `networks.json`: they select the network entry whose aggregators include their aggregator endpoint and download the signed registry file it references. Aggregators read the registry from a local file path. Both reject any SNARK certificate whose circuit verification keys are not certified by it. `networks.json` is pure routing, never trust: a wrong entry can only yield a registry that fails the genesis signature verification. Use this guide when: + +- a new circuit verification key enters service (new deployment, or a circuit change following [update-circuit-keys](../update-circuit-keys/README.md)), +- a circuit verification key is rotated out normally, +- a circuit or key turns out to be insecure and its certificates must be revoked. + +## Registry format + +The registry is a JSON document published per network as `mithril-infra/configuration//circuit-verification-key-registry.json`, next to the network's other trust material, and referenced from the network's `networks.json` entry (devnet and the end to end tests generate theirs on the fly with the test-only `circuit-key-registry bootstrap` command): + +```json +"circuit-verification-key-registry": { + "url": "https://raw.githubusercontent.com/IntersectMBO/mithril/main/mithril-infra/configuration/release-mainnet/circuit-verification-key-registry.json" +} +``` + +Its content is: + +```json +{ + "registry": { + "version": 2, + "entries": [ + { + "digest": "3e5a…9c1f", + "name": "certificate-circuit v1", + "status": "allowed", + "start_epoch": 500, + "end_epoch": null, + "comment": null + }, + { + "digest": "7bd2…04aa", + "name": "ivc-circuit v1", + "status": "revoked", + "start_epoch": 520, + "end_epoch": null, + "comment": "revoked: soundness issue in the accumulator check" + } + ] + }, + "signature": "…" +} +``` + +Semantics enforced by the nodes: + +- A digest absent from the registry is rejected (whitelist). +- An `allowed` entry covers the inclusive epoch range `[start_epoch, end_epoch]` (`end_epoch: null` is open-ended). +- A `revoked` entry overrides any `allowed` entry wherever their ranges overlap (revocation wins), so certificates in the revoked range are rejected retroactively. +- `version` must increase at every publication; nodes reject a registry older than their compiled minimum version. That minimum (`MINIMUM_REGISTRY_VERSION`) is a single floor shared by every network: bump it only once every network's registry has been re-signed at or above the new floor, otherwise clients of the other networks fail closed. +- The `signature` is the Ed25519 half of the genesis key over the canonical JSON bytes of the `registry` object. +- The registry is scoped by the genesis key that signs it: a registry signed for another network fails the signature check. This relies on the invariant that networks never share a genesis key. + +The digests are domain separated Poseidon hashes (SNARK-friendly, native to the scalar field of the circuits) of the transcript representation of the verification keys, the field element Halo2 derives from the pinned constraint system, the evaluation domain and the commitments, so they bind the circuit gates and not only the serialized key bytes. The two circuits behave differently: + +- The IVC circuit does not depend on the protocol parameters: its verification key is the production constant embedded in `mithril-stm`, so its digest is the same for every network. +- The certificate circuit depends on the protocol parameters (`k` and `m`): its verification key, hence its digest, changes with them. The registry must allow the digest of the protocol parameters the network signs with, so a protocol parameters update changing `k` or `m` must be preceded by a registry publication allowing the digest of the new parameters. + +The digests are exported with the aggregator (the certificate circuit key is derived from the trusted setup for the given protocol parameters, which is fast for small `k`; without `--protocol-parameters` the production certificate circuit key embedded in `mithril-stm` is used): + +```bash +./mithril-aggregator circuit-key-registry export \ + --protocol-parameters '{"k":2422,"m":20973,"phi_f":0.2}' \ + --target-path digests.json +``` + +The digests can also be read from the ancillary verifier data of a certificate produced with the keys (certificate circuit first, then IVC circuit), and the production ones are printed by the ignored `print_circuit_verification_key_digests_for_production` test of `mithril-stm`. + +## Publish a new registry version + +1. Export the digests of the keys concerned (see above) on a machine with network access. +2. On the air-gapped machine holding the genesis secret key, add the entry to the current signed registry with `whitelist` or `revoke`: the command verifies the existing registry signature, appends the entry, increments the `version`, signs and writes the file in place (it creates the registry when the file does not exist yet): + +```bash +./mithril-aggregator circuit-key-registry whitelist \ + --registry-path circuit-verification-key-registry.json \ + --genesis-secret-key-path genesis.sk \ + --digest 3e5a…9c1f \ + --name "certificate-circuit v1" \ + --start-epoch 500 + +./mithril-aggregator circuit-key-registry revoke \ + --registry-path circuit-verification-key-registry.json \ + --genesis-secret-key-path genesis.sk \ + --digest 7bd2…04aa \ + --name "ivc-circuit v1" \ + --start-epoch 520 \ + --comment "revoked: soundness issue in the accumulator check" +``` + +Both commands accept `--end-epoch` to close the range and `--comment` to record the reason (mandatory for a revocation). A hand-authored unsigned registry can still be signed as a whole with `circuit-key-registry sign --to-sign-registry-path registry.json --target-signed-registry-path circuit-verification-key-registry.json --genesis-secret-key-path genesis.sk`. + +3. Open a PR committing the signed file at `mithril-infra/configuration//circuit-verification-key-registry.json`, referencing it from the network's `networks.json` entry (first publication only), and have it reviewed by the tech lead and the cryptographers. +4. Once merged on `main`, the file is live: clients resolve and download it through `networks.json`, and aggregators read it from the path set in their `circuit_verification_key_registry_path` configuration, which defaults to `circuit-verification-key-registry.json` in the aggregator data stores directory (update the deployed file accordingly). + +Running nodes cache the verified registry for one hour before retrieving and verifying it again (downloads are retried), so a new version (including a revocation) is picked up without restarting them; a refresh never accepts a registry version lower than the one already verified. + +For tests and local deployments, the unstable `--circuit-verification-key-registry-path` parameter of the client CLI (or the `CIRCUIT_VERIFICATION_KEY_REGISTRY_PATH` environment variable) makes it read the signed registry from a local file instead of resolving it through `networks.json`; it only changes the source, the genesis signature verification still applies. Library users get the same through `with_circuit_verification_key_registry_retriever` on the client builder. + +## Lifecycle procedures + +### New key entering service + +`whitelist` the digest with `--start-epoch` set to the first epoch the key certifies and no `--end-epoch`. + +### Normal rotation + +Close the outgoing key's `allowed` entry by setting its `end_epoch` to the last epoch it legitimately certified (edit the registry and sign it as a whole), and `whitelist` the incoming key. Certificates from the closed range keep verifying forever. + +### Revocation after a vulnerability + +1. Determine the epoch range in which the flaw is exploitable. For a soundness flaw (forged certificates possible), always revoke the key's whole lifetime: a forger chooses the epoch its certificate claims, so a partial range only helps for flaws that are externally time-bound. +2. `revoke` the digest over that range with an explanatory `--comment`; the original `allowed` entry stays untouched as an audit trail. +3. Publish the new registry version (see above). +4. Bump the `MINIMUM_REGISTRY_VERSION` constant in `mithril-common` (`crypto_helper/circuit_key_registry/certifier.rs`) to the new version and release the clients, so an attacker replaying the previous signed registry cannot resurrect the revoked key. +5. Proceed to a re-genesis with the fixed circuit keys ([update-circuit-keys](../update-circuit-keys/README.md) and [genesis-manually](../genesis-manually/README.md)). diff --git a/docs/runbook/update-circuit-keys/README.md b/docs/runbook/update-circuit-keys/README.md index c478416e61f..bd97f86d1d4 100644 --- a/docs/runbook/update-circuit-keys/README.md +++ b/docs/runbook/update-circuit-keys/README.md @@ -42,6 +42,7 @@ Release manager: - Prepares the release of this update - Schedule the re-genesis of the certificate chain +- Publishes the new version of the circuit verification key registry (see [circuit-key-registry](../circuit-key-registry/README.md)) ## Update of the golden value @@ -63,6 +64,10 @@ cargo test -p mithril-stm --features future_snark,rustls --release write_recursi that will update the files holding the values of the production keys, `mithril-stm/src/circuits/halo2/non_recursive_circuit_verification_key_for_production.bin` and `mithril-stm/src/circuits/halo2_ivc/recursive_circuit_verification_key_for_production.bin`. +## Update of the circuit verification key registry + +Changing a circuit changes its verification key, and thus its digest in the circuit verification key registry. A new registry version must be authored, signed with the genesis key and published at the root of the repository, whitelisting the new keys and closing (or revoking, in case of a vulnerability) the outgoing ones, following the [circuit-key-registry](../circuit-key-registry/README.md) runbook. Clients download the registry file from its raw GitHub URL on `main`, so without this publication they reject the certificates produced with the new keys. + ## Scheduling of the re-genesis Once the review is done and all the circuit verification keys are updated, the release manager can schedule the re-genesis. Re-genesis is scheduled with the release of the next distribution where the new circuit is deployed. It goes through the sequence: diff --git a/mithril-aggregator/src/commands/circuit_key_registry_command.rs b/mithril-aggregator/src/commands/circuit_key_registry_command.rs new file mode 100644 index 00000000000..0e8933cadf9 --- /dev/null +++ b/mithril-aggregator/src/commands/circuit_key_registry_command.rs @@ -0,0 +1,316 @@ +use std::{collections::HashMap, path::PathBuf}; + +use anyhow::Context; +use clap::{Args, Parser, Subcommand}; +use slog::{Logger, debug}; + +use mithril_common::{ + StdResult, + crypto_helper::{ + CircuitVerificationKeyDigest, CircuitVerificationKeyEntry, CircuitVerificationKeyStatus, + }, + entities::{Epoch, HexEncodedGenesisSecretKey, ProtocolParameters}, +}; +use mithril_doc::StructDoc; + +use crate::{extract_all, tools::CircuitKeyRegistryTools}; + +/// Circuit verification key registry tools +#[derive(Parser, Debug, Clone)] +pub struct CircuitKeyRegistryCommand { + /// commands + #[clap(subcommand)] + pub circuit_key_registry_subcommand: CircuitKeyRegistrySubCommand, +} + +impl CircuitKeyRegistryCommand { + pub async fn execute(&self, root_logger: Logger) -> StdResult<()> { + self.circuit_key_registry_subcommand.execute(root_logger).await + } + + pub fn extract_config(command_path: String) -> HashMap { + extract_all!( + command_path, + CircuitKeyRegistrySubCommand, + Export = { ExportCircuitKeyRegistrySubCommand }, + Whitelist = { WhitelistCircuitKeyRegistrySubCommand }, + Revoke = { RevokeCircuitKeyRegistrySubCommand }, + Sign = { SignCircuitKeyRegistrySubCommand }, + Bootstrap = { BootstrapCircuitKeyRegistrySubCommand }, + ) + } + + /// Parse protocol parameters from their JSON representation. + fn parse_protocol_parameters(value: &str) -> Result { + serde_json::from_str(value) + .map_err(|error| format!("invalid protocol parameters JSON: {error}")) + } +} + +/// Circuit verification key registry commands. +#[derive(Debug, Clone, Subcommand)] +pub enum CircuitKeyRegistrySubCommand { + /// Circuit verification key digests export command. + Export(ExportCircuitKeyRegistrySubCommand), + + /// Circuit verification key whitelist command. + Whitelist(WhitelistCircuitKeyRegistrySubCommand), + + /// Circuit verification key revoke command. + Revoke(RevokeCircuitKeyRegistrySubCommand), + + /// Circuit verification key registry sign command. + Sign(SignCircuitKeyRegistrySubCommand), + + /// Circuit verification key registry bootstrap command (test only). + Bootstrap(BootstrapCircuitKeyRegistrySubCommand), +} + +impl CircuitKeyRegistrySubCommand { + pub async fn execute(&self, root_logger: Logger) -> StdResult<()> { + match self { + Self::Export(cmd) => cmd.execute(root_logger).await, + Self::Whitelist(cmd) => cmd.execute(root_logger).await, + Self::Revoke(cmd) => cmd.execute(root_logger).await, + Self::Sign(cmd) => cmd.execute(root_logger).await, + Self::Bootstrap(cmd) => cmd.execute(root_logger).await, + } + } +} + +/// Circuit verification key digests export command +#[derive(Parser, Debug, Clone)] +pub struct ExportCircuitKeyRegistrySubCommand { + /// Protocol parameters of the network as JSON (e.g. '{"k":5,"m":9,"phi_f":0.95}'), defaults to + /// the production protocol parameters of the embedded certificate circuit key + #[clap(long, value_parser = CircuitKeyRegistryCommand::parse_protocol_parameters)] + protocol_parameters: Option, + + /// Target Path + #[clap(long)] + target_path: PathBuf, +} + +impl ExportCircuitKeyRegistrySubCommand { + pub async fn execute(&self, root_logger: Logger) -> StdResult<()> { + debug!(root_logger, "EXPORT CIRCUIT KEY REGISTRY command"); + println!( + "Circuit verification key digests export to {}", + self.target_path.display() + ); + + let digests = CircuitKeyRegistryTools::export_digests( + self.protocol_parameters.as_ref(), + &self.target_path, + ) + .with_context(|| "circuit-key-registry-tools: export digests error")?; + println!("certificate-circuit: {}", digests.certificate_circuit); + println!("ivc-circuit: {}", digests.ivc_circuit); + + Ok(()) + } + + pub fn extract_config(_parent: String) -> HashMap { + HashMap::new() + } +} + +/// Arguments identifying a registry entry and the signed registry to add it to +#[derive(Args, Debug, Clone)] +pub struct CircuitKeyRegistryEntryArguments { + /// Signed Registry Path, created when missing and updated in place + #[clap(long)] + registry_path: PathBuf, + + /// Genesis Secret Key Path + #[clap(long)] + genesis_secret_key_path: PathBuf, + + /// Digest of the circuit verification key (hex encoded) + #[clap(long)] + digest: CircuitVerificationKeyDigest, + + /// Name of the circuit verification key (e.g. 'certificate-circuit v1') + #[clap(long)] + name: String, + + /// First epoch (inclusive) covered by the entry + #[clap(long)] + start_epoch: u64, + + /// Last epoch (inclusive) covered by the entry, open-ended when omitted + #[clap(long)] + end_epoch: Option, +} + +impl CircuitKeyRegistryEntryArguments { + /// Add the entry with the given status and comment to the signed registry and print the + /// resulting registry version. + fn add_entry( + &self, + status: CircuitVerificationKeyStatus, + comment: Option, + ) -> StdResult<()> { + let entry = CircuitVerificationKeyEntry { + digest: self.digest, + name: self.name.clone(), + status, + start_epoch: Epoch(self.start_epoch), + end_epoch: self.end_epoch.map(Epoch), + comment, + }; + let registry = CircuitKeyRegistryTools::add_entry( + &self.registry_path, + &self.genesis_secret_key_path, + entry, + ) + .with_context(|| "circuit-key-registry-tools: add entry error")?; + println!( + "Circuit verification key registry version {} signed and written to {}", + registry.version, + self.registry_path.display() + ); + + Ok(()) + } +} + +/// Circuit verification key whitelist command +#[derive(Parser, Debug, Clone)] +pub struct WhitelistCircuitKeyRegistrySubCommand { + #[clap(flatten)] + entry: CircuitKeyRegistryEntryArguments, + + /// Comment recorded in the entry + #[clap(long)] + comment: Option, +} + +impl WhitelistCircuitKeyRegistrySubCommand { + pub async fn execute(&self, root_logger: Logger) -> StdResult<()> { + debug!(root_logger, "WHITELIST CIRCUIT KEY REGISTRY command"); + println!( + "Circuit verification key '{}' whitelist in {}", + self.entry.name, + self.entry.registry_path.display() + ); + + self.entry + .add_entry(CircuitVerificationKeyStatus::Allowed, self.comment.clone()) + } + + pub fn extract_config(_parent: String) -> HashMap { + HashMap::new() + } +} + +/// Circuit verification key revoke command +#[derive(Parser, Debug, Clone)] +pub struct RevokeCircuitKeyRegistrySubCommand { + #[clap(flatten)] + entry: CircuitKeyRegistryEntryArguments, + + /// Comment recorded in the entry, explaining the revocation + #[clap(long)] + comment: String, +} + +impl RevokeCircuitKeyRegistrySubCommand { + pub async fn execute(&self, root_logger: Logger) -> StdResult<()> { + debug!(root_logger, "REVOKE CIRCUIT KEY REGISTRY command"); + println!( + "Circuit verification key '{}' revocation in {}", + self.entry.name, + self.entry.registry_path.display() + ); + + self.entry.add_entry( + CircuitVerificationKeyStatus::Revoked, + Some(self.comment.clone()), + ) + } + + pub fn extract_config(_parent: String) -> HashMap { + HashMap::new() + } +} + +/// Circuit verification key registry sign command +#[derive(Parser, Debug, Clone)] +pub struct SignCircuitKeyRegistrySubCommand { + /// To Sign Registry Path + #[clap(long)] + to_sign_registry_path: PathBuf, + + /// Target Signed Registry Path + #[clap(long)] + target_signed_registry_path: PathBuf, + + /// Genesis Secret Key Path + #[clap(long)] + genesis_secret_key_path: PathBuf, +} + +impl SignCircuitKeyRegistrySubCommand { + pub async fn execute(&self, root_logger: Logger) -> StdResult<()> { + debug!(root_logger, "SIGN CIRCUIT KEY REGISTRY command"); + println!( + "Circuit verification key registry sign from {} to {}", + self.to_sign_registry_path.display(), + self.target_signed_registry_path.display() + ); + + CircuitKeyRegistryTools::sign( + &self.to_sign_registry_path, + &self.target_signed_registry_path, + &self.genesis_secret_key_path, + ) + .with_context(|| "circuit-key-registry-tools: sign registry error")?; + + Ok(()) + } + + pub fn extract_config(_parent: String) -> HashMap { + HashMap::new() + } +} + +/// Circuit verification key registry bootstrap command (test only) +#[derive(Parser, Debug, Clone)] +pub struct BootstrapCircuitKeyRegistrySubCommand { + /// Genesis Secret Key (test only) + #[clap(long, env = "GENESIS_SECRET_KEY")] + genesis_secret_key: HexEncodedGenesisSecretKey, + + /// Protocol parameters of the network as JSON (e.g. '{"k":5,"m":9,"phi_f":0.95}'), defaults to + /// the production protocol parameters of the embedded certificate circuit key + #[clap(long, value_parser = CircuitKeyRegistryCommand::parse_protocol_parameters)] + protocol_parameters: Option, + + /// Target Registry Path + #[clap(long)] + target_registry_path: PathBuf, +} + +impl BootstrapCircuitKeyRegistrySubCommand { + pub async fn execute(&self, root_logger: Logger) -> StdResult<()> { + debug!(root_logger, "BOOTSTRAP CIRCUIT KEY REGISTRY command"); + println!( + "Circuit verification key registry bootstrap for test only, to {}", + self.target_registry_path.display() + ); + + CircuitKeyRegistryTools::bootstrap( + &self.genesis_secret_key, + self.protocol_parameters.as_ref(), + &self.target_registry_path, + ) + .with_context(|| "circuit-key-registry-tools: bootstrap registry error")?; + + Ok(()) + } + + pub fn extract_config(_parent: String) -> HashMap { + HashMap::new() + } +} diff --git a/mithril-aggregator/src/commands/mod.rs b/mithril-aggregator/src/commands/mod.rs index ee4c55bc268..0c91bb8052b 100644 --- a/mithril-aggregator/src/commands/mod.rs +++ b/mithril-aggregator/src/commands/mod.rs @@ -1,3 +1,5 @@ +#[cfg(feature = "future_snark")] +mod circuit_key_registry_command; mod config_association; mod database_command; mod era_command; @@ -26,6 +28,8 @@ pub enum MainCommand { Tools(tools_command::ToolsCommand), Database(database_command::DatabaseCommand), ProtocolConfiguration(protocol_configuration_command::ProtocolConfigurationCommand), + #[cfg(feature = "future_snark")] + CircuitKeyRegistry(circuit_key_registry_command::CircuitKeyRegistryCommand), #[clap(alias("doc"), hide(true))] GenerateDoc(GenerateDocCommands), } @@ -51,6 +55,8 @@ impl MainCommand { Self::Tools(cmd) => cmd.execute(root_logger, config_builder).await, Self::Database(cmd) => cmd.execute(root_logger, config_builder).await, Self::ProtocolConfiguration(cmd) => cmd.execute(root_logger, config_builder).await, + #[cfg(feature = "future_snark")] + Self::CircuitKeyRegistry(cmd) => cmd.execute(root_logger).await, Self::GenerateDoc(cmd) => { let commands_configs = Self::extract_config(Self::format_crate_name_to_config_key()); @@ -62,18 +68,37 @@ impl MainCommand { } pub fn extract_config(command_path: String) -> HashMap { - extract_all!( - command_path, - MainCommand, - Database = { database_command::DatabaseCommand }, - Era = { era_command::EraCommand }, - Genesis = { genesis_command::GenesisCommand }, - Serve = { serve_command::ServeCommand }, - Tools = { tools_command::ToolsCommand }, - ProtocolConfiguration = - { protocol_configuration_command::ProtocolConfigurationCommand }, - GenerateDoc = {}, - ) + #[cfg(feature = "future_snark")] + { + extract_all!( + command_path, + MainCommand, + Database = { database_command::DatabaseCommand }, + Era = { era_command::EraCommand }, + Genesis = { genesis_command::GenesisCommand }, + Serve = { serve_command::ServeCommand }, + Tools = { tools_command::ToolsCommand }, + ProtocolConfiguration = + { protocol_configuration_command::ProtocolConfigurationCommand }, + CircuitKeyRegistry = { circuit_key_registry_command::CircuitKeyRegistryCommand }, + GenerateDoc = {}, + ) + } + #[cfg(not(feature = "future_snark"))] + { + extract_all!( + command_path, + MainCommand, + Database = { database_command::DatabaseCommand }, + Era = { era_command::EraCommand }, + Genesis = { genesis_command::GenesisCommand }, + Serve = { serve_command::ServeCommand }, + Tools = { tools_command::ToolsCommand }, + ProtocolConfiguration = + { protocol_configuration_command::ProtocolConfigurationCommand }, + GenerateDoc = {}, + ) + } } fn format_crate_name_to_config_key() -> String { @@ -88,6 +113,8 @@ impl MainCommand { MainCommand::Tools(_) => CommandType::CommandLine, MainCommand::Database(_) => CommandType::CommandLine, MainCommand::ProtocolConfiguration(_) => CommandType::CommandLine, + #[cfg(feature = "future_snark")] + MainCommand::CircuitKeyRegistry(_) => CommandType::CommandLine, MainCommand::GenerateDoc(_) => CommandType::CommandLine, } } diff --git a/mithril-aggregator/src/configuration.rs b/mithril-aggregator/src/configuration.rs index ddde2612d1b..0e91c6a8ca9 100644 --- a/mithril-aggregator/src/configuration.rs +++ b/mithril-aggregator/src/configuration.rs @@ -322,6 +322,15 @@ pub trait ConfigurationSource { panic!("custom_origin_tag_white_list is not implemented."); } + /// Path to the signed circuit verification key registry file enforced on the certificates + /// whose aggregate signature type requires it. + /// + /// Defaults to the `circuit-verification-key-registry.json` file in the data stores directory. + fn circuit_verification_key_registry_path(&self) -> PathBuf { + self.data_stores_directory() + .join(CIRCUIT_VERIFICATION_KEY_REGISTRY_FILE_NAME) + } + /// Get the server URL. fn get_server_url(&self) -> StdResult { panic!("get_server_url is not implemented."); @@ -427,6 +436,11 @@ pub trait ConfigurationSource { } } +/// File name of the signed circuit verification key registry, joined to the data stores +/// directory as the default registry path. +pub const CIRCUIT_VERIFICATION_KEY_REGISTRY_FILE_NAME: &str = + "circuit-verification-key-registry.json"; + /// Serve command configuration #[derive(Debug, Clone, Deserialize, Documenter)] pub struct ServeCommandConfiguration { @@ -657,6 +671,12 @@ pub struct ServeCommandConfiguration { /// Delay to wait between two signature processing attempts after an error pub signature_processor_wait_delay_on_error_ms: u64, + + /// Path to the signed circuit verification key registry file enforced on the certificates + /// whose aggregate signature type requires it. + /// + /// Defaults to the `circuit-verification-key-registry.json` file in the data stores directory. + pub circuit_verification_key_registry_path: Option, } /// Uploader needed to copy the snapshot once computed. @@ -835,6 +855,7 @@ impl ServeCommandConfiguration { custom_origin_tag_white_list: None, aggregate_signature_type: AggregateSignatureType::Concatenation, signature_processor_wait_delay_on_error_ms: 5000, + circuit_verification_key_registry_path: None, } } @@ -1040,6 +1061,15 @@ impl ConfigurationSource for ServeCommandConfiguration { self.custom_origin_tag_white_list.clone() } + fn circuit_verification_key_registry_path(&self) -> PathBuf { + self.circuit_verification_key_registry_path + .clone() + .unwrap_or_else(|| { + self.data_stores_directory + .join(CIRCUIT_VERIFICATION_KEY_REGISTRY_FILE_NAME) + }) + } + fn get_server_url(&self) -> StdResult { match &self.public_server_url { Some(url) => SanitizedUrlWithTrailingSlash::parse(url), diff --git a/mithril-aggregator/src/dependency_injection/builder/mod.rs b/mithril-aggregator/src/dependency_injection/builder/mod.rs index e7b4c58ba7e..b108790b641 100644 --- a/mithril-aggregator/src/dependency_injection/builder/mod.rs +++ b/mithril-aggregator/src/dependency_injection/builder/mod.rs @@ -25,6 +25,10 @@ use mithril_cardano_node_internal_database::{ ImmutableFileObserver, digesters::{ImmutableDigester, cache::ImmutableFileDigestCacheProvider}, }; +#[cfg(feature = "future_snark")] +use mithril_common::crypto_helper::{ + CircuitVerificationKeyCertifier, CircuitVerificationKeyRegistryRetriever, +}; use mithril_common::{ api_version::APIVersionProvider, certificate_chain::CertificateVerifier, @@ -191,6 +195,15 @@ pub struct DependenciesBuilder { /// Certificate verifier service. pub certificate_verifier: Option>, + /// Circuit verification key registry retriever service. + #[cfg(feature = "future_snark")] + pub circuit_verification_key_registry_retriever: + Option>, + + /// Circuit verification key certifier service. + #[cfg(feature = "future_snark")] + pub circuit_verification_key_certifier: Option>, + /// Genesis signature verifier service. pub genesis_verifier: Option>, @@ -332,6 +345,10 @@ impl DependenciesBuilder { file_archiver: None, snapshotter: None, certificate_verifier: None, + #[cfg(feature = "future_snark")] + circuit_verification_key_registry_retriever: None, + #[cfg(feature = "future_snark")] + circuit_verification_key_certifier: None, genesis_verifier: None, certificate_chain_synchronizer: None, mithril_signer_registration_leader: None, @@ -498,6 +515,10 @@ impl DependenciesBuilder { protocol_parameters_retriever: self.get_protocol_parameters_retriever().await?, verification_key_store: self.get_verification_key_store().await?, mithril_era, + #[cfg(feature = "future_snark")] + circuit_verification_key_registry_retriever: self + .get_circuit_verification_key_registry_retriever() + .await?, logger: self.root_logger(), }; diff --git a/mithril-aggregator/src/dependency_injection/builder/protocol/certificates.rs b/mithril-aggregator/src/dependency_injection/builder/protocol/certificates.rs index 0d6afc4bbb6..3156ffd3a92 100644 --- a/mithril-aggregator/src/dependency_injection/builder/protocol/certificates.rs +++ b/mithril-aggregator/src/dependency_injection/builder/protocol/certificates.rs @@ -3,6 +3,12 @@ use std::sync::Arc; use mithril_common::certificate_chain::{CertificateVerifier, MithrilCertificateVerifier}; use mithril_common::crypto_helper::GenesisVerifier; +#[cfg(feature = "future_snark")] +use mithril_common::crypto_helper::{ + CachedCircuitVerificationKeyCertifier, CircuitVerificationKeyCertifier, + CircuitVerificationKeyRegistryRetriever, FileCircuitVerificationKeyRegistryRetriever, + MithrilCircuitVerificationKeyCertifier, +}; use crate::database::repository::{BufferedSingleSignatureRepository, SingleSignatureRepository}; use crate::dependency_injection::{DependenciesBuilder, DependenciesBuilderError, Result}; @@ -98,6 +104,8 @@ impl DependenciesBuilder { self.root_logger(), leader_aggregator_client.clone(), self.get_genesis_verifier().await?, + #[cfg(feature = "future_snark")] + self.get_circuit_verification_key_certifier().await?, )); Arc::new(MithrilCertificateChainSynchronizer::new( @@ -127,11 +135,54 @@ impl DependenciesBuilder { self.root_logger(), self.get_certificate_repository().await?, self.get_genesis_verifier().await?, + #[cfg(feature = "future_snark")] + self.get_circuit_verification_key_certifier().await?, )); Ok(verifier) } + /// Build the certifier enforcing the signed circuit verification key registry read from the + /// configured registry path, with a caching decorator refreshing it periodically. + #[cfg(feature = "future_snark")] + async fn build_circuit_verification_key_certifier( + &mut self, + ) -> Result> { + Ok(Arc::new(CachedCircuitVerificationKeyCertifier::new( + Arc::new(MithrilCircuitVerificationKeyCertifier::new( + self.get_circuit_verification_key_registry_retriever().await?, + self.get_genesis_verifier().await?, + )), + ))) + } + + /// Build the retriever reading the signed circuit verification key registry from the + /// configured registry path. + #[cfg(feature = "future_snark")] + async fn build_circuit_verification_key_registry_retriever( + &mut self, + ) -> Result> { + Ok(Arc::new(FileCircuitVerificationKeyRegistryRetriever::new( + self.configuration.circuit_verification_key_registry_path(), + ))) + } + + /// [CircuitVerificationKeyRegistryRetriever] service. + #[cfg(feature = "future_snark")] + pub async fn get_circuit_verification_key_registry_retriever( + &mut self, + ) -> Result> { + get_dependency!(self.circuit_verification_key_registry_retriever) + } + + /// [CircuitVerificationKeyCertifier] service. + #[cfg(feature = "future_snark")] + pub async fn get_circuit_verification_key_certifier( + &mut self, + ) -> Result> { + get_dependency!(self.circuit_verification_key_certifier) + } + /// [CertificateVerifier] service. pub async fn get_certificate_verifier(&mut self) -> Result> { get_dependency!(self.certificate_verifier) diff --git a/mithril-aggregator/src/dependency_injection/containers/genesis.rs b/mithril-aggregator/src/dependency_injection/containers/genesis.rs index f20ce4730d0..f3284edf94d 100644 --- a/mithril-aggregator/src/dependency_injection/containers/genesis.rs +++ b/mithril-aggregator/src/dependency_injection/containers/genesis.rs @@ -3,6 +3,8 @@ use std::sync::Arc; use slog::Logger; use mithril_cardano_node_chain::chain_observer::ChainObserver; +#[cfg(feature = "future_snark")] +use mithril_common::crypto_helper::CircuitVerificationKeyRegistryRetriever; use mithril_common::{CardanoNetwork, entities::SupportedEra}; use crate::database::repository::CertificateRepository; @@ -28,6 +30,11 @@ pub struct GenesisCommandDependenciesContainer { /// Mithril era to use for the genesis certificate. pub mithril_era: SupportedEra, + /// Circuit verification key registry retriever. + #[cfg(feature = "future_snark")] + pub circuit_verification_key_registry_retriever: + Arc, + /// Logger. pub logger: Logger, } diff --git a/mithril-aggregator/src/services/certificate_chain_synchronizer/synchronizer_service.rs b/mithril-aggregator/src/services/certificate_chain_synchronizer/synchronizer_service.rs index bca39d4a316..2b4df28db3e 100644 --- a/mithril-aggregator/src/services/certificate_chain_synchronizer/synchronizer_service.rs +++ b/mithril-aggregator/src/services/certificate_chain_synchronizer/synchronizer_service.rs @@ -267,6 +267,8 @@ mod tests { use mithril_common::certificate_chain::MithrilCertificateVerifier; use mithril_common::crypto_helper::GenesisVerifier; use mithril_common::entities::Epoch; + #[cfg(feature = "future_snark")] + use mithril_common::test::double::FakeCircuitVerificationKeyCertifier; use mithril_common::test::{ builder::{CertificateChainBuilder, CertificateChainFixture}, double::{Dummy, FakeCertificaterRetriever, fake_data}, @@ -368,6 +370,8 @@ mod tests { remote_certificate_chain, )), genesis_verifier, + #[cfg(feature = "future_snark")] + Arc::new(FakeCircuitVerificationKeyCertifier::that_fails()), ); Arc::new(verifier) } diff --git a/mithril-aggregator/src/tools/circuit_key_registry.rs b/mithril-aggregator/src/tools/circuit_key_registry.rs new file mode 100644 index 00000000000..0401ef8bd13 --- /dev/null +++ b/mithril-aggregator/src/tools/circuit_key_registry.rs @@ -0,0 +1,574 @@ +//! Tools for the circuit verification key registry: export the circuit key digests, extend a +//! registry with genesis-signed whitelist and revocation entries, sign and bootstrap it. + +use std::{fs::read_to_string, path::Path}; + +use anyhow::{Context, anyhow}; +use serde::{Deserialize, Serialize}; + +use mithril_common::{ + StdResult, + crypto_helper::{ + CircuitVerificationKeyDigest, CircuitVerificationKeyEntry, CircuitVerificationKeyRegistry, + CircuitVerificationKeyStatus, GenesisSigner, MINIMUM_REGISTRY_VERSION, + SignedCircuitVerificationKeyRegistry, + }, + entities::{Epoch, ProtocolParameters}, +}; + +/// Digests of the circuit verification keys a network signs with. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct CircuitVerificationKeyDigests { + /// Digest of the certificate circuit verification key. + pub certificate_circuit: CircuitVerificationKeyDigest, + + /// Digest of the IVC circuit verification key. + pub ivc_circuit: CircuitVerificationKeyDigest, +} + +impl CircuitVerificationKeyDigests { + /// Compute the digests for the given protocol parameters, deriving the certificate circuit key + /// from the trusted setup when it is not cached yet, or using the embedded production + /// certificate circuit key when no parameters are given. + pub fn compute(protocol_parameters: Option<&ProtocolParameters>) -> StdResult { + let certificate_circuit = match protocol_parameters { + Some(parameters) => CircuitVerificationKeyDigest::compute_for_certificate_circuit( + ¶meters.clone().into(), + ) + .with_context(|| { + format!( + "Failed to compute the certificate circuit verification key digest for protocol parameters {parameters:?}" + ) + })?, + None => CircuitVerificationKeyDigest::for_production_certificate_circuit() + .with_context(|| { + "Failed to compute the production certificate circuit verification key digest" + })?, + }; + let ivc_circuit = CircuitVerificationKeyDigest::for_ivc_circuit() + .with_context(|| "Failed to compute the IVC circuit verification key digest")?; + + Ok(Self { + certificate_circuit, + ivc_circuit, + }) + } +} + +/// Circuit verification key registry tools. +pub struct CircuitKeyRegistryTools; + +impl CircuitKeyRegistryTools { + /// Export the circuit verification key digests for the given protocol parameters as a JSON + /// file. + pub fn export_digests( + protocol_parameters: Option<&ProtocolParameters>, + target_path: &Path, + ) -> StdResult { + let digests = CircuitVerificationKeyDigests::compute(protocol_parameters)?; + std::fs::write(target_path, serde_json::to_string_pretty(&digests)?).with_context( + || { + format!( + "Failed to write circuit verification key digests file at '{}'", + target_path.display() + ) + }, + )?; + + Ok(digests) + } + + /// Add an entry to the signed registry at the given path, creating the registry when the file + /// does not exist, then sign the incremented version with the genesis secret key and write it + /// back in place. An existing registry must carry a valid signature of the same genesis key. + pub fn add_entry( + registry_path: &Path, + genesis_secret_key_path: &Path, + entry: CircuitVerificationKeyEntry, + ) -> StdResult { + let genesis_signer = GenesisSigner::read_from_file(genesis_secret_key_path)?; + let mut registry = match Self::read_signed_registry(registry_path, &genesis_signer)? { + Some(current_registry) => CircuitVerificationKeyRegistry { + version: current_registry.version + 1, + entries: current_registry.entries, + }, + None => CircuitVerificationKeyRegistry { + version: MINIMUM_REGISTRY_VERSION, + entries: vec![], + }, + }; + registry.entries.push(entry); + Self::check_registry_can_be_signed(®istry)?; + Self::sign_and_write(®istry, &genesis_signer, registry_path)?; + + Ok(registry) + } + + /// Sign a circuit verification key registry with the Ed25519 half of the genesis signing key + /// and write the signed registry JSON, after verifying the produced signature. + pub fn sign( + to_sign_registry_path: &Path, + target_signed_registry_path: &Path, + genesis_secret_key_path: &Path, + ) -> StdResult<()> { + let genesis_signer = GenesisSigner::read_from_file(genesis_secret_key_path)?; + let registry_json = read_to_string(to_sign_registry_path).with_context(|| { + format!( + "Failed to read registry file at '{}'", + to_sign_registry_path.display() + ) + })?; + let registry: CircuitVerificationKeyRegistry = serde_json::from_str(®istry_json) + .with_context(|| { + format!( + "Failed to parse registry file at '{}'", + to_sign_registry_path.display() + ) + })?; + Self::check_registry_can_be_signed(®istry)?; + + Self::sign_and_write(®istry, &genesis_signer, target_signed_registry_path) + } + + /// Create and sign the circuit verification key registry whitelisting the certificate circuit + /// key of the network protocol parameters and the IVC circuit key from epoch 0, and write the + /// signed registry JSON. For test only. + pub fn bootstrap( + genesis_secret_key: &str, + protocol_parameters: Option<&ProtocolParameters>, + target_registry_path: &Path, + ) -> StdResult<()> { + let genesis_signer = GenesisSigner::try_from_hex(genesis_secret_key) + .with_context(|| "hex decode of genesis secret key failure")?; + let digests = CircuitVerificationKeyDigests::compute(protocol_parameters)?; + let registry = CircuitVerificationKeyRegistry { + version: MINIMUM_REGISTRY_VERSION, + entries: vec![ + Self::allowed_circuit_key_entry(digests.certificate_circuit, "certificate-circuit"), + Self::allowed_circuit_key_entry(digests.ivc_circuit, "ivc-circuit"), + ], + }; + + Self::sign_and_write(®istry, &genesis_signer, target_registry_path) + } + + /// Read and verify the signed registry at the given path with the verifier of the genesis + /// signer, or return nothing when the file does not exist. + fn read_signed_registry( + registry_path: &Path, + genesis_signer: &GenesisSigner, + ) -> StdResult> { + if !registry_path.exists() { + return Ok(None); + } + let signed_registry: SignedCircuitVerificationKeyRegistry = + serde_json::from_str(&read_to_string(registry_path).with_context(|| { + format!( + "Failed to read signed registry file at '{}'", + registry_path.display() + ) + })?) + .with_context(|| { + format!( + "Failed to parse signed registry file at '{}'", + registry_path.display() + ) + })?; + let registry = signed_registry + .verify(&genesis_signer.create_verifier()) + .with_context(|| { + format!( + "The signed registry at '{}' does not verify with the given genesis key", + registry_path.display() + ) + })?; + + Ok(Some(registry)) + } + + /// Check that a registry is well formed before signing it: the version reaches the minimum + /// accepted by the nodes, and no entry has an inverted epoch range (which would silently + /// never match). + fn check_registry_can_be_signed(registry: &CircuitVerificationKeyRegistry) -> StdResult<()> { + if registry.version < MINIMUM_REGISTRY_VERSION { + return Err(anyhow!( + "The registry version {} is below the minimum accepted version {MINIMUM_REGISTRY_VERSION}", + registry.version + )); + } + for entry in ®istry.entries { + if let Some(end_epoch) = entry.end_epoch + && entry.start_epoch > end_epoch + { + return Err(anyhow!( + "The entry '{}' has an inverted epoch range ({} > {}), it would never match", + entry.name, + entry.start_epoch, + end_epoch + )); + } + } + + Ok(()) + } + + /// Sign the registry with the genesis signer, verify the produced signature and write the + /// signed registry JSON at the given path. + fn sign_and_write( + registry: &CircuitVerificationKeyRegistry, + genesis_signer: &GenesisSigner, + target_path: &Path, + ) -> StdResult<()> { + let signed_registry = + SignedCircuitVerificationKeyRegistry::try_new(registry.clone(), genesis_signer)?; + signed_registry + .verify(&genesis_signer.create_verifier()) + .with_context(|| "The produced registry signature does not verify")?; + std::fs::write(target_path, serde_json::to_string_pretty(&signed_registry)?).with_context( + || { + format!( + "Failed to write signed registry file at '{}'", + target_path.display() + ) + }, + )?; + + Ok(()) + } + + /// Build a registry entry allowing the given circuit verification key digest from epoch 0. + fn allowed_circuit_key_entry( + digest: CircuitVerificationKeyDigest, + name: &str, + ) -> CircuitVerificationKeyEntry { + CircuitVerificationKeyEntry { + digest, + name: name.to_string(), + status: CircuitVerificationKeyStatus::Allowed, + start_epoch: Epoch(0), + end_epoch: None, + comment: None, + } + } +} + +#[cfg(test)] +mod tests { + use std::path::PathBuf; + + use mithril_common::{crypto_helper::GenesisEd25519Signer, test::TempDir}; + + use super::*; + + fn get_temp_dir(dir_name: &str) -> PathBuf { + TempDir::create("circuit_key_registry", dir_name) + } + + fn write_genesis_secret_key(temp_dir: &Path) -> (PathBuf, GenesisSigner) { + let genesis_signer = GenesisSigner::create_deterministic_signer(); + let genesis_secret_key_path = temp_dir.join("genesis.sk"); + genesis_signer.write_to_file(&genesis_secret_key_path).unwrap(); + + (genesis_secret_key_path, genesis_signer) + } + + fn read_signed_registry(path: &Path) -> SignedCircuitVerificationKeyRegistry { + serde_json::from_str(&read_to_string(path).unwrap()).unwrap() + } + + fn entry(digest_byte: u8, status: CircuitVerificationKeyStatus) -> CircuitVerificationKeyEntry { + CircuitVerificationKeyEntry { + digest: hex::encode([digest_byte; 32]).parse().unwrap(), + name: format!("circuit-{digest_byte}"), + status, + start_epoch: Epoch(10), + end_epoch: None, + comment: Some("a comment".to_string()), + } + } + + mod export_digests { + use super::*; + + #[test] + fn exports_the_production_digests_without_protocol_parameters() { + let temp_dir = get_temp_dir("export_digests_production"); + let target_path = temp_dir.join("digests.json"); + + let digests = CircuitKeyRegistryTools::export_digests(None, &target_path).unwrap(); + + let expected = CircuitVerificationKeyDigests { + certificate_circuit: + CircuitVerificationKeyDigest::for_production_certificate_circuit().unwrap(), + ivc_circuit: CircuitVerificationKeyDigest::for_ivc_circuit().unwrap(), + }; + assert_eq!(expected, digests); + let exported: CircuitVerificationKeyDigests = + serde_json::from_str(&read_to_string(&target_path).unwrap()).unwrap(); + assert_eq!(expected, exported); + } + } + + mod add_entry { + use super::*; + + #[test] + fn creates_a_signed_registry_at_the_minimum_version_when_the_file_is_missing() { + let temp_dir = get_temp_dir("add_entry_creates_registry"); + let (genesis_secret_key_path, genesis_signer) = write_genesis_secret_key(&temp_dir); + let registry_path = temp_dir.join("registry.json"); + let added_entry = entry(1, CircuitVerificationKeyStatus::Allowed); + + let registry = CircuitKeyRegistryTools::add_entry( + ®istry_path, + &genesis_secret_key_path, + added_entry.clone(), + ) + .unwrap(); + + assert_eq!(MINIMUM_REGISTRY_VERSION, registry.version); + assert_eq!(vec![added_entry], registry.entries); + let verified_registry = read_signed_registry(®istry_path) + .verify(&genesis_signer.create_verifier()) + .expect("the written registry must carry a valid genesis signature"); + assert_eq!(registry, verified_registry); + } + + #[test] + fn appends_the_entry_and_increments_the_version_of_an_existing_registry() { + let temp_dir = get_temp_dir("add_entry_appends"); + let (genesis_secret_key_path, genesis_signer) = write_genesis_secret_key(&temp_dir); + let registry_path = temp_dir.join("registry.json"); + let first_entry = entry(1, CircuitVerificationKeyStatus::Allowed); + let second_entry = entry(1, CircuitVerificationKeyStatus::Revoked); + CircuitKeyRegistryTools::add_entry( + ®istry_path, + &genesis_secret_key_path, + first_entry.clone(), + ) + .unwrap(); + + let registry = CircuitKeyRegistryTools::add_entry( + ®istry_path, + &genesis_secret_key_path, + second_entry.clone(), + ) + .unwrap(); + + assert_eq!(MINIMUM_REGISTRY_VERSION + 1, registry.version); + assert_eq!(vec![first_entry, second_entry], registry.entries); + let verified_registry = read_signed_registry(®istry_path) + .verify(&genesis_signer.create_verifier()) + .expect("the written registry must carry a valid genesis signature"); + assert_eq!(registry, verified_registry); + } + + #[test] + fn fails_on_an_existing_registry_signed_by_another_genesis_key() { + let temp_dir = get_temp_dir("add_entry_rejects_other_key"); + let (genesis_secret_key_path, _) = write_genesis_secret_key(&temp_dir); + let registry_path = temp_dir.join("registry.json"); + let other_genesis_signer = GenesisSigner::from_ed25519( + GenesisEd25519Signer::create_non_deterministic_signer(), + ); + let other_signed_registry = SignedCircuitVerificationKeyRegistry::try_new( + CircuitVerificationKeyRegistry { + version: MINIMUM_REGISTRY_VERSION, + entries: vec![], + }, + &other_genesis_signer, + ) + .unwrap(); + std::fs::write( + ®istry_path, + serde_json::to_string(&other_signed_registry).unwrap(), + ) + .unwrap(); + + CircuitKeyRegistryTools::add_entry( + ®istry_path, + &genesis_secret_key_path, + entry(1, CircuitVerificationKeyStatus::Allowed), + ) + .expect_err("a registry signed by another genesis key must not be extended"); + + assert_eq!( + other_signed_registry, + read_signed_registry(®istry_path), + "the registry file must be left untouched" + ); + } + + #[test] + fn fails_on_an_entry_with_an_inverted_epoch_range() { + let temp_dir = get_temp_dir("add_entry_inverted_range"); + let (genesis_secret_key_path, _) = write_genesis_secret_key(&temp_dir); + let registry_path = temp_dir.join("registry.json"); + + CircuitKeyRegistryTools::add_entry( + ®istry_path, + &genesis_secret_key_path, + CircuitVerificationKeyEntry { + start_epoch: Epoch(20), + end_epoch: Some(Epoch(10)), + ..entry(1, CircuitVerificationKeyStatus::Allowed) + }, + ) + .expect_err("an entry with an inverted epoch range must be rejected"); + + assert!(!registry_path.exists()); + } + } + + mod sign { + use super::*; + + #[test] + fn signs_a_registry_and_writes_a_verifiable_signed_registry() { + let temp_dir = get_temp_dir("sign"); + let (genesis_secret_key_path, genesis_signer) = write_genesis_secret_key(&temp_dir); + let registry = CircuitVerificationKeyRegistry { + version: 1, + entries: vec![], + }; + let to_sign_registry_path = temp_dir.join("registry.json"); + let target_signed_registry_path = temp_dir.join("signed-registry.json"); + std::fs::write( + &to_sign_registry_path, + serde_json::to_string(®istry).unwrap(), + ) + .unwrap(); + + CircuitKeyRegistryTools::sign( + &to_sign_registry_path, + &target_signed_registry_path, + &genesis_secret_key_path, + ) + .unwrap(); + + let verified_registry = read_signed_registry(&target_signed_registry_path) + .verify(&genesis_signer.create_verifier()) + .expect("the written signed registry must carry a valid genesis signature"); + assert_eq!(registry, verified_registry); + } + + #[test] + fn fails_on_a_registry_with_an_inverted_epoch_range() { + let temp_dir = get_temp_dir("sign_inverted"); + let (genesis_secret_key_path, _) = write_genesis_secret_key(&temp_dir); + let registry = CircuitVerificationKeyRegistry { + version: 1, + entries: vec![CircuitVerificationKeyEntry { + start_epoch: Epoch(20), + end_epoch: Some(Epoch(10)), + ..entry(1, CircuitVerificationKeyStatus::Allowed) + }], + }; + let to_sign_registry_path = temp_dir.join("registry.json"); + std::fs::write( + &to_sign_registry_path, + serde_json::to_string(®istry).unwrap(), + ) + .unwrap(); + + CircuitKeyRegistryTools::sign( + &to_sign_registry_path, + &temp_dir.join("signed-registry.json"), + &genesis_secret_key_path, + ) + .expect_err("a registry with an inverted epoch range must fail signing"); + } + + #[test] + fn fails_on_a_registry_version_below_the_minimum() { + let temp_dir = get_temp_dir("sign_version"); + let (genesis_secret_key_path, _) = write_genesis_secret_key(&temp_dir); + let registry = CircuitVerificationKeyRegistry { + version: MINIMUM_REGISTRY_VERSION - 1, + entries: vec![], + }; + let to_sign_registry_path = temp_dir.join("registry.json"); + std::fs::write( + &to_sign_registry_path, + serde_json::to_string(®istry).unwrap(), + ) + .unwrap(); + + CircuitKeyRegistryTools::sign( + &to_sign_registry_path, + &temp_dir.join("signed-registry.json"), + &genesis_secret_key_path, + ) + .expect_err("a registry version below the minimum must fail signing"); + } + + #[test] + fn fails_on_an_invalid_registry_file() { + let temp_dir = get_temp_dir("sign_invalid"); + let (genesis_secret_key_path, _) = write_genesis_secret_key(&temp_dir); + let to_sign_registry_path = temp_dir.join("registry.json"); + std::fs::write(&to_sign_registry_path, "not a registry").unwrap(); + + CircuitKeyRegistryTools::sign( + &to_sign_registry_path, + &temp_dir.join("signed-registry.json"), + &genesis_secret_key_path, + ) + .expect_err("an invalid registry file must fail signing"); + } + } + + mod bootstrap { + use super::*; + + #[test] + fn bootstraps_a_verifiable_registry_whitelisting_the_production_circuit_keys_without_protocol_parameters() + { + let temp_dir = get_temp_dir("bootstrap"); + let genesis_secret_key_hex = GenesisEd25519Signer::create_deterministic_signer() + .secret_key() + .to_json_hex() + .unwrap(); + let target_registry_path = temp_dir.join("registry.json"); + + CircuitKeyRegistryTools::bootstrap( + &genesis_secret_key_hex, + None, + &target_registry_path, + ) + .unwrap(); + + let verified_registry = read_signed_registry(&target_registry_path) + .verify( + &GenesisSigner::from_ed25519( + GenesisEd25519Signer::create_deterministic_signer(), + ) + .create_verifier(), + ) + .expect("the bootstrapped registry must carry a valid genesis signature"); + assert_eq!(MINIMUM_REGISTRY_VERSION, verified_registry.version); + assert_eq!( + vec![ + ( + "certificate-circuit", + CircuitVerificationKeyDigest::for_production_certificate_circuit().unwrap() + ), + ( + "ivc-circuit", + CircuitVerificationKeyDigest::for_ivc_circuit().unwrap() + ), + ], + verified_registry + .entries + .iter() + .map(|entry| (entry.name.as_str(), entry.digest)) + .collect::>() + ); + assert!(verified_registry.entries.iter().all(|entry| { + entry.status == CircuitVerificationKeyStatus::Allowed + && entry.start_epoch == Epoch(0) + && entry.end_epoch.is_none() + })); + } + } +} diff --git a/mithril-aggregator/src/tools/genesis/operations.rs b/mithril-aggregator/src/tools/genesis/operations.rs index e39f24eac8d..78015207dc7 100644 --- a/mithril-aggregator/src/tools/genesis/operations.rs +++ b/mithril-aggregator/src/tools/genesis/operations.rs @@ -28,6 +28,10 @@ use crate::{ dependency_injection::GenesisCommandDependenciesContainer, }; #[cfg(feature = "future_snark")] +use mithril_common::crypto_helper::{ + CircuitVerificationKeyRegistryRetriever, MithrilCircuitVerificationKeyCertifier, +}; +#[cfg(feature = "future_snark")] use mithril_common::crypto_helper::{ GenesisBundleError, GenesisEd25519SecretKey, GenesisSchnorrSigner, GenesisSigningKeyBundle, GenesisVerificationKeyBundle, ProtocolKey, sha256_digest, signed_message_from_digest, @@ -55,6 +59,8 @@ pub struct GenesisToolsConfiguration { pub struct GenesisTools { configuration: GenesisToolsConfiguration, certificate_repository: Arc, + #[cfg(feature = "future_snark")] + circuit_verification_key_registry_retriever: Arc, logger: Logger, } @@ -63,11 +69,16 @@ impl GenesisTools { pub fn new( configuration: GenesisToolsConfiguration, certificate_repository: Arc, + #[cfg(feature = "future_snark")] circuit_verification_key_registry_retriever: Arc< + dyn CircuitVerificationKeyRegistryRetriever, + >, logger: Logger, ) -> Self { Self { configuration, certificate_repository, + #[cfg(feature = "future_snark")] + circuit_verification_key_registry_retriever, logger, } } @@ -82,6 +93,11 @@ impl GenesisTools { self.logger.clone(), self.certificate_repository.clone(), Arc::new(genesis_verifier.clone()), + #[cfg(feature = "future_snark")] + Arc::new(MithrilCircuitVerificationKeyCertifier::new( + self.circuit_verification_key_registry_retriever.clone(), + Arc::new(genesis_verifier.clone()), + )), ) } @@ -125,6 +141,8 @@ impl GenesisTools { Ok(Self::new( configuration, certificate_repository, + #[cfg(feature = "future_snark")] + dependencies.circuit_verification_key_registry_retriever, dependencies.logger, )) } @@ -466,6 +484,11 @@ mod tests { test::{TempDir, builder::MithrilFixtureBuilder, double::fake_data}, }; + #[cfg(feature = "future_snark")] + use mithril_common::test::double::{ + FakeCircuitVerificationKeyCertifier, FakeCircuitVerificationKeyRegistryRetriever, + }; + use crate::database::test_helper::main_db_connection; use crate::test::TestLogger; @@ -512,6 +535,8 @@ mod tests { TestLogger::stdout(), certificate_store.clone(), genesis_verifier.clone(), + #[cfg(feature = "future_snark")] + Arc::new(FakeCircuitVerificationKeyCertifier::that_fails()), )); let configuration = GenesisToolsConfiguration { network: fake_data::network(), @@ -523,6 +548,8 @@ mod tests { let genesis_tools = GenesisTools::new( configuration, certificate_store.clone(), + #[cfg(feature = "future_snark")] + Arc::new(FakeCircuitVerificationKeyRegistryRetriever::that_fails()), TestLogger::stdout(), ); diff --git a/mithril-aggregator/src/tools/mod.rs b/mithril-aggregator/src/tools/mod.rs index 6387f9b2c70..c1635c87a96 100644 --- a/mithril-aggregator/src/tools/mod.rs +++ b/mithril-aggregator/src/tools/mod.rs @@ -1,4 +1,6 @@ mod certificates_hash_migrator; +#[cfg(feature = "future_snark")] +mod circuit_key_registry; mod era; mod genesis; pub mod kubo_rpc_client; @@ -9,6 +11,8 @@ pub mod url_sanitizer; mod vacuum_tracker; pub use certificates_hash_migrator::CertificatesHashMigrator; +#[cfg(feature = "future_snark")] +pub use circuit_key_registry::CircuitKeyRegistryTools; pub use era::EraTools; #[cfg(feature = "future_snark")] pub use genesis::GenesisSignedPayload; diff --git a/mithril-client-cli/src/command_context.rs b/mithril-client-cli/src/command_context.rs index e58d4d4e559..87cbf2e3153 100644 --- a/mithril-client-cli/src/command_context.rs +++ b/mithril-client-cli/src/command_context.rs @@ -1,8 +1,12 @@ use anyhow::anyhow; use slog::Logger; +#[cfg(feature = "future_snark")] +use std::path::PathBuf; use std::str::FromStr; use std::sync::Arc; +#[cfg(feature = "future_snark")] +use mithril_client::circuit_key_registry::FileCircuitVerificationKeyRegistryRetriever; use mithril_client::{ AggregatorDiscoveryType, ClientBuilder, GenesisVerificationKey, MithrilResult, }; @@ -128,6 +132,17 @@ impl CommandContext { builder = builder.with_era_fetcher(Arc::new(ForcedEraFetcher::new(era.to_string()))); } + #[cfg(feature = "future_snark")] + if let Some(registry_path) = params.get("circuit_verification_key_registry_path") { + self.require_unstable( + "--circuit-verification-key-registry-path ", + Some("cardano-db download latest"), + )?; + builder = builder.with_circuit_verification_key_registry_retriever(Arc::new( + FileCircuitVerificationKeyRegistryRetriever::new(PathBuf::from(registry_path)), + )); + } + Ok(builder) } } @@ -169,6 +184,46 @@ mod tests { assert!(result.is_err(), "Expected Err, got {result:?}"); } + #[cfg(feature = "future_snark")] + mod circuit_verification_key_registry_path { + use super::*; + + fn context_with_registry_path(unstable_enabled: bool) -> CommandContext { + CommandContext::new( + ConfigParameters::build(&[ + ( + "aggregator_endpoint", + "https://aggregator.example/aggregator", + ), + ("genesis_verification_key", "whatever"), + ( + "circuit_verification_key_registry_path", + "./circuit-verification-key-registry.json", + ), + ]), + unstable_enabled, + true, + Logger::root(slog::Discard, o!()), + ) + } + + #[test] + fn is_refused_without_the_unstable_flag() { + context_with_registry_path(false) + .setup_mithril_client_builder() + .map(|_| ()) + .expect_err("the registry path must require the unstable flag"); + } + + #[test] + fn is_accepted_with_the_unstable_flag() { + context_with_registry_path(true) + .setup_mithril_client_builder() + .map(|_| ()) + .expect("the registry path must be accepted with the unstable flag"); + } + } + #[test] fn can_edit_config_parameters() { struct ParamSource { diff --git a/mithril-client-cli/src/main.rs b/mithril-client-cli/src/main.rs index 575d1ba1858..29e949c2178 100644 --- a/mithril-client-cli/src/main.rs +++ b/mithril-client-cli/src/main.rs @@ -95,6 +95,13 @@ pub struct Args { #[clap(long, global = true)] origin_tag: Option, + /// Read the circuit verification key registry from a local signed registry file instead of + /// resolving it through the published networks configuration (unstable, for local deployments) + #[cfg(feature = "future_snark")] + #[clap(long, env = "CIRCUIT_VERIFICATION_KEY_REGISTRY_PATH", global = true)] + #[example = "`./circuit-verification-key-registry.json`"] + circuit_verification_key_registry_path: Option, + /// Override the Mithril era #[clap(long, global = true)] #[example = "`pythagoras`"] @@ -217,6 +224,12 @@ impl Source for Args { register_config_value_option!(map, &namespace, myself.aggregator_endpoint); register_config_value_option!(map, &namespace, myself.origin_tag); register_config_value_option!(map, &namespace, myself.era); + #[cfg(feature = "future_snark")] + register_config_value_option!( + map, + &namespace, + myself.circuit_verification_key_registry_path + ); Ok(map) } diff --git a/mithril-client/src/certificate_client/mod.rs b/mithril-client/src/certificate_client/mod.rs index c345179f34f..dbcb4d319c1 100644 --- a/mithril-client/src/certificate_client/mod.rs +++ b/mithril-client/src/certificate_client/mod.rs @@ -70,6 +70,8 @@ pub use verify_cache::MemoryCertificateVerifierCache; pub(crate) mod tests_utils { use mithril_common::entities::Certificate; use mithril_common::messages::CertificateMessage; + #[cfg(feature = "future_snark")] + use mithril_common::test::double::FakeCircuitVerificationKeyRegistryRetriever; use mockall::predicate::eq; use std::sync::Arc; @@ -136,6 +138,8 @@ pub(crate) mod tests_utils { FeedbackSender::new(&self.feedback_receivers), #[cfg(feature = "unstable")] self.verifier_cache, + #[cfg(feature = "future_snark")] + Arc::new(FakeCircuitVerificationKeyRegistryRetriever::that_fails()), logger.clone(), ) .unwrap(), diff --git a/mithril-client/src/certificate_client/verify.rs b/mithril-client/src/certificate_client/verify.rs index 230ca8cef25..d99d0217d27 100644 --- a/mithril-client/src/certificate_client/verify.rs +++ b/mithril-client/src/certificate_client/verify.rs @@ -15,6 +15,12 @@ use mithril_common::{ logging::LoggerExtensions, }; +#[cfg(feature = "future_snark")] +use mithril_common::crypto_helper::{ + CachedCircuitVerificationKeyCertifier, CircuitVerificationKeyRegistryRetriever, + MithrilCircuitVerificationKeyCertifier, +}; + #[cfg(feature = "unstable")] use crate::certificate_client::CertificateVerifierCache; use crate::certificate_client::fetch::InternalCertificateRetriever; @@ -60,6 +66,9 @@ impl MithrilCertificateVerifier { genesis_verification_key: &str, feedback_sender: FeedbackSender, #[cfg(feature = "unstable")] verifier_cache: Option>, + #[cfg(feature = "future_snark")] circuit_key_registry_retriever: Arc< + dyn CircuitVerificationKeyRegistryRetriever, + >, logger: Logger, ) -> MithrilResult { let logger = logger.new_with_component_name::(); @@ -71,7 +80,14 @@ impl MithrilCertificateVerifier { let internal_verifier = Arc::new(CommonMithrilCertificateVerifier::new( logger.clone(), retriever.clone(), - genesis_verifier, + genesis_verifier.clone(), + #[cfg(feature = "future_snark")] + Arc::new(CachedCircuitVerificationKeyCertifier::new(Arc::new( + MithrilCircuitVerificationKeyCertifier::new( + circuit_key_registry_retriever, + genesis_verifier, + ), + ))), )); Ok(Self { @@ -244,6 +260,8 @@ impl CertificateVerifier for MithrilCertificateVerifier { #[cfg(test)] mod tests { use mithril_common::test::builder::CertificateChainBuilder; + #[cfg(feature = "future_snark")] + use mithril_common::test::double::FakeCircuitVerificationKeyRegistryRetriever; use crate::certificate_client::tests_utils::CertificateClientTestBuilder; use crate::certificate_client::{ @@ -334,6 +352,8 @@ mod tests { FeedbackSender::new(&[]), #[cfg(feature = "unstable")] None, + #[cfg(feature = "future_snark")] + Arc::new(FakeCircuitVerificationKeyRegistryRetriever::that_fails()), TestLogger::stdout(), ) .map(|_| ()) @@ -470,6 +490,8 @@ mod tests { &genesis_verification_key, FeedbackSender::new(&[]), Some(cache), + #[cfg(feature = "future_snark")] + Arc::new(FakeCircuitVerificationKeyRegistryRetriever::that_fails()), TestLogger::stdout(), ) .unwrap() diff --git a/mithril-client/src/circuit_key_registry.rs b/mithril-client/src/circuit_key_registry.rs new file mode 100644 index 00000000000..353e5d85832 --- /dev/null +++ b/mithril-client/src/circuit_key_registry.rs @@ -0,0 +1,475 @@ +//! Retrieval of the signed circuit verification key registry of the client's network, resolved +//! from the networks configuration file published in the Mithril repository. + +use std::collections::HashMap; + +use anyhow::{Context, anyhow}; +use async_trait::async_trait; +use serde::Deserialize; + +use mithril_common::StdResult; +#[cfg(not(target_family = "wasm"))] +pub use mithril_common::crypto_helper::FileCircuitVerificationKeyRegistryRetriever; +use mithril_common::crypto_helper::{ + CircuitVerificationKeyRegistryRetriever, CircuitVerificationKeyRegistryRetrieverError, + SignedCircuitVerificationKeyRegistry, +}; + +/// URL of the networks configuration file published in the Mithril repository. +pub const DEFAULT_NETWORKS_CONFIGURATION_URL: &str = + "https://raw.githubusercontent.com/IntersectMBO/mithril/main/networks.json"; + +const DOWNLOAD_MAX_ATTEMPTS: usize = 3; + +#[cfg(not(target_family = "wasm"))] +const DOWNLOAD_RETRY_DELAY_IN_MILLISECONDS: u64 = 1000; + +#[cfg(not(target_family = "wasm"))] +const DOWNLOAD_TIMEOUT_IN_SECONDS: u64 = 10; + +const DOWNLOAD_MAX_BODY_SIZE_IN_BYTES: u64 = 1024 * 1024; + +/// Representation of a Cardano network environment in the networks configuration file. +#[derive(Debug, Clone, Deserialize)] +struct CardanoNetworkConfiguration { + /// Mithril networks of the environment, keyed by their name. + #[serde(rename = "mithril-networks", default)] + mithril_networks: Vec>, +} + +/// Representation of a Mithril network in the networks configuration file. +#[derive(Debug, Clone, Deserialize)] +struct MithrilNetworkConfiguration { + /// Aggregators serving the network. + #[serde(default)] + aggregators: Vec, + + /// Reference to the signed circuit verification key registry of the network. + #[serde(rename = "circuit-verification-key-registry")] + circuit_verification_key_registry: Option, +} + +impl MithrilNetworkConfiguration { + /// Whether one of the network's aggregators serves the given endpoint. + fn is_served_by(&self, aggregator_endpoint: &str) -> bool { + self.aggregators.iter().any(|aggregator| { + Self::normalize_endpoint(&aggregator.url) + == Self::normalize_endpoint(aggregator_endpoint) + }) + } + + /// Return the registry URL referenced by the network. + fn registry_url(&self) -> Option { + self.circuit_verification_key_registry + .as_ref() + .map(|registry| registry.url.clone()) + } + + /// Normalize an aggregator endpoint for comparison, ignoring surrounding whitespace and + /// trailing slashes. + fn normalize_endpoint(endpoint: &str) -> &str { + endpoint.trim().trim_end_matches('/') + } +} + +/// Representation of an aggregator in the networks configuration file. +#[derive(Debug, Clone, Deserialize)] +struct AggregatorConfiguration { + /// Endpoint of the aggregator. + url: String, +} + +/// Reference to a remote resource by URL in the networks configuration file. +#[derive(Debug, Clone, Deserialize)] +struct UrlReference { + /// URL of the resource. + url: String, +} + +/// The Mithril networks listed in the networks configuration file. +struct MithrilNetworksConfiguration { + networks: Vec, +} + +impl MithrilNetworksConfiguration { + /// Parse the networks configuration file, tolerating top-level entries that do not follow the + /// Cardano network environment shape, so a future metadata field cannot fail the whole + /// resolution. + fn parse(networks_configuration_json: &str) -> StdResult { + let root: HashMap = + serde_json::from_str(networks_configuration_json)?; + let networks = root + .into_values() + .filter_map(|cardano_network| { + serde_json::from_value::(cardano_network).ok() + }) + .flat_map(|cardano_network| cardano_network.mithril_networks) + .flat_map(|mithril_networks| mithril_networks.into_values()) + .collect(); + + Ok(Self { networks }) + } + + /// Return the registry URL of the network served by the given aggregator endpoint. + fn registry_url_of_aggregator(&self, aggregator_endpoint: &str) -> Option { + self.networks + .iter() + .find(|network| network.is_served_by(aggregator_endpoint)) + .and_then(MithrilNetworkConfiguration::registry_url) + } +} + +/// HTTP downloader of the documents involved in the registry resolution, bounding the request +/// duration and the response size, and retrying failed attempts. +struct BoundedHttpDownloader { + client: reqwest::Client, +} + +impl BoundedHttpDownloader { + /// Build a downloader with a request timeout, so a hung download cannot stall certificate + /// verification. + #[cfg(not(target_family = "wasm"))] + fn new() -> Self { + Self { + client: reqwest::Client::builder() + .timeout(std::time::Duration::from_secs(DOWNLOAD_TIMEOUT_IN_SECONDS)) + .build() + .unwrap_or_default(), + } + } + + /// Build a downloader relying on the browser to bound the request duration, as the request + /// timeout builder is not available on WASM. + #[cfg(target_family = "wasm")] + fn new() -> Self { + Self { + client: reqwest::Client::new(), + } + } + + /// Download the document at the given URL, retrying failed attempts up to + /// [DOWNLOAD_MAX_ATTEMPTS] times. + async fn download_with_retry(&self, url: &str) -> StdResult { + let mut attempts = 0; + loop { + attempts += 1; + match self.download(url).await { + Ok(document) => return Ok(document), + Err(_) if attempts < DOWNLOAD_MAX_ATTEMPTS => Self::wait_before_retry().await, + Err(error) => { + return Err(error.context(format!( + "Failed to download '{url}' after {DOWNLOAD_MAX_ATTEMPTS} attempts" + ))); + } + } + } + } + + /// Download the document at the given URL, failing on a non success status or a response + /// exceeding the size limit. + async fn download(&self, url: &str) -> StdResult { + let response = self + .client + .get(url) + .send() + .await + .with_context(|| format!("Failed to download '{url}'"))?; + if !response.status().is_success() { + return Err(anyhow!( + "Failed to download '{url}': status {}", + response.status() + )); + } + Self::check_size_limit(url, response.content_length().unwrap_or_default())?; + let document = response + .text() + .await + .with_context(|| format!("Failed to read the response of '{url}'"))?; + Self::check_size_limit(url, document.len() as u64)?; + + Ok(document) + } + + /// Fail when the response size exceeds [DOWNLOAD_MAX_BODY_SIZE_IN_BYTES], the memory bound + /// on the documents served by the untrusted routing URLs. + fn check_size_limit(url: &str, size_in_bytes: u64) -> StdResult<()> { + if size_in_bytes > DOWNLOAD_MAX_BODY_SIZE_IN_BYTES { + return Err(anyhow!( + "Failed to download '{url}': response of {size_in_bytes} bytes exceeds the {DOWNLOAD_MAX_BODY_SIZE_IN_BYTES} bytes limit" + )); + } + + Ok(()) + } + + /// Wait for [DOWNLOAD_RETRY_DELAY_IN_MILLISECONDS] before the next download attempt. + #[cfg(not(target_family = "wasm"))] + async fn wait_before_retry() { + tokio::time::sleep(std::time::Duration::from_millis( + DOWNLOAD_RETRY_DELAY_IN_MILLISECONDS, + )) + .await; + } + + /// Retry immediately: no timer is available on WASM. + #[cfg(target_family = "wasm")] + async fn wait_before_retry() {} +} + +/// A [CircuitVerificationKeyRegistryRetriever] resolving the signed registry of the client's +/// network from the networks configuration file, then downloading it over HTTP. +/// +/// The network entry is selected by matching the aggregator endpoint. The networks +/// configuration is pure routing, never trust: a wrong selection can only yield a registry that +/// fails the genesis signature verification of the certifier. +pub struct RemoteCircuitVerificationKeyRegistryRetriever { + networks_configuration_url: String, + aggregator_endpoint: String, + downloader: BoundedHttpDownloader, +} + +impl RemoteCircuitVerificationKeyRegistryRetriever { + /// Build a retriever for the network served by the given aggregator endpoint. + pub fn new(aggregator_endpoint: String) -> Self { + Self::new_with_networks_configuration_url( + DEFAULT_NETWORKS_CONFIGURATION_URL.to_string(), + aggregator_endpoint, + ) + } + + /// Build a retriever resolving from the given networks configuration file URL. + pub fn new_with_networks_configuration_url( + networks_configuration_url: String, + aggregator_endpoint: String, + ) -> Self { + Self { + networks_configuration_url, + aggregator_endpoint, + downloader: BoundedHttpDownloader::new(), + } + } + + /// Resolve the registry URL of the client's network from the networks configuration file, + /// then download and parse the signed registry. + async fn resolve_and_download_registry( + &self, + ) -> StdResult { + let networks_configuration_json = self + .downloader + .download_with_retry(&self.networks_configuration_url) + .await?; + let networks_configuration = MithrilNetworksConfiguration::parse( + &networks_configuration_json, + ) + .with_context(|| { + format!( + "Failed to parse networks configuration downloaded from '{}'", + self.networks_configuration_url + ) + })?; + let registry_url = networks_configuration + .registry_url_of_aggregator(&self.aggregator_endpoint) + .ok_or_else(|| { + anyhow!( + "No circuit verification key registry is referenced in '{}' for the network of aggregator '{}'", + self.networks_configuration_url, + self.aggregator_endpoint + ) + })?; + let registry_json = self.downloader.download_with_retry(®istry_url).await?; + + serde_json::from_str(®istry_json).with_context(|| { + format!("Failed to parse signed registry downloaded from '{registry_url}'") + }) + } +} + +#[cfg_attr(target_family = "wasm", async_trait(?Send))] +#[cfg_attr(not(target_family = "wasm"), async_trait)] +impl CircuitVerificationKeyRegistryRetriever for RemoteCircuitVerificationKeyRegistryRetriever { + async fn retrieve_signed_registry( + &self, + ) -> Result + { + self.resolve_and_download_registry() + .await + .map_err(CircuitVerificationKeyRegistryRetrieverError) + } +} + +#[cfg(all(test, not(target_family = "wasm")))] +mod tests { + use httpmock::MockServer; + + use mithril_common::crypto_helper::{ + CircuitVerificationKeyRegistry, GenesisEd25519Signer, GenesisSigner, + }; + + use super::*; + + const AGGREGATOR_ENDPOINT: &str = "https://aggregator.devnet.example/aggregator"; + + fn genesis_signer() -> GenesisSigner { + GenesisSigner::from_ed25519(GenesisEd25519Signer::create_deterministic_signer()) + } + + fn signed_registry(genesis_signer: &GenesisSigner) -> SignedCircuitVerificationKeyRegistry { + SignedCircuitVerificationKeyRegistry::try_new( + CircuitVerificationKeyRegistry { + version: 1, + entries: vec![], + }, + genesis_signer, + ) + .unwrap() + } + + fn networks_configuration_json(server: &MockServer, aggregator_endpoint: &str) -> String { + format!( + r#"{{ + "devnet": {{ + "mithril-networks": [ + {{ + "release-devnet": {{ + "aggregators": [{{ "url": "{aggregator_endpoint}" }}], + "circuit-verification-key-registry": {{ "url": "{registry_url}" }} + }} + }} + ] + }} + }}"#, + registry_url = server.url("/registry.json"), + ) + } + + fn retriever_over( + server: &MockServer, + aggregator_endpoint: &str, + ) -> RemoteCircuitVerificationKeyRegistryRetriever { + RemoteCircuitVerificationKeyRegistryRetriever::new_with_networks_configuration_url( + server.url("/networks.json"), + aggregator_endpoint.to_string(), + ) + } + + #[tokio::test] + async fn resolves_the_registry_of_the_network_matching_the_aggregator_endpoint() { + let server = MockServer::start(); + let genesis_signer = genesis_signer(); + let signed_registry = signed_registry(&genesis_signer); + server.mock(|when, then| { + when.method(httpmock::Method::GET).path("/networks.json"); + then.status(200) + .body(networks_configuration_json(&server, AGGREGATOR_ENDPOINT)); + }); + server.mock(|when, then| { + when.method(httpmock::Method::GET).path("/registry.json"); + then.status(200) + .body(serde_json::to_string(&signed_registry).unwrap()); + }); + + let retrieved = retriever_over(&server, AGGREGATOR_ENDPOINT) + .retrieve_signed_registry() + .await + .unwrap(); + + assert_eq!(signed_registry, retrieved); + } + + #[tokio::test] + async fn tolerates_top_level_metadata_entries_in_the_networks_configuration() { + let server = MockServer::start(); + let genesis_signer = genesis_signer(); + let signed_registry = signed_registry(&genesis_signer); + let networks_configuration_with_metadata = format!( + r#"{{ "version": 2, {} }}"#, + networks_configuration_json(&server, AGGREGATOR_ENDPOINT) + .trim() + .trim_start_matches('{') + .trim_end_matches('}') + ); + server.mock(|when, then| { + when.method(httpmock::Method::GET).path("/networks.json"); + then.status(200).body(networks_configuration_with_metadata); + }); + server.mock(|when, then| { + when.method(httpmock::Method::GET).path("/registry.json"); + then.status(200) + .body(serde_json::to_string(&signed_registry).unwrap()); + }); + + let retrieved = retriever_over(&server, AGGREGATOR_ENDPOINT) + .retrieve_signed_registry() + .await + .unwrap(); + + assert_eq!(signed_registry, retrieved); + } + + #[tokio::test] + async fn fails_on_a_response_exceeding_the_body_size_limit_before_using_it() { + let server = MockServer::start(); + let oversized_networks_configuration = format!( + "{}{}", + networks_configuration_json(&server, AGGREGATOR_ENDPOINT), + " ".repeat(DOWNLOAD_MAX_BODY_SIZE_IN_BYTES as usize) + ); + server.mock(|when, then| { + when.method(httpmock::Method::GET).path("/networks.json"); + then.status(200).body(oversized_networks_configuration); + }); + let registry = server.mock(|when, then| { + when.method(httpmock::Method::GET).path("/registry.json"); + then.status(200); + }); + + retriever_over(&server, AGGREGATOR_ENDPOINT) + .retrieve_signed_registry() + .await + .expect_err("an oversized response must fail retrieval"); + + assert_eq!(0, registry.hits()); + } + + #[tokio::test] + async fn fails_when_no_network_is_served_by_the_aggregator_endpoint() { + let server = MockServer::start(); + let registry = server.mock(|when, then| { + when.method(httpmock::Method::GET).path("/registry.json"); + then.status(200); + }); + server.mock(|when, then| { + when.method(httpmock::Method::GET).path("/networks.json"); + then.status(200) + .body(networks_configuration_json(&server, AGGREGATOR_ENDPOINT)); + }); + + retriever_over(&server, "https://another-network.example/aggregator") + .retrieve_signed_registry() + .await + .expect_err("an unmatched aggregator endpoint must fail retrieval"); + + assert_eq!(0, registry.hits()); + } + + #[tokio::test] + async fn retries_a_failed_download() { + let server = MockServer::start(); + server.mock(|when, then| { + when.method(httpmock::Method::GET).path("/networks.json"); + then.status(200) + .body(networks_configuration_json(&server, AGGREGATOR_ENDPOINT)); + }); + let failing_registry = server.mock(|when, then| { + when.method(httpmock::Method::GET).path("/registry.json"); + then.status(500); + }); + + retriever_over(&server, AGGREGATOR_ENDPOINT) + .retrieve_signed_registry() + .await + .expect_err("a persistently failing download must fail retrieval"); + + assert_eq!(DOWNLOAD_MAX_ATTEMPTS, failing_registry.hits()); + } +} diff --git a/mithril-client/src/client.rs b/mithril-client/src/client.rs index b588000412d..c9e3bd5362d 100644 --- a/mithril-client/src/client.rs +++ b/mithril-client/src/client.rs @@ -20,6 +20,8 @@ use mithril_aggregator_discovery::{ CapableAggregatorDiscoverer, HttpConfigAggregatorDiscoverer, RequiredAggregatorCapabilities, ShuffleAggregatorDiscoverer, }; +#[cfg(feature = "future_snark")] +use mithril_common::crypto_helper::CircuitVerificationKeyRegistryRetriever; use mithril_common::{MITHRIL_CLIENT_TYPE_HEADER, MITHRIL_ORIGIN_TAG_HEADER}; use crate::MithrilResult; @@ -35,6 +37,8 @@ use crate::certificate_client::CertificateVerifierCache; use crate::certificate_client::{ CertificateClient, CertificateVerifier, MithrilCertificateVerifier, }; +#[cfg(feature = "future_snark")] +use crate::circuit_key_registry::RemoteCircuitVerificationKeyRegistryRetriever; #[cfg(not(target_family = "wasm"))] use crate::common::MithrilNetwork; use crate::era::{EraFetcher, MithrilEraClient}; @@ -228,6 +232,8 @@ pub struct ClientBuilder { ipfs_rpc_base_url: Option, #[cfg(feature = "unstable")] certificate_verifier_cache: Option>, + #[cfg(feature = "future_snark")] + circuit_key_registry_retriever: Option>, era_fetcher: Option>, logger: Option, feedback_receivers: Vec>, @@ -281,6 +287,8 @@ impl ClientBuilder { ipfs_rpc_base_url: None, #[cfg(feature = "unstable")] certificate_verifier_cache: None, + #[cfg(feature = "future_snark")] + circuit_key_registry_retriever: None, era_fetcher: None, logger: None, feedback_receivers: vec![], @@ -341,13 +349,22 @@ impl ClientBuilder { let feedback_sender = FeedbackSender::new(&self.feedback_receivers); - let aggregator_client = Arc::new(self.build_aggregator_client(logger.clone())?); + let aggregator_endpoint = self.resolve_aggregator_endpoint()?; + let aggregator_client = + Arc::new(self.build_aggregator_client(aggregator_endpoint.clone(), logger.clone())?); let mithril_era_client = match self.era_fetcher { None => Arc::new(MithrilEraClient::new(aggregator_client.clone())), Some(era_fetcher) => Arc::new(MithrilEraClient::new(era_fetcher)), }; + #[cfg(feature = "future_snark")] + let circuit_key_registry_retriever = match self.circuit_key_registry_retriever { + Some(circuit_key_registry_retriever) => circuit_key_registry_retriever, + None => Arc::new(RemoteCircuitVerificationKeyRegistryRetriever::new( + aggregator_endpoint.clone(), + )), + }; let certificate_verifier = match self.certificate_verifier { None => Arc::new( MithrilCertificateVerifier::new( @@ -356,6 +373,8 @@ impl ClientBuilder { feedback_sender.clone(), #[cfg(feature = "unstable")] self.certificate_verifier_cache, + #[cfg(feature = "future_snark")] + circuit_key_registry_retriever, logger.clone(), ) .with_context(|| "Building certificate verifier failed")?, @@ -507,16 +526,24 @@ impl ClientBuilder { )) } - fn build_aggregator_client(&self, logger: Logger) -> MithrilResult { - let aggregator_endpoint = match self.aggregator_discovery { - AggregatorDiscoveryType::Url(ref url) => url.clone(), + /// Resolve the aggregator endpoint from the configured URL or through discovery. + fn resolve_aggregator_endpoint(&self) -> MithrilResult { + match self.aggregator_discovery { + AggregatorDiscoveryType::Url(ref url) => Ok(url.clone()), #[cfg(not(target_family = "wasm"))] - AggregatorDiscoveryType::Automatic(ref network) => self + AggregatorDiscoveryType::Automatic(ref network) => Ok(self .discover_aggregator(network)? .next() .with_context(|| "No aggregator was available through discovery")? - .into(), - }; + .into()), + } + } + + fn build_aggregator_client( + &self, + aggregator_endpoint: String, + logger: Logger, + ) -> MithrilResult { let headers = self.compute_http_headers(); AggregatorHttpClient::builder(aggregator_endpoint) @@ -570,6 +597,18 @@ impl ClientBuilder { } } + /// Set a custom registry retriever for the circuit verification key registry check on + /// certificate verification, replacing the default one resolving the registry of the + /// client's network from the published networks configuration. + #[cfg(feature = "future_snark")] + pub fn with_circuit_verification_key_registry_retriever( + mut self, + circuit_key_registry_retriever: Arc, + ) -> ClientBuilder { + self.circuit_key_registry_retriever = Some(circuit_key_registry_retriever); + self + } + cfg_fs! { /// Set the [FileDownloader] that will be used to download artifacts with HTTP. pub fn with_http_file_downloader( diff --git a/mithril-client/src/lib.rs b/mithril-client/src/lib.rs index 0de029a6c4c..8da6e9f1f4e 100644 --- a/mithril-client/src/lib.rs +++ b/mithril-client/src/lib.rs @@ -185,6 +185,8 @@ cfg_unstable! { pub mod cardano_transaction_v2_client; } pub mod certificate_client; +#[cfg(feature = "future_snark")] +pub mod circuit_key_registry; mod client; pub mod era; pub mod feedback; diff --git a/mithril-common/src/certificate_chain/certificate_verifier.rs b/mithril-common/src/certificate_chain/certificate_verifier.rs index 563485a7ece..5d86ebbda01 100644 --- a/mithril-common/src/certificate_chain/certificate_verifier.rs +++ b/mithril-common/src/certificate_chain/certificate_verifier.rs @@ -11,7 +11,9 @@ use mithril_stm::{AggregateSignatureType, AncillaryVerifierData}; use crate::StdResult; #[cfg(feature = "future_snark")] -use crate::crypto_helper::ProtocolAggregateVerificationKeyForSnark; +use crate::crypto_helper::{ + CircuitVerificationKeyCertifier, ProtocolAggregateVerificationKeyForSnark, +}; use crate::crypto_helper::{ GenesisEd25519Error, GenesisVerifier, ProtocolAggregateVerificationKey, ProtocolAggregateVerificationKeyForConcatenation, ProtocolMultiSignature, @@ -90,6 +92,14 @@ pub enum CertificateVerifierError { /// certificate that's not a standard certificate. #[error("can't validate standard certificate: given certificate isn't a standard certificate")] InvalidStandardCertificateProvided, + + /// Error raised when a certificate whose aggregate signature type requires certified circuit + /// verification keys carries no ancillary verifier data. + #[cfg(feature = "future_snark")] + #[error( + "certificate is missing the ancillary verifier data carrying its circuit verification keys" + )] + MissingAncillaryVerifierData, } /// CertificateVerifier is the cryptographic engine in charge of verifying multi signatures and @@ -128,20 +138,30 @@ pub struct MithrilCertificateVerifier { logger: Logger, certificate_retriever: Arc, genesis_verifier: Arc, + #[cfg(feature = "future_snark")] + circuit_verification_key_certifier: Arc, } impl MithrilCertificateVerifier { /// MithrilCertificateVerifier factory + /// + /// The circuit verification key certifier enforces the circuit verification key registry on + /// the certificates whose aggregate signature type requires it. pub fn new( logger: Logger, certificate_retriever: Arc, genesis_verifier: Arc, + #[cfg(feature = "future_snark")] circuit_verification_key_certifier: Arc< + dyn CircuitVerificationKeyCertifier, + >, ) -> Self { debug!(logger, "New MithrilCertificateVerifier created"); Self { logger: logger.new_with_component_name::(), certificate_retriever, genesis_verifier, + #[cfg(feature = "future_snark")] + circuit_verification_key_certifier, } } @@ -188,6 +208,44 @@ impl MithrilCertificateVerifier { .map_err(|e| CertificateVerifierError::VerifyMultiSignature(e.to_string())) } + /// Verify that the circuit verification keys carried by the certificate are certified by the + /// genesis-signed registry, for the aggregate signature types that require it. + /// + /// The digests are taken from the ancillary verifier data the aggregate signature is verified + /// against, so certifying them certifies the circuits used to produce the signature. + #[cfg(feature = "future_snark")] + async fn verify_certified_circuit_verification_keys( + &self, + certificate: &Certificate, + ) -> StdResult<()> { + let requires_certification = certificate.signature.aggregate_signature_type().is_some_and( + |aggregate_signature_type| { + aggregate_signature_type.requires_certified_circuit_verification_keys() + }, + ); + if !requires_certification { + return Ok(()); + } + + let ancillary_verifier_data = certificate + .ancillary_verifier_data + .as_ref() + .ok_or(CertificateVerifierError::MissingAncillaryVerifierData)?; + + self.circuit_verification_key_certifier + .check( + &ancillary_verifier_data.circuit_verification_key_digests(), + certificate.epoch, + ) + .await + .with_context(|| { + format!( + "Certificate verifier failed certifying the circuit verification keys of certificate '{}'", + certificate.hash + ) + }) + } + /// Verify the parts of a standard certificate that do not depend on its predecessor: its hash, /// signed message, multi-signature and epoch. fn verify_standard_certificate_integrity(&self, certificate: &Certificate) -> StdResult<()> { @@ -462,6 +520,8 @@ impl CertificateVerifier for MithrilCertificateVerifier { certificate: &Certificate, previous_certificate: &Certificate, ) -> StdResult<()> { + #[cfg(feature = "future_snark")] + self.verify_certified_circuit_verification_keys(certificate).await?; self.verify_standard_certificate_integrity(certificate)?; self.verify_epoch_chaining(certificate, previous_certificate)?; self.verify_previous_hash_matches_previous_certificate_hash( @@ -498,6 +558,8 @@ impl CertificateVerifier for MithrilCertificateVerifier { if certificate.signature.aggregate_signature_type().is_some_and( |aggregate_signature_type| aggregate_signature_type.certifies_full_certificate_chain(), ) { + #[cfg(feature = "future_snark")] + self.verify_certified_circuit_verification_keys(certificate).await?; self.verify_standard_certificate_integrity(certificate)?; return Ok(None); @@ -520,6 +582,8 @@ mod tests { use mithril_stm::{AggregateSignatureType, AncillaryProofInput}; + #[cfg(feature = "future_snark")] + use crate::test::double::FakeCircuitVerificationKeyCertifier; use crate::test::{ TestLogger, builder::{CertificateChainBuilder, CertificateChainBuilderContext, MithrilFixtureBuilder}, @@ -573,6 +637,8 @@ mod tests { TestLogger::stdout(), Arc::new(self.mock_certificate_retriever), genesis_verifier, + #[cfg(feature = "future_snark")] + Arc::new(FakeCircuitVerificationKeyCertifier::that_fails()), ) } } @@ -616,6 +682,8 @@ mod tests { TestLogger::stdout(), Arc::new(MockCertificateRetriever::new()), Arc::new(genesis_verifier), + #[cfg(feature = "future_snark")] + Arc::new(FakeCircuitVerificationKeyCertifier::that_fails()), ); let message_tampered = message_hash[1..].to_vec(); assert!( @@ -1198,6 +1266,8 @@ mod tests { TestLogger::stdout(), Arc::new(certificate_retriever), Arc::new(fake_certificates.genesis_verifier.clone()), + #[cfg(feature = "future_snark")] + Arc::new(FakeCircuitVerificationKeyCertifier::that_fails()), ); let certificate_to_verify = fake_certificates[0].clone(); @@ -1218,6 +1288,8 @@ mod tests { TestLogger::stdout(), Arc::new(certificate_retriever), Arc::new(fake_certificates.genesis_verifier.clone()), + #[cfg(feature = "future_snark")] + Arc::new(FakeCircuitVerificationKeyCertifier::that_fails()), ); let certificate_to_verify = fake_certificates[0].clone(); @@ -1835,4 +1907,101 @@ mod tests { } } } + + #[cfg(feature = "future_snark")] + mod certified_circuit_verification_keys { + use crate::crypto_helper::{ + CircuitVerificationKeyCertifier, MithrilCircuitVerificationKeyCertifier, + }; + use crate::test::double::{ + FakeCircuitVerificationKeyRegistryRetriever, fake_data::snark_aggregate_signature, + }; + + use super::*; + + fn promote_to_snark_aggregate_signature_without_ancillary_data( + mut certificate: Certificate, + ) -> Certificate { + let CertificateSignature::MultiSignature(entity_type, _) = + certificate.signature.clone() + else { + panic!("certificate signature must be a multi signature"); + }; + certificate.signature = + CertificateSignature::MultiSignature(entity_type, snark_aggregate_signature()); + certificate.ancillary_verifier_data = None; + certificate + } + + fn failing_certifier() -> Arc { + Arc::new(MithrilCircuitVerificationKeyCertifier::new( + Arc::new(FakeCircuitVerificationKeyRegistryRetriever::that_fails()), + fake_genesis_verifier(), + )) + } + + #[tokio::test] + async fn skips_the_check_for_a_concatenation_certificate() { + let fake_certificates = setup_certificate_chain(2, 1); + let verifier = MithrilCertificateVerifier::new( + TestLogger::stdout(), + Arc::new(MockCertificateRetriever::new()), + Arc::new(fake_certificates.genesis_verifier.clone()), + failing_certifier(), + ); + + verifier + .verify_certified_circuit_verification_keys(&fake_certificates[0]) + .await + .expect("a concatenation certificate must not be checked against the registry"); + } + + #[tokio::test] + async fn rejects_a_snark_certificate_without_ancillary_verifier_data() { + let fake_certificates = setup_certificate_chain(2, 1); + let certificate = promote_to_snark_aggregate_signature_without_ancillary_data( + fake_certificates[0].clone(), + ); + let verifier = MithrilCertificateVerifier::new( + TestLogger::stdout(), + Arc::new(MockCertificateRetriever::new()), + Arc::new(fake_certificates.genesis_verifier.clone()), + failing_certifier(), + ); + + let error = verifier + .verify_certified_circuit_verification_keys(&certificate) + .await + .expect_err("a SNARK certificate without ancillary verifier data must be rejected"); + + assert_error_matches!( + CertificateVerifierError::MissingAncillaryVerifierData, + error + ); + } + + #[tokio::test] + async fn verify_standard_certificate_enforces_the_check_before_any_other_verification() { + let fake_certificates = setup_certificate_chain(2, 1); + let certificate = promote_to_snark_aggregate_signature_without_ancillary_data( + fake_certificates[0].clone(), + ); + let verifier = MithrilCertificateVerifier::new( + TestLogger::stdout(), + Arc::new(MockCertificateRetriever::new()), + Arc::new(fake_certificates.genesis_verifier.clone()), + failing_certifier(), + ); + + let error = verifier + .verify_standard_certificate(&certificate, &fake_certificates[1]) + .await + .expect_err("standard certificate verification must enforce the check"); + + assert_error_matches!( + CertificateVerifierError::MissingAncillaryVerifierData, + error + ); + } + } } diff --git a/mithril-common/src/test/double/circuit_key_registry_certifier.rs b/mithril-common/src/test/double/circuit_key_registry_certifier.rs new file mode 100644 index 00000000000..dec7aa818a0 --- /dev/null +++ b/mithril-common/src/test/double/circuit_key_registry_certifier.rs @@ -0,0 +1,37 @@ +//! A module used for a fake implementation of a circuit verification key certifier +//! + +use anyhow::anyhow; +use async_trait::async_trait; + +use crate::StdResult; +use crate::crypto_helper::{CircuitVerificationKeyCertifier, CircuitVerificationKeyRegistry}; + +/// A fake [CircuitVerificationKeyCertifier] serving a configured registry as verified. +pub struct FakeCircuitVerificationKeyCertifier { + registry: Option, +} + +impl FakeCircuitVerificationKeyCertifier { + /// Create a fake certifier serving the given registry as verified. + pub fn from_registry(registry: CircuitVerificationKeyRegistry) -> Self { + Self { + registry: Some(registry), + } + } + + /// Create a fake certifier failing every check, as when no registry can be retrieved. + pub fn that_fails() -> Self { + Self { registry: None } + } +} + +#[cfg_attr(target_family = "wasm", async_trait(?Send))] +#[cfg_attr(not(target_family = "wasm"), async_trait)] +impl CircuitVerificationKeyCertifier for FakeCircuitVerificationKeyCertifier { + async fn get_verified_registry(&self) -> StdResult { + self.registry + .clone() + .ok_or_else(|| anyhow!("Verified registry not available")) + } +} diff --git a/mithril-common/src/test/double/mod.rs b/mithril-common/src/test/double/mod.rs index e95f5685b85..8b631a98a16 100644 --- a/mithril-common/src/test/double/mod.rs +++ b/mithril-common/src/test/double/mod.rs @@ -5,6 +5,8 @@ mod api_version; mod certificate_retriever; #[cfg(feature = "future_snark")] +mod circuit_key_registry_certifier; +#[cfg(feature = "future_snark")] mod circuit_key_registry_retriever; mod dummies; pub mod fake_data; @@ -14,6 +16,8 @@ pub(super) mod precomputed_kes_key; pub use api_version::DummyApiVersionDiscriminantSource; pub use certificate_retriever::FakeCertificaterRetriever; #[cfg(feature = "future_snark")] +pub use circuit_key_registry_certifier::FakeCircuitVerificationKeyCertifier; +#[cfg(feature = "future_snark")] pub use circuit_key_registry_retriever::FakeCircuitVerificationKeyRegistryRetriever; /// A trait for giving a type a dummy value. diff --git a/mithril-test-lab/mithril-end-to-end/src/mithril/aggregator.rs b/mithril-test-lab/mithril-end-to-end/src/mithril/aggregator.rs index bb91ae19477..f6562794a58 100644 --- a/mithril-test-lab/mithril-end-to-end/src/mithril/aggregator.rs +++ b/mithril-test-lab/mithril-end-to-end/src/mithril/aggregator.rs @@ -63,6 +63,8 @@ pub struct Aggregator { process: RwLock>, chain_observer: Arc, full_node: FullNode, + aggregate_signature_type: AggregateSignatureType, + startup_protocol_parameters: ProtocolParameters, } impl Aggregator { @@ -107,6 +109,8 @@ impl Aggregator { let public_server_url = format!("http://localhost:{server_port_parameter}/aggregator"); let cardano_node_version = aggregator_config.cardano_node_version.to_string(); let aggregate_signature_type = aggregator_config.aggregate_signature_type.to_string(); + let circuit_verification_key_registry_path = + Self::circuit_verification_key_registry_path().display().to_string(); let mut env = EnvVars::from([ ("NETWORK", "devnet"), ("NETWORK_MAGIC", &magic_id), @@ -136,6 +140,10 @@ impl Aggregator { ), ("GENESIS_VERIFICATION_KEY", GENESIS_VERIFICATION_KEY), ("GENESIS_SECRET_KEY", GENESIS_SECRET_KEY), + ( + "CIRCUIT_VERIFICATION_KEY_REGISTRY_PATH", + &circuit_verification_key_registry_path, + ), ( "ERA_READER_ADAPTER_TYPE", aggregator_config.mithril_era_reader_adapter, @@ -233,6 +241,8 @@ impl Aggregator { process: RwLock::new(None), chain_observer, full_node: aggregator_config.full_node.clone(), + aggregate_signature_type: aggregator_config.aggregate_signature_type, + startup_protocol_parameters: aggregator_config.startup_protocol_parameters.clone(), }) } @@ -240,6 +250,10 @@ impl Aggregator { format!("{}", index + 1) } + pub fn circuit_verification_key_registry_path() -> PathBuf { + std::env::temp_dir().join("circuit-verification-key-registry.json") + } + pub fn copy_configuration(other: &Aggregator) -> Self { Self { index: other.index, @@ -252,6 +266,8 @@ impl Aggregator { process: RwLock::new(None), chain_observer: other.chain_observer.clone(), full_node: other.full_node.clone(), + aggregate_signature_type: other.aggregate_signature_type, + startup_protocol_parameters: other.startup_protocol_parameters.clone(), } } @@ -323,6 +339,13 @@ impl Aggregator { .with_context(|| "`mithril-aggregator genesis bootstrap` crashed")?; if exit_status.success() { + drop(command); + if matches!( + self.aggregate_signature_type, + AggregateSignatureType::Snark | AggregateSignatureType::IvcSnark + ) { + self.bootstrap_circuit_key_registry().await?; + } Ok(()) } else { command.tail_logs(Some(command_name), 40).await?; @@ -339,6 +362,46 @@ impl Aggregator { } } + async fn bootstrap_circuit_key_registry(&self) -> StdResult<()> { + let mut command = self.command.write().await; + let command_name = &format!( + "mithril-aggregator-circuit-key-registry-bootstrap-{}", + self.name_suffix, + ); + command.set_log_name(command_name); + + let args = vec![ + "circuit-key-registry".to_string(), + "bootstrap".to_string(), + "--protocol-parameters".to_string(), + serde_json::to_string(&self.startup_protocol_parameters)?, + "--target-registry-path".to_string(), + Self::circuit_verification_key_registry_path().display().to_string(), + ]; + + let exit_status = command + .start(&args)? + .wait() + .await + .with_context(|| "`mithril-aggregator circuit-key-registry bootstrap` crashed")?; + + if exit_status.success() { + Ok(()) + } else { + command.tail_logs(Some(command_name), 40).await?; + + Err(match exit_status.code() { + Some(c) => anyhow!( + "`mithril-aggregator circuit-key-registry bootstrap` exited with code: {c}" + ), + None => anyhow!( + "`mithril-aggregator circuit-key-registry bootstrap` was terminated with a signal" + ), + }) + .map_err(|e| anyhow!(RetryableDevnetError(e.to_string()))) + } + } + pub async fn stop(&self) -> StdResult<()> { let mut process = self.process.write().await; if let Some(mut process_running) = process.take() { diff --git a/mithril-test-lab/mithril-end-to-end/src/mithril/client.rs b/mithril-test-lab/mithril-end-to-end/src/mithril/client.rs index 3cb527d6d9f..3916f87cdbb 100644 --- a/mithril-test-lab/mithril-end-to-end/src/mithril/client.rs +++ b/mithril-test-lab/mithril-end-to-end/src/mithril/client.rs @@ -8,7 +8,7 @@ use mithril_common::StdResult; use mithril_common::entities::{BlockHash, EpochSpecifier, TransactionHash}; use crate::utils::{MithrilCommand, NodeVersion}; -use crate::{ANCILLARY_MANIFEST_VERIFICATION_KEY, GENESIS_VERIFICATION_KEY}; +use crate::{ANCILLARY_MANIFEST_VERIFICATION_KEY, Aggregator, GENESIS_VERIFICATION_KEY}; #[derive(Debug)] pub struct Client { @@ -430,6 +430,9 @@ impl Client { pub const BIN_NAME: &'static str = "mithril-client"; pub fn new(aggregator_endpoint: String, work_dir: &Path, bin_dir: &Path) -> StdResult { + let registry_file_path = Aggregator::circuit_verification_key_registry_path() + .display() + .to_string(); let env = HashMap::from([ ("GENESIS_VERIFICATION_KEY", GENESIS_VERIFICATION_KEY), ("AGGREGATOR_ENDPOINT", &aggregator_endpoint), @@ -437,6 +440,10 @@ impl Client { "ANCILLARY_VERIFICATION_KEY", ANCILLARY_MANIFEST_VERIFICATION_KEY, ), + ( + "CIRCUIT_VERIFICATION_KEY_REGISTRY_PATH", + registry_file_path.as_str(), + ), ]); let version = NodeVersion::fetch(Self::BIN_NAME, bin_dir)?;