Upgrade to the latest commonware and revm#72
Open
faddat wants to merge 21 commits intorefcell:mainfrom
Open
Conversation
…unknown addrs Implements three RPC fixes that block forge/cannon/most JSON-RPC clients against Kora 0.1.0. Closes refcell#2; addresses the deferred-deploy items from refcell#1. ## Bugs fixed A. `eth_getCode` returned `-32001 account not found` for unknown addresses, violating EIP-1474 which mandates `0x` for accounts with no code. Many tools branch on `getCode === '0x'` to decide "is this a contract?" — the error broke that check. B. `eth_call` returned `-32004 method not implemented`. `IndexedStateProvider` inherited the trait default but never wired the existing `RevmExecutor` to serve calls. C. `eth_estimateGas` had the same root cause as B. ## Approach - Added `RevmExecutor::simulate_call` that mirrors `BlockExecutor::execute` for one transaction but discards state changes — reuses the existing `StateDbAdapter` `DatabaseRef` impl, the same `Context::new(db, spec_id)` + `modify_*_chained` pattern, and the same `evm.replay()` invocation. - Added `RevmExecutor::estimate_gas` that binary-searches over `simulate_call` in the `[21k, block_gas_limit]` range, capped at 25 iterations. - Added `kora_executor::CallParams` — a small parameter struct so the executor doesn't depend on rpc-layer types. `IndexedStateProvider` converts `CallRequest` → `CallParams` inline. - Added a new `ExecutionError::Revert(Bytes)` variant so revert payloads can be passed through to clients (Solidity ABI-decodable). - Patched `IndexedStateProvider::code()` to map `AccountNotFound` and the empty/zero code-hash sentinels to `Ok(Bytes::new())` per EIP-1474. - Wired `Arc<RevmExecutor>` into `IndexedStateProvider` via constructor; added `with_chain_id` convenience for tests and one-off setups. Runner passes the executor at startup (one extra `Arc::new` line). ## What's still broken (out of scope — separate issue) - `eth_blockNumber` returns 0 — the indexer's `head_block_number()` doesn't advance because consensus → indexer wiring is missing (`BlockContextProvider` / `FinalizedReporter` are not feeding `index.insert_block(...)`). - `eth_getBlockByNumber("latest")` returns null — same root cause. - `eth_getTransactionReceipt`/`eth_getTransactionByHash` return null — same root cause; receipts are indexed only after blocks insert. - `net_peerCount = 0` on devnet — believed intentional for single-node, but worth documenting. - `eth_sendRawTransaction` accepts duplicate and far-future nonces without rejection — mempool-level validation gap. ## Pre-existing clippy fixes CI clippy was failing on test code unrelated to this PR. Fixed the smallest set of warnings touching the same crates so this PR turns CI green: - `crates/node/rpc/src/server.rs` — `default_http_addr` made `const fn`. - `crates/node/rpc/src/config.rs` — three `redundant_clone` test warnings fixed by also asserting against the original (preserves test intent). - `crates/node/executor/src/{context,revm}.rs` — five `field_reassign_with_default` warnings rewritten as struct-update-syntax with `..Header::default()`. ## Verification ``` cargo check --workspace # green cargo clippy -p kora-executor -p kora-rpc -p kora-runner --all-targets -- -Dwarn # green cargo test -p kora-executor -p kora-rpc --lib # 86 passing ``` End-to-end (against a local Kora running this branch): ``` curl -X POST -H 'Content-Type: application/json' \ --data '{"jsonrpc":"2.0","method":"eth_getCode","params":["0xdead...","latest"],"id":1}' \ http://localhost:8545 # Expect: {"result":"0x"} not -32001 curl -X POST ... eth_call ... # expect a result, not -32004 ``` Then `bunx cannon build packages/exchange/cannonfile.toml --rpc-url http://localhost:8545` should complete, unblocking the deferred ClearingHouse + MarketRegistry stack from issue refcell#1. Closes refcell#2. Refs refcell#1.
- Updated versions of various dependencies in Cargo.lock, including `alloy-consensus`, `alloy-eips`, and `commonware` packages. - Refactored DKG protocol to improve clarity and functionality, including renaming methods for better understanding of dealer logs. - Introduced a new configuration for MCP servers in `.cursor/mcp.json`. - Adjusted buffer pool references in network and transport modules to use `CacheRef` for improved memory management. - Enhanced unit tests for DKG to ensure comprehensive coverage of dealer log requirements and finalization conditions.
- Added support for secondary peers in the Kora devnet, allowing them to follow validator traffic without participating in consensus. - Updated the `Cargo.toml` and `Cargo.lock` files to include new dependencies for commonware modules. - Enhanced the README and Docker documentation to explain the role and setup of secondary peers. - Modified the CLI to accept secondary peer arguments and adjusted the setup process to create secondary peer identity keys. - Updated Docker configurations to include a secondary peer service in the devnet setup. - Refactored network transport logic to track both primary and secondary peers effectively.
- Added functionality to fetch and display the status of the secondary peer in the devnet-stats script. - Introduced new variables for follower service and P2P port. - Updated the render function to include follower health, state, uptime, and network information in the output. - Adjusted the endpoints section to reflect the follower P2P port.
Resolves the IndexedStateProvider::new conflict in crates/node/runner/src/runner.rs by keeping the PR's 3-arg form (block_index, qmdb_state, rpc_executor) — required because the PR adds the executor to the struct. Adapts ExecutionResult test constructors in crates/node/executor/src/revm.rs to the new revm 17/38 shape: the old (gas_used, gas_refunded) fields collapsed into a single gas: ResultGas. ExecutionResult::Revert also gained a logs field. Production code already used `..` patterns and was unaffected. cargo check --workspace clean; 86/86 tests pass on kora-executor + kora-rpc.
feat(rpc): implement eth_call + eth_estimateGas; fix eth_getCode for unknown addrs
- Added `kora-indexer` as a dependency in `Cargo.toml` and updated its path. - Specified versions for `alloy-consensus` and `alloy-eips` in `Cargo.lock`. - Enhanced the `handle_finalized_update` function to utilize `kora-indexer` for block indexing. - Updated `NodeRunner` to initialize `block_index` when RPC configuration is present, allowing for improved state management during RPC calls.
- Added `alloy-eips` and `k256` as dependencies in `Cargo.toml` and updated `Cargo.lock`. - Introduced `sha3` dependency for cryptographic functions. - Modified `EthApiImpl` to support pending transactions with a new `pending_txs` field. - Updated transaction submission callback to return a future, allowing for asynchronous handling. - Enhanced block retrieval methods to include an option for full transaction details. - Implemented a new function to convert raw transactions to RPC format. - Improved tests for block retrieval and transaction handling, ensuring robust functionality.
Bursty contract-deploy workloads (Foundry Forge, Cannon, raw-RPC scripts
that submit the full Mirage agents+exchange parity stack of ~39 sequential
txs from a single deployer) hit the per-sender nonce-gap cap before any
txs mine. The validator computes:
max_accepted_nonce = state_nonce + max_txs_per_sender
and rejects anything beyond that as `TxPoolError::NonceGap`. With the old
default of 16 and `state_nonce = 0` for a fresh deployer, every tx with
nonce 17+ is dropped at admission, so devnet deploys silently lose ~half
of a 39-tx run even when the chain itself is healthy.
256 covers all current contract-deploy patterns (Mirage = 39, future stacks
trend higher with more wiring/seed calls), is still small enough to bound
memory per malicious sender, and lives behind the existing
`with_max_txs_per_sender(...)` builder so production deployments can tune
down if needed.
Adds `accept_burst_of_39_sequential_nonces_from_single_sender` which signs
39 contiguous-nonce legacy txs with one key and asserts every one passes
validation under `PoolConfig::default()`. Repros the failure with the old
cap (`NonceGap { got: 17, expected: 0 }`) and locks in the fix.
…rst-deploys fix(txpool): raise default max_txs_per_sender 16 → 256 to unblock contract-deploy bursts
…tx_submit Devnet 1337 (`http://65.109.61.210:8545`) was running a kora binary built before commit `beb637a` ("Enhance Ethereum JSON-RPC API and update dependencies"), so the runner had no `with_tx_submit(...)` wiring on the `RpcServer`. Symptom in production: - `eth_sendRawTransaction` returned a valid hash for every tx - `eth_getTransactionByHash` returned the full tx (RPC-side `pending_txs` cache populated) - `eth_getTransactionReceipt` was `null` forever - Block production advanced ~2 blocks/s but every block had `transactionsRoot = 0x000…000`, `gasUsed = 0`, `txs = []` - `eth_getTransactionCount(addr, "latest")` and `(addr, "pending")` both stuck at the pre-startup nonce Empty blocks forever, no error anywhere. Took half a day to diagnose because every individual layer looked healthy. This commit adds two small things to make the next occurrence visible within seconds and impossible to merge: ## 1. Producer-side warn log when build_block produces an empty block while the mempool has unincluded txs (`crates/node/runner/src/app.rs`) Captures the smoking gun directly: "mempool_len=N, excluded_len=M, produced empty block." If `N > M` and we still drain zero, something is wrong and operators see it in logs. ## 2. RPC-side tracing for tx flow (`crates/node/runner/src/runner.rs`) `debug!` on successful insert, `warn!` on validator rejection, `warn!` on `ledger.submit_tx` returning false (duplicate or pool error). Fills the gap where a tx silently dies between RPC and mempool. ## 3. Two regression tests in `crates/node/rpc/src/eth.rs` - `send_raw_transaction_with_no_callback_silently_accepts_but_drops` pins the failure mode that hit devnet: when no `tx_submit` is wired, `send_raw_transaction` returns Ok with the hash and populates `pending_txs` but does *not* signal the caller that the tx will not be processed. Test exists so any refactor that "fixes" this by erroring out (or any regression that re-introduces the unwired state) is visible in CI. - `send_raw_transaction_passes_full_tx_bytes_to_callback` strengthens the existing `eth_send_raw_transaction` test (which only checked an `AtomicBool`). Asserts the bytes the callback receives are exactly the bytes the caller submitted — catches re-encoding, truncation, or wrong-buffer regressions. `cargo test -p kora-rpc --lib` → 91/91 passing. `cargo check -p kora-runner` → clean.
…d-regress-empty-blocks diagnostics: warn-log silent tx drops + regression tests for unwired tx_submit
The block codec was capped at 1024 bytes per tx, but the txpool validator admits txs up to `PoolConfig::default().max_tx_size = 128 KiB` and the domain-level `BlockCfg::default().tx.max_tx_bytes` is also 128 KiB. So contracts up to 128 KiB pass admission, sit in the mempool, and are then silently dropped by the producer when the codec rejects them at block-build time. Empirical signature on devnet 1337 (`http://65.109.61.210:8545`): - 21000-gas value transfers mine instantly - ~22-byte init-code CREATEs mine instantly - 1 KiB+ Solidity contracts (e.g. RoleRegistry, 1615 bytes input, 10M gas) sit in the mempool indefinitely; producer keeps shipping empty blocks while the tx is right there in `eth_getTransactionByHash` Bumping to 128 KiB matches the validator + the domain default. Full parity restores: validator admits up to N → producer encodes up to N → no silent drops between layers. Updates the same constant in `crates/e2e/src/harness.rs` so test infrastructure stays in sync. The constant could plausibly be hoisted to a single source of truth (or read directly from `BlockCfg::default()`) as a follow-up.
…max-tx-bytes-1k-to-128k fix(runner): raise BLOCK_CODEC_MAX_TX_BYTES 1 KiB → 128 KiB
…nc storage access - Adjusted `Cargo.toml` to refine Tokio features for development dependencies. - Modified `adapter.rs` to utilize `block_in_place` for async operations within a Tokio runtime. - Enhanced `QmdbHandle` in `qmdb.rs` to include a `Mutex` for managing concurrent access to storage, ensuring safe operations during backend partition modifications. - Updated state management in `state.rs` and `ledger.rs` to incorporate the new storage access mechanism, improving concurrency handling during state commits and root computations.
- Updated `lib.rs` to export additional resolver types: `ResolverChannels`, `ResolverMailbox`, and `ResolverReceiver`. - Modified `peers.rs` to utilize these new types in the `PeerInitializer` struct, improving clarity and type safety in peer initialization. - Adjusted function signatures to return the new resolver types, streamlining the interface for peer-related operations.
- Added logic to return the parent snapshot's state root when the changeset is empty, preventing unnecessary state recomputation. - Introduced a new test to verify that an empty child inherits the state root from its parent after persistence, ensuring correct behavior in ledger operations.
…leanup - Introduced a new method in `StateRoot` for computing a deterministic consensus root based on a parent root and state transition changes. - Enhanced the `compute_root_from_store` function in `LedgerView` to utilize the new state transition logic. - Added a script to clear runtime state for Docker volumes, improving development workflow by ensuring clean state between runs. - Updated CLI and runner configurations to initialize the Tokio runtime with a specified storage directory.
- Reduced the number of initial GenesisAllocations from seven to one, streamlining the setup process. - Updated the allocation for the address "0x0000000000000000000000000000000000000001" to maintain a balance of "1000000000000000000000000".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR upgrades kora to the latest commonware and revm libraries, and also fixes up some issues with kora's rpc.
It's still early, just like kora itself, but it's been tested and we've done contract deployments on it.
As before,
just devnetis the best way to go about testing / verifying.It's been great working on Kora!