feat: Add state-analyzer panel in the lab #248
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implements comprehensive State Analytics for Ethereum state growth monitoring:
## Backend Implementation
- Proto definitions for State Analytics gRPC service
- Service layer with ClickHouse integration for state metrics
- Query implementations for latest block delta, top adders/removers, growth charts
- gRPC server implementation with comprehensive unit and integration tests
- REST API handlers for v1 endpoints
- Route registration for /api/v1/{network}/state/* endpoints
## Frontend Implementation
- React State Analyzer page with interactive visualizations
- REST API client methods for State Analytics endpoints
- TypeScript type definitions for all API responses
- Navigation link to State Analyzer page
- Period filters (24H, 7D, 30D) and interactive charts
- Top state adders/removers tables with contract addresses
## Configuration
- Updated package.json for React 19 compatibility (react-query 5.62.8)
- Added frontend .env for production ethPandaOps backend URLs
Three critical fixes for State Analytics functionality: 1. **Geolocation service**: Skip database load when disabled - Added config field to Client struct - Check enabled flag before loading database in Start() - Prevents startup failure when geolocation is disabled 2. **State Analytics Stop() signature**: Match service.Service interface - Changed Stop() return type from 'error' to void - Ensures proper interface implementation - Required for service to be added to services list 3. **Service initialization**: Add State Analytics to services slice - State Analytics service now gets Start() called on startup - Properly initializes ClickHouse clients for each network - Fixes "network mainnet not configured" API errors Without these fixes, State Analytics service was created but never started, leaving ClickHouse clients uninitialized and causing all API requests to fail with 404 errors.
Frontend was expecting camelCase fields but API returns snake_case. This caused TypeErrors when trying to access undefined properties. Changes: 1. Updated TypeScript types to match API response format: - block_number, new_slots_count, cleared_slots_count, etc. - slots_added, estimated_bytes_added, percentage_of_total - slots_cleared, estimated_bytes_freed, estimated_gas_refund 2. Updated UI component to use correct field names: - latestData.block_number instead of blockNumber - adder.slots_added instead of slotsAdded - remover.percentage_of_total instead of percentageOfTotal 3. Added null safety to formatNumber function: - Returns '0' for undefined/null values - Prevents TypeError when accessing undefined properties 4. Added optional chaining (?.) for percentage_of_total.toFixed() - Prevents crash if percentage is undefined - Falls back to '0' if value is missing The State Analyzer page now displays all data correctly with proper formatting and no runtime errors.
- Add smart polling: lightweight 1-second polls for latest block, heavy data fetch only on block changes - Fix TypeScript types to use snake_case matching API responses (block_number, new_slots_count, etc.) - Fix Nivo chart color interpolation by using direct hex values instead of CSS custom properties - Add comprehensive null checks for chart data to prevent rendering errors - Update all field references throughout UI to use snake_case format
- Add proto definitions for GetContractStateComposition and GetHierarchicalState
- Create ClickHouse query for contract state sizes aggregation
- Implement service methods: contract_state_composition.go and hierarchical_state.go
- Add hierarchical tree builder (Category -> Protocol -> Contract)
- Wire up gRPC wrapper layer for new methods
- Add REST API handlers and routes for /state/composition and /state/hierarchical
- Add getFloat64 helper function for ClickHouse result parsing
Endpoints tested and working:
- GET /api/v1/{network}/state/composition?limit=N&min_size_bytes=N
- GET /api/v1/{network}/state/hierarchical?max_depth=N&contracts_per_protocol=N
Add TypeScript types, endpoints, and client methods for fetching contract state composition data to power the Paradigm-style interactive treemap visualization. - Add ContractStateCompositionResponse and HierarchicalStateResponse types - Add stateComposition and stateHierarchical endpoint paths - Add getContractStateComposition() and getHierarchicalState() client methods
…g and color variations - Add Plotly.js treemap visualization for state composition - Implement dynamic color variations for categories, protocols, and contracts - Improve layout with flexbox for full-height container - Default to showing contracts (depth=3) with 100 contracts per protocol - Remove auto-refresh interval to prevent unnecessary re-renders - Fix viewport-relative sizing to use full available space
…radigm-style) Add monthly/yearly state growth analysis by contract address, similar to Paradigm's "How to raise the gas limit" research (Figures 2 & 3). Backend changes: - Add GetStateGrowthByCategory RPC endpoint with granularity support (daily/monthly/yearly) - Implement category_state_growth.go service with execution layer data queries - Update queries.go to use canonical_execution_storage_diffs table - Fix genesis time to Ethereum mainnet (July 30, 2015) instead of Beacon Chain - Add protojson marshaling for proper RFC3339 timestamp serialization - Add REST API handler and route for /state/growth-by-category Frontend changes: - Add StateGrowthByCategory component with Plotly stacked bar chart - Support daily/monthly/yearly granularity toggle - Display summary statistics (total contracts, growth, earliest data) - Add REST client method and endpoint for categorized growth data - Integrate "Categorized Growth" tab into State Analyzer page Note: Queries currently timeout on canonical_execution_storage_diffs (10.6B rows). Performance fix proposed in ethpandaops/spamoor#146
Backend changes: - Fix block range calculation formula for daily queries (subtract genesis time before dividing) - Automatically calculate latest day's block range to avoid timeouts on large tables - Only query ~7,200 blocks (one day) instead of scanning billions of rows Frontend changes: - Simplify to daily-only view (remove granularity selector) - Display bar chart with categories on X-axis instead of time series - Show all possible categories (DeFi, NFT, ERC20, Gaming, Bridge, Oracle, DAO, Identity, Other) even if 0 state - Update summary stats to show "Latest Date" instead of time range
CPerezz
changed the title
- Query last 7,150 execution blocks (approximately 24 hours) - Use only canonical_execution_storage_diffs (execution layer data) - Remove all beacon chain references and JOINs - Simplify to daily-only granularity without timestamps - Display all 9 categories on X-axis (bar chart) - Query time: 2-5 minutes for ~50MB of data BREAKING: Removed monthly/yearly granularity options NOTE: All contracts currently categorized as "Other" (labeling not implemented)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Leaving this here just so that other team members can quickly check updates.
Once most of the things are cleared, the plan is to move to lab2 as agreed with @samcm