As part of the performance improvements task that we are going to do, we would need to have a proper setup to have a decent benchmarking setup. The requirements for this benchmark setup:
- Works at the JSON-RPC level
- Concurrent request are being done and we have a low throughout, medium throughout and high throughput values (actual values will need to be determine by poking Edge&Node team so they give us information about their rate)
- We most be able to run it against other providers (if the first point is respected, should be the case already)
- The test cases must cover different contracts and different form of access (for example, only doing the test against ERC20 contracts for name/symbol/decimals is far from being good). Ideally the call sample would be large enough to cover most patterns of access (probably that we might requires automated extraction and collaborate with Edge&Node for this). We could get away with manually exporting multiple pages of their Alchemy input.
- The test cases must cover “live” segments and historical segments. The benchmark should not inspect the responses (although a different test case should validate that the RPC endpoints respect all historical response). Doing the live portion is important because in the EVM executor, the live portion is backed by ForkDB which is heavily used internally when serving the request and has a known inefficient implementation.
- Performance metrics can be extracted and ideally, some baseline can be provided to output a +/- differences between the baseline and the current run so that we can compare our solution against other RPCs providers as well as comparing improvements of performance tweaks we do.
All this can be implement using pure Go code and libraries but we could also leverage existing tools like Vegeta and k6 (I have personally used Vegeta in the past and worked great for my past use case, unsure about what is the scope and extent of k6, but it was referenced by Dan from Figment.
The person taking this task will need to design about the design as well as implementing it.
As part of the performance improvements task that we are going to do, we would need to have a proper setup to have a decent benchmarking setup. The requirements for this benchmark setup:
All this can be implement using pure Go code and libraries but we could also leverage existing tools like Vegeta and k6 (I have personally used Vegeta in the past and worked great for my past use case, unsure about what is the scope and extent of k6, but it was referenced by Dan from Figment.
The person taking this task will need to design about the design as well as implementing it.