Amp-powered subgraphs #6218
Amp-powered subgraphs #6218
Conversation
Only require block number columns and try to load block hashes and timestamps from the source tables
isum
force-pushed
the
ion/amp-powered-subgraphs
branch
from
957ef3a to
3ebfc27
Compare
lutter
left a comment
lutter
left a comment
There was a problem hiding this comment.
LGTM! (Been waiting to do this on a huge PR for ages)
| }; | ||
| } | ||
|
|
||
| pub(super) trait Compat<T> { |
There was a problem hiding this comment.
Not a big deal, but Compat is maybe a bit of a misnomer, maybe FromAlloy would be better
|
|
||
| use super::{Compat, Context, Error, LatestBlocks}; | ||
|
|
||
| pub(super) async fn check_and_handle_reorg<NC>( |
There was a problem hiding this comment.
This is really important. It would be good to have a comment on how this works and why. It might also be worth explaining what the other two methods here do.
core/src/nozzle_subgraph/metrics.rs
Outdated
| let stopwatch = StopwatchMetrics::new( | ||
| logger.cheap_clone(), | ||
| deployment, | ||
| "nozzle-process", |
There was a problem hiding this comment.
Just make that process, it's easier on existing dashboards and it's clear from the manifest what kind of subgraph it is
|
|
||
| let is_close_to_chain_head = latest_block.saturating_sub(block_number) <= 100; | ||
|
|
||
| cx.store |
There was a problem hiding this comment.
This should be marked in the stopwatch as transact_block
| })?; | ||
| } | ||
|
|
||
| let ModificationsAndCache { |
There was a problem hiding this comment.
This should be marked in the stopwatch as as_modifications
| // Q: How to make this failure loud enough so it is not missed? | ||
| // | ||
| // println!("Subgraph panicked"); | ||
| // std::process::abort(); |
There was a problem hiding this comment.
We shouldn't do that; yes, it's hard to spot a panic in lots of logs, but taking down the process would mean that one subgraph can affect lots of other subgraphs.
| } | ||
|
|
||
| /// Tracks the target block number of a deployment. | ||
| pub(super) struct DeploymentTarget(IntGauge); |
There was a problem hiding this comment.
I don't understand what 'target' means here - is this basically the latest block that Amp has available?
core/src/amp_subgraph/metrics.rs
Outdated
| ) -> Self { | ||
| let int_gauge = metrics_registry | ||
| .new_int_gauge( | ||
| "amp_deployment_head", |
There was a problem hiding this comment.
I think that was addressed in a later commit, so ignore
I think prefixing all these with amp when they correspond to an existing metric like deployment_head makes using them in dashboards much harder. When metrics mean the same thing regardless of subgraph flavor, we should use the same metric name
|
|
||
| ## ENV variables | ||
|
|
||
| Amp-powered subgraphs feature introduces the following new ENV variables: |
There was a problem hiding this comment.
While this is still experimental, it's fine to have that here, but at some point we should move this to docs/environment-variables.md
Amp-powered subgraphs are a new kind of subgraphs with SQL data sources that query and index data from the Amp servers. They are significantly more efficient than the standard subgraphs, and the indexing time can be reduced from days and weeks, to minutes and hours in most cases.
Documentation
Documentation on creating Amp-powered subgraphs is available here.
Examples
Complete examples of how to create, deploy and query Amp-powered subgraphs are available in a separate repository:
https://github.com/edgeandnode/amp-subgraph-examples