diff --git a/README.md b/README.md index 31fc1bf..5dfc43a 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,10 @@ Extract logs expected by [firehose-starknet](https://github.com/starknet-graph/f This tool exists because it could be slow to sync a Starknet node from scratch, which is currently the only way to bootstrap a [firehose-starknet](https://github.com/starknet-graph/firehose-starknet) deployment. This tool makes calls to the JSON-RPC endpoint offered by a trusted synced node, and emits the exact same format as an instrumented client node would do. +## Checkpoint file + +This tool keeps track of the sync progress through a _checkpoint_ file, specified by the `--checkpoint` command line option or `CHECKPOINT_FILE` environment variable. The content of the checkpoint file is updated every `50` blocks, which means it's likely that some blocks might be sycned more than once when the tool restarts. This is fine as Firehose handles duplicated blocks without issue. + ## License Licensed under either of diff --git a/src/main.rs b/src/main.rs index f1abffa..e25f34b 100644 --- a/src/main.rs +++ b/src/main.rs @@ -135,6 +135,7 @@ async fn run( }; if checkpoint.latest_blocks.len() >= MAX_BLOCK_HISTORY { + // TODO: use ring buffer instead as this is extremely inefficient checkpoint.latest_blocks.remove(0); } checkpoint.latest_blocks.push(new_block_info);