Handle RPC range errors in getLogs #606
Conversation
| ).map((gauge) => gauge.gaugeAddress); | ||
| ) | ||
| .map((gauge) => gauge.gaugeAddress) | ||
| .filter((address, i, arr) => arr.indexOf(address) === i); // Make unique |
There was a problem hiding this comment.
they should already be unique?
There was a problem hiding this comment.
true, good catch, removing.
modules/web3/events.ts
Outdated
| .catch((e: any) => { | ||
| // Ankr RPC returns error if block range is too wide | ||
| if (e.includes('block range is too wide')) { | ||
| return getEvents(from, to, addresses, topics, rpcUrl, rpcMaxBlockRange / 2); | ||
| } | ||
|
|
||
| // Infura returns 'more than 10000 results' error if block range is too wide | ||
| if (e.includes('query returned more than 10000 results')) { | ||
| const range = e | ||
| .match(/\[([0-9a-fA-F, x]+)\]/) | ||
| .pop() | ||
| .split(', ') | ||
| .map((hex: string) => parseInt(hex, 16)); | ||
|
|
||
| return getEvents(from, to, addresses, topics, rpcUrl, range[1] - range[0]); | ||
| } |
There was a problem hiding this comment.
These retries will mess with the "latestSyncBlock" logic. If it retries, it will still persist the toBlock as the latest think block and not the one it actually synced to. You either need to make sure all blocks up until toBlock are queried or get rid of the retries and set reasonable block limits.
I guess we anyway need to make sure that it fails if one of the promises fails, or not do promise.all but do they sequentially and return the block number until which we synced
There was a problem hiding this comment.
thanks, I double checked and unless I'm missing something obvious, catch is refetching only the chunk that failed, not the whole range, so it's just filling the gaps. there is promise.all instead of allSettled to make sure it gets rejected as a whole, not just a part of it.
calling getEvents should return the whole range or nothing.
modules/web3/events.ts
Outdated
| if (e.includes('query returned more than 10000 results')) { | ||
| const range = e | ||
| .match(/\[([0-9a-fA-F, x]+)\]/) | ||
| .pop() | ||
| .split(', ') | ||
| .map((hex: string) => parseInt(hex, 16)); | ||
|
|
||
| return getEvents(from, to, addressChunk, topics, rpcUrl, range[1] - range[0]); |
There was a problem hiding this comment.
Could you describe the actual string you are parsing here in the comments? Hard to say whether the string manipulation is correct
Extracted getLogs into separate function where we can handle RPC errors related to maximum block range thrown differently by different providers. Added an option to decode event data when ABI is passed.