Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chunked multicalls, more logs, memory snapshots, nuke explorer apis #17

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

meetmangukiya
Copy link
Collaborator

@meetmangukiya meetmangukiya commented May 3, 2024

  • Added a new option to set the max number of calls in 1 multicall
  • Add logging with pino
  • logMemory will log memory in debug level
  • Nuking explorer(ethplorer, blockscout) token fetchers, directly fetch them from chain with RPC with Trade logs. The block explorer APIs return a lot of tokens(99% of which are airdropped shitcoins). We used to have a usd based filter earlier that filtered most of them out, but we removed that in 76d4d1c in Address review comments #11. ATM the ethplorer api returns like 7k tokens, it then goes on to request quote from the cow api for all of them which led to it using a lot of memory (6GB+) and taking too long(because of the rate limiter with exponential backoff logic). On the other hand, the chain strategy executed successfully in 6 minutes and has no dependency on external block explorer APIs and just requires querying the chain for Trade events.

@meetmangukiya meetmangukiya requested review from fleupold and mfw78 May 3, 2024 13:34
Copy link
Contributor

@fleupold fleupold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks fine, although this could easily be three separate PRs (I'm also not sure if we actually want to check in the memory snapshots)

index.ts Show resolved Hide resolved
ts/explorer-apis.ts Outdated Show resolved Hide resolved
ts/common.ts Outdated Show resolved Hide resolved
ts/cowfee.ts Outdated Show resolved Hide resolved
ts/token-fetcher.ts Outdated Show resolved Hide resolved
ts/token-fetcher.ts Outdated Show resolved Hide resolved
@meetmangukiya meetmangukiya requested a review from mfw78 May 22, 2024 03:12
index.ts Outdated Show resolved Hide resolved
@mfw78 mfw78 requested a review from fleupold May 22, 2024 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants