Skip to content

Commit

Permalink
Store erc20 loaded addresses in cache
Browse files Browse the repository at this point in the history
- In ERC20Indexer, every chunk of addresses store them in cache
- It should not take any time, and in case task is interrupted during address loading it will be very useful, as loading can be restarted from that point
  • Loading branch information
Uxio0 committed Nov 15, 2024
1 parent fa72c03 commit 0d6a048
Showing 1 changed file with 5 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -279,13 +279,17 @@ def get_almost_updated_addresses(
500k sounds like a good compromise memory/speed wise
"""
created: Optional[datetime.datetime] = None
for created, address in (
for i, (created, address) in enumerate(
query.values_list("created", "address")
.order_by("created")
.iterator(chunk_size=self.eth_erc20_load_addresses_chunk_size)
):
addresses.add(address)

# Store addresses in cache every chunk, just in case task is interrupted during address loading
if i % self.eth_erc20_load_addresses_chunk_size == 0:
self.addresses_cache = AddressesCache(addresses, created)

if created:
last_checked = created

Expand Down

0 comments on commit 0d6a048

Please sign in to comment.