Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gather - best way to issue parallel commands? #185

Open
eoghanmurray opened this issue Feb 22, 2021 · 0 comments
Open

gather - best way to issue parallel commands? #185

eoghanmurray opened this issue Feb 22, 2021 · 0 comments

Comments

@eoghanmurray
Copy link
Contributor

I wanted to add up (read) a bunch of sorted sets in as short as possible time window so that I could write an estimated total and minimize the time window where the sets could be modified:

approx_total = sum(await asyncio.gather(
                *map(cx.zcard, list_of_keys)))
await cx.set('zcards-approx-total', approx_total)

This didn't work as it hit a 'Too many connections' error. As this is on RedisCluster, there is a Connection Pool in effect with a limit of 32.

I'm asking this as it struck me that aredis could provide a built in gather function to do this in a max_connections-aware manner.
Is that something that is feasible? or desirable?

@eoghanmurray eoghanmurray changed the title Best way to issue parallel commands? gather - best way to issue parallel commands? Feb 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant