You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have an (admittedly rare) use case where some of my tasks are in the range of 2-10 MB large.
This means that tourbillon will consume a large amount of RAM (I suspect up to 100GB, ouch!).
A brief investigation leads me to suspect that the reason for this is a LRU cache in celery's events.State class. State has a tasks attribute that is defined as:
Hi,
thanks for making tourbillon-celery!
I have an (admittedly rare) use case where some of my tasks are in the range of 2-10 MB large.
This means that tourbillon will consume a large amount of RAM (I suspect up to 100GB, ouch!).
A brief investigation leads me to suspect that the reason for this is a LRU cache in celery's events.State class. State has a tasks attribute that is defined as:
https://github.com/celery/celery/blob/82e478e07cb9beee490369b71b7008fcf0f08f89/celery/events/state.py#L377
with the default parameter in its
__init__
method set tomax_tasks_in_memory=10000
.So - it would be very nice if one could tune this parameter.
The text was updated successfully, but these errors were encountered: