Memory management

[PaulaPerezL]

I have some troubles managing memory when reading and saving images. How do you handle it? Do you store each image in its own Zarr, or do you stack them up into one big Zarr?

Thank you very much

[peterdudfield]

We do a bit of both, for our production service we store each image as its own zarr, and then we join then together to make a joint zarr. We seperate non hrv and hrv out.

We run the service on AWS using these memory limits - https://github.com/openclimatefix/ocf-infrastructure/blob/main/terraform/modules/services/sat/ecs.tf#L4

What memory limits did you have?

@jacobbieker should we put some of this in the readme? We could add it to the https://github.com/openclimatefix/Satip?tab=readme-ov-file#running-in-production section.

[peterdudfield]

Does this make sense @PaulaPerezL ?

[PaulaPerezL]

Yes, thank you, it makes sense. What happened to us is that the command "del object" is not working, or at least it does not release the RAM consumed by that object. However, we devised a method to launch each image in an independent process, and by doing so, we are able to clear the RAM after reading the image.