Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Increase memory and timeout configuration for the transform lambda
We started seeing out of memory errors in the transform lambda when onboarding the variant-nowcast-hub. This is the first time we've onboarded a hub to S3 that includes sample data, and it looks like the default memory allocation of 128MB isn't sufficient. For example, a single UMass submission is 27MB in memory after the parquet file is read by pyarrow. We can and should investigate ways to make the transform process better about memory, but let's increase the memory now to ensure we're in a good place until then.
- Loading branch information