Beyla cache: prevent OOMing during start in big clusters

[mariomac]

In big clusters (800 nodes), when the cache pod starts, it receives a huge amount of K8s metadata, and it is enqueued in the memory faster than the Beyla cache service it is really able to forward to the Beyla client instances.

This causes that, when a beyla-k8s-cache pod starts, it accumulates GBs of memory until it is OOMed, then connected Beyla clients might move to another instances, and when the pod is restarted, it is idle enough to process all the information faster than it accumulates in main memory.

We need to find a way to desynchronize the informers receive-transform-store thread from the client message submission.

[mariomac]

This has been fixed by properly setting GOMEMLIMIT. We just need to document this.