Add code to batch request across different users on the server side

[CoderPat]

Currently each request is handled independently, even if they request only 1 completion.
This can lead to situations here there is big wait time that could be avoided if the requests were batched together.
TODO: describe problem further

[aksj98]

Hi @CoderPat can you assign this to me? I can start taking a dig at it