Add FAQ on ASGI vs WSGI

jpmorganchase · Apr 11, 2024 · 90e6c30 · 90e6c30
1 parent c8ce5c7
commit 90e6c30
Showing 1 changed file with 9 additions and 0 deletions.
diff --git a/docs/faq.rst b/docs/faq.rst
@@ -23,6 +23,15 @@ For example, when using Gunicorn, this could be done from a post-fork Gunicorn h
 Does **inference-server** support async/ASGI webservers?
 --------------------------------------------------------
 
+No.
+
+**inference-server** is a WSGI application to be used by synchronous webservers.
+
+For most ML models that will be the correct choice as model inference is typically CPU-bound.
+Therefore, a multi-process based WSGI server is a good choice whereby the number of workers is equal to the number of CPU cores available.
+
+For more details see :ref:`deployment:Configuring Gunicorn workers`.
+
 
 My model is leaking memory, how do I address that?
 --------------------------------------------------