-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
maximum response time #28
Comments
The response time measures the time between your server receiving the request and sending the response back, it doesn't capture any network latency. So it's possible to be zero if it's a quick function, but it sounds like you should be getting a much larger response time, so something doesn't sound right there. There isn't any upper limit set. Which API framework are you using? |
It is FastAPI. I am positively sure the function itself takes so long as it's an AI inference task on CPU (which will take ~2 seconds when switching to a GPU later). I get a server response 200 OK back, so the function itself is working fine. |
I just tried a few more calls and now it registers a median of ~30k milliseconds. So it does work now. Not sure why it showed 0 at first... |
Interesting, so it's measuring it inconsistently, I'll look into that. How are you hosting your API? |
My API is hosted in a docker on Huggingface spaces, maybe that has something to do with it? |
I am having a POST function that currently takes up to 20 seconds. I see the request to the endpoint in my dashboard, but the response time seems to be 0 . Is there some kind of upper limit here?
The text was updated successfully, but these errors were encountered: