Skip to content

Commit

Permalink
fix: don't loop forever if TGI server fails
Browse files Browse the repository at this point in the history
  • Loading branch information
thundergolfer committed Sep 27, 2023
1 parent d5f92ab commit 39c7b86
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions 06_gpu_and_ml/text_generation_inference.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,11 @@ def webserver_ready():
socket.create_connection(("127.0.0.1", 8000), timeout=1).close()
return True
except (socket.timeout, ConnectionRefusedError):
# Check if launcher webserving process has exited.
# If so, a connection can never be made.
retcode = self.launcher.poll()
if retcode is not None:
raise RuntimeError(f"launcher exited unexpectedly with code {retcode}")
return False

while not webserver_ready():
Expand Down

0 comments on commit 39c7b86

Please sign in to comment.