Skip to content

Commit

Permalink
fix: don't loop forever if TGI server fails (#456)
Browse files Browse the repository at this point in the history
  • Loading branch information
thundergolfer authored Sep 28, 2023
1 parent d5f92ab commit 8939086
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions 06_gpu_and_ml/text_generation_inference.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,13 @@ def webserver_ready():
socket.create_connection(("127.0.0.1", 8000), timeout=1).close()
return True
except (socket.timeout, ConnectionRefusedError):
# Check if launcher webserving process has exited.
# If so, a connection can never be made.
retcode = self.launcher.poll()
if retcode is not None:
raise RuntimeError(
f"launcher exited unexpectedly with code {retcode}"
)
return False

while not webserver_ready():
Expand Down

0 comments on commit 8939086

Please sign in to comment.