Skip to content
This repository has been archived by the owner on Jul 16, 2024. It is now read-only.

when i run poker_ai train start, the training get stuck at INFO sending sentinel to worker server.py:131 #118

Open
Adnan-annan opened this issue Nov 6, 2020 · 1 comment

Comments

@Adnan-annan
Copy link

No description provided.

@Adnan-annan Adnan-annan changed the title when i run poker_ai train start, the training get sick at INFO sending sentinel to worker server.py:131 when i run poker_ai train start, the training get stuck at INFO sending sentinel to worker server.py:131 Nov 6, 2020
@martingouy
Copy link

It is probably not stuck but the message can be misguiding.
In the terminate method, it is waiting for all the workers to be idle before exiting. What I experienced, and what I suspect you experienced also, is that a few workers were still busy with tasks.
You can try using the flags --sync_cfr --sync_discount --sync_serialise, the server will wait for each step to complete on all the workers before moving to the next one. It will take more time, but the progress bar will be more reliable.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants