-
Notifications
You must be signed in to change notification settings - Fork 487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test/spmd/test_spmd_debugging.py fails when run on a v4-8 TPU #6252
Comments
@ManfeiBai can you take a look since you added this test? |
Thanks, will do |
thanks, reproduced locally on v4-8 too, #6263 will fixed this issue, please pull the newest commit and this test has been verified locally too:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
🐛 Bug
test/spmd/test_spmd_debugging.py fails when run on a v4-8 TPU, with the following output:
I ensured my setup was correct by running test/test_operations.py beforehand, which passed, and skipped individual tests which were not applicable to TPUs.
To Reproduce
Steps to reproduce the behavior:
export BUNDLE_LIBTPU=1; export TPUVM_MODE=1
before running the pytorch/xla setup script.export PJRT_DEVICE=TPU; python3 -u test/test_operations.py -v
to ensure my setup was working.python3 -u test/spmd/test_spmd_debugging.py
, which resulted in the above failure.Expected behavior
Expected the test to pass.
Environment
Additional context
The text was updated successfully, but these errors were encountered: