Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Support for PyTorch-XLA 2.0 #1938

Open
zw615 opened this issue Aug 30, 2023 · 2 comments
Open

[FEATURE] Support for PyTorch-XLA 2.0 #1938

zw615 opened this issue Aug 30, 2023 · 2 comments
Labels
enhancement New feature or request

Comments

@zw615
Copy link

zw615 commented Aug 30, 2023

Thanks for your wonderful work!

Given now that PyTorch-XLA 2.1 is just released, and many new features are added compared to PyTorch-XLA 1.xx, I wonder if you have any plan to integrate those features in to this codebase? Those useful features includes torch.compile, PJRT Runtime, AMP, and so on. For the user end, torch.compile may be no different on GPU and TPU.

@zw615 zw615 added the enhancement New feature or request label Aug 30, 2023
@rwightman
Copy link
Collaborator

rwightman commented Aug 30, 2023

@zw615 there was lots of nice updates to XLA on the horizon when I was still using it regularly via the bits_and_tpu brach. I was excited to start testing PJRT in XLA when my v4 TPU access was suddenly reduced in # devices and downgraded to preemptable. That basically made developing, supporting/debugging, and training w/ PT XLA impractical given my time constraints :(

I know PT XLA can be used on GPU but at least prior to 2.x it wasn't as good for GPU as PyTorch eager (especially with compile) so didn't make much sense to try and support it without developing on TPUs...

@zw615
Copy link
Author

zw615 commented Aug 31, 2023

Sigh. In this case, it indeed does not make sense to support pytorch-xla 2.0. I can see the last commit to the bits_and_tpu brach was made last year. Hopefully your access to TPU can be restored in the near future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants