Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPMD] Add debug of SPMD for single/multi host #5742

Merged
merged 53 commits into from
Dec 5, 2023
Merged

[SPMD] Add debug of SPMD for single/multi host #5742

merged 53 commits into from
Dec 5, 2023

Conversation

ManfeiBai
Copy link
Collaborator

@ManfeiBai ManfeiBai commented Oct 26, 2023 •
edited
Loading

Add debug of SPMD for single/multi device for visualize sharded tensors

@ManfeiBai ManfeiBai changed the title [debug] add debug for FSDP for single device [debug] add debug for SPMD for single device Oct 26, 2023
@yeounoh yeounoh self-requested a review October 26, 2023 17:17
@ManfeiBai ManfeiBai force-pushed the newmastercheckcommitoct19 branch from ba40a9b to ef482ea Compare October 26, 2023 17:45
@yeounoh
Copy link
Contributor

yeounoh commented Oct 27, 2023

Hi @ManfeiBai could you also move the file to torch_xla/experimental/spmd/ you can create a new dir, I will refactor in the upcoming release.

@ManfeiBai
Copy link
Collaborator Author

Hi @ManfeiBai could you also move the file to torch_xla/experimental/spmd/ you can create a new dir, I will refactor in the upcoming release.

Thanks, updated to torch_xla/experimental/spmd/ and created torch_xla/experimental/spmd/ too

@ManfeiBai ManfeiBai marked this pull request as ready for review November 8, 2023 19:38
@ManfeiBai ManfeiBai changed the title [debug] add debug for SPMD for single device Add debug of SPMD for single device Nov 8, 2023
@ManfeiBai ManfeiBai marked this pull request as draft November 8, 2023 19:41
@ManfeiBai ManfeiBai marked this pull request as ready for review November 26, 2023 22:49
@ManfeiBai ManfeiBai changed the title Add debug of SPMD for single device Add debug of SPMD for single host Nov 28, 2023
yeounoh
yeounoh suggested changes Nov 28, 2023
View reviewed changes
Copy link
Contributor

@yeounoh yeounoh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left comments, thanks @ManfeiBai

@ManfeiBai ManfeiBai force-pushed the newmastercheckcommitoct19 branch 2 times, most recently from c83a2a5 to 281685b Compare November 29, 2023 00:25
@ManfeiBai ManfeiBai requested a review from yeounoh December 4, 2023 02:18
@ManfeiBai ManfeiBai mentioned this pull request Dec 4, 2023
Closed
yeounoh
yeounoh approved these changes Dec 4, 2023
View reviewed changes
Copy link
Contributor

@yeounoh yeounoh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left one more minor comment https://github.com/pytorch/xla/pull/5742/files#r1414546255

@ManfeiBai ManfeiBai merged commit f94b53c into master Dec 5, 2023
19 checks passed
chunnienc pushed a commit to chunnienc/xla that referenced this pull request Dec 14, 2023
@ManfeiBai @chunnienc
[SPMD] Add debug of SPMD for single/multi host (pytorch#5742)
5175072
golechwierowicz pushed a commit that referenced this pull request Jan 12, 2024
@ManfeiBai @golechwierowicz
[SPMD] Add debug of SPMD for single/multi host (#5742)
4d8c0c8
bhavya01 pushed a commit that referenced this pull request Apr 22, 2024
@ManfeiBai @bhavya01
[SPMD] Add debug of SPMD for single/multi host (#5742)
071a0c4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yeounoh yeounoh yeounoh approved these changes

@JackCaoG JackCaoG Awaiting requested review from JackCaoG

Assignees

@ManfeiBai ManfeiBai

Labels
backport_2.2 SPMD / Distributed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ManfeiBai @yeounoh @JackCaoG