Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

关于流水线并行的一个问题 #173

Open
Cheung-Z opened this issue May 14, 2024 · 0 comments
Open

关于流水线并行的一个问题 #173

Cheung-Z opened this issue May 14, 2024 · 0 comments

Comments

@Cheung-Z
Copy link

Cheung-Z commented May 14, 2024 •
edited
Loading

作者您好，
最近在跑流水线并行的代码，并尝试在qwen上复现，但发现一个问题，pipemodel是基于basemodel来加载的，以7B模型为例，相当于内存中最多的时候需要加载2 * 14g，是这样吗？（虽然后续使用了del model可以释放出内存
感觉想在8 * 32G v100上跑qwen 32B的希望破灭了....
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Cheung-Z