Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] AssertionError: Only flash cross entropy support parallel_output #245

Open
wen020 opened this issue Jun 6, 2024 · 1 comment
Open

[Bug] AssertionError: Only flash cross entropy support parallel_output #245

wen020 opened this issue Jun 6, 2024 · 1 comment
Assignees
@sunpengsdu
Labels
bug Something isn't working

Comments

@wen020
Copy link

wen020 commented Jun 6, 2024

Describe the bug

image
已经按照安装指引安装好了环境，在pretrain的时候报这个错误

Environment

环境完全按照 https://github.com/InternLM/InternEvo/blob/develop/doc/install.md 安装

Other information

No response
@wen020 wen020 added the bug Something isn't working label Jun 6, 2024
@li126com
Copy link
Collaborator

你这个是用了flash attention还是没用呢，如果不用fa的话就把config里parallel_output设置为False。如果是要用的话看起来是FA没装上，import flash_attn看看什么结果

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sunpengsdu sunpengsdu

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sunpengsdu @li126com @wen020