New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RTP-LLM 模式下，llama3.1 FP16 效果不一样 #114

Open

anigi98932 opened this issue Sep 13, 2024 · 1 comment

anigi98932 commented Sep 13, 2024

使用huggingface载入 llama3.1的生成结果，和RTP-LLM 载入结果不一致
同样的prompt，无法重现在RTP-LLM

Collaborator

ySingularity commented Nov 18, 2024

对比一下input id和first token的hidden states?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment