Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The accuracy of MATH test set trained on 7B is not consistent with paper. #27

Open
caihaunqai opened this issue Jan 30, 2024 · 1 comment

Comments

@caihaunqai
Copy link

Hello,Using MetaMATH dataset and codes, I reproduce the experiments on 7B base model. Howerver, The accuracy I get is as follows, 17.14%, which is not consistent with yours 19.8%. Is there anything wrong with your results in the paper or anything wrong with my experiments? Can you help me? Thank you.

start=== 0 , end==== 9223372036854775807
length==== 5000 , acc==== 0.1714

@2018211801
Copy link

Hi, I train 100k subset of Metamath on 7B base model, while get 0 with math test and 0.5 with gsm8k. Is there anything wrong with my experiments?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants