Confused about number of steps

[cinjon]

Hi, I saw your training curve for Gemma 9b SimPO here: https://wandb.ai/yumeng0818/simpo/runs/4w25j650?nw=nwuseryumeng0818.
How is it that there's only 92 steps? At 128 batch size, that would only be 11k total examples seen, but there's ~60k in the dataset.
Thanks.

[cchenv]

Hi @cinjon did you figure it out? It's confusing. Also the actual batch size seems to be 256 (2 * 8 * 16), so there should be about 232 steps for 1 epoch.

[cinjon]

• Still confused but our training runs are reasonable, so I gave up trying to guess theirs.
• Yeah I was confused if it was 128 or 256.
• I'm also confused about their eval templates and scores on Gemma.

[cchenv]

@cinjon I tried to use TRL's implementation (https://huggingface.co/docs/trl/cpo_trainer#simple-preference-optimization-simpo) for training runs, but I cannot reproduce their Gemma2-9B-it-SimPO model. The resulting model after 1 epoch on the dataset is so much worse. I noticed there is another PR by the authors to create a separate SimPOTrainer with TRL: huggingface/trl#1725 I hope that can fix the issues.