DPO code

[HaoshengZou]

Any plans on releasing the DPO code, or a brief intro of how you conducted long-context DPO?

[bys0318]

Hi! We refer to section 4.2 of our paper for details of DPO. We use the same codebase as ChatGLM-RLHF. We currently do not have plan to release the code and data for DPO.

[HaoshengZou]

Thanks for the reply!

ChatGLM-RLHF doesn't have code released. Did you modify Megatron-LM for long-context DPO, or use NeMo-Aligner, or other implementation?

[bys0318]

Hi, our DPO code is based on Megatron-LM.

[heyzude]

Hi, and thanks for sharing your work!

Could you elaborate more on which specific part of Megatraon-LM (https://github.com/NVIDIA/Megatron-LM) you used?