Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SUBMISSION] December 2024 - Module 2: Preference Alignment #133

Open
wants to merge 53 commits into
base: december-2024
Choose a base branch
from

Conversation

jucanbe
Copy link

@jucanbe jucanbe commented Dec 22, 2024

December 2024 Student Submission

Module Completed

  • Module 1: Instruction Tuning
  • Module 2: Preference Alignment
  • Module 3: Parameter-efficient Fine-tuning
  • Module 4: Evaluation
  • Module 5: Vision-language Models
  • Module 6: Synthetic Datasets
  • Module 7: Inference
  • Module 8: Deployment

Changes Made

Describe what you've done in this PR:

  1. What concepts did you learn?
    How to do DPO and ORPO, and how to work with the datasets in each case
  2. What changes or additions did you make?
    Working with a different dataset in the ORPO case, and testing the models in each case
  3. Any challenges you faced?

Notebooks Added/Modified

List any notebooks you've added or modified:

  • Added new example in module_name/student_examples/my_example.ipynb
  • Modified existing notebook with additional examples
  • Added documentation or comments

Checklist

  • I have read the module materials
  • My code runs without errors
  • I have pushed models and datasets to the huggingface hub
  • My PR is based on the december-2024 branch

Questions or Discussion Points

Add any questions you have or points you'd like to discuss:
1.
2.

Additional Notes

Any other information that might be helpful for reviewers:

nsbg and others added 30 commits December 12, 2024 21:30
[TRANSLATION] Add 3_parameter_efficient_tuning EN-KO translation
…_tutorials

Add SmolVLM TRL recipes to VLM resources (SFT/DPO)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants