From bd6afbc0aba0ab4231a8d689002bf70171a94a63 Mon Sep 17 00:00:00 2001 From: "Seunghyuk Park (shepark)" Date: Tue, 17 Dec 2024 09:56:20 -0800 Subject: [PATCH] Fix run_generation test commands for TRL out usage example (#1621) Fix run_generation example for TRL out --- examples/trl/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/examples/trl/README.md b/examples/trl/README.md index 058a42ced..750fc82b0 100644 --- a/examples/trl/README.md +++ b/examples/trl/README.md @@ -229,7 +229,7 @@ steps like: To merge the adaptors into the base model we can use the `merge_peft_adapter.py` helper script that comes with TRL: ``` -python merge_peft_adapter.py --base_model_name="meta-llama/Llama-2-7b-hf" --adapter_model_name="dpo" --output_name="stack-llama-2" +python merge_peft_adapter.py --base_model_name="meta-llama/Llama-2-70b-hf" --adapter_model_name="dpo" --output_name="stack-llama-2" ``` which will also push the model to your HuggingFace hub account. @@ -239,7 +239,7 @@ which will also push the model to your HuggingFace hub account. We can load the DPO-trained LoRA adaptors which were saved by the DPO training step and run it through the [text-generation example](https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation). ``` -python run_generation.py \ +python ../gaudi_spawn.py --world_size 8 --use_deepspeed run_generation.py \ --model_name_or_path ../trl/stack-llama-2/ \ --use_hpu_graphs --use_kv_cache --batch_size 1 --bf16 --max_new_tokens 100 \ --prompt "Here is my prompt"