Skip to content

Commit

Permalink
fix qlora yml for phi, fix phi test validation
Browse files Browse the repository at this point in the history
  • Loading branch information
winglian committed Jan 23, 2024
1 parent 06f1c55 commit ee608fe
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 4 deletions.
2 changes: 0 additions & 2 deletions examples/phi/phi-qlora.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
base_model: microsoft/phi-1_5
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
is_llama_derived_model: false
trust_remote_code: true

load_in_8bit: false
load_in_4bit: true
Expand Down
4 changes: 2 additions & 2 deletions tests/test_validation.py
Original file line number Diff line number Diff line change
Expand Up @@ -759,7 +759,7 @@ def test_phi_add_tokens_adapter(self):
"adapter": "qlora",
"load_in_4bit": True,
"tokens": ["<|imstart|>"],
"lora_modules_to_save": ["embed_tokens", "lm_head"],
"lora_modules_to_save": ["embd.wte", "lm_head.linear"],
}
)

Expand All @@ -774,7 +774,7 @@ def test_phi_add_tokens_adapter(self):
"adapter": "qlora",
"load_in_4bit": True,
"tokens": ["<|imstart|>"],
"lora_modules_to_save": ["embd.wte", "lm_head.linear"],
"lora_modules_to_save": ["embed_tokens", "lm_head"],
}
)

Expand Down

0 comments on commit ee608fe

Please sign in to comment.