Update TUTORIAL.md (#957)

* Update TUTORIAL.md fix indentation problem * Update TUTORIAL.md --------- Co-authored-by: Mihir Patel <[email protected]> Co-authored-by: Daniel King <[email protected]>
mosaicml · Feb 7, 2024 · fe17c22 · fe17c22
1 parent 8c7d6f4
commit fe17c22
Showing 1 changed file with 0 additions and 2 deletions.
diff --git a/TUTORIAL.md b/TUTORIAL.md
@@ -371,8 +371,6 @@ model:
       target_modules:
       - q_proj
       - k_proj
-    target_modules:
-    - 'Wqkv'
 ```
 - For efficiency, The MPT model concatenates the `Q`, `K`, and `V` matrices in each attention block into a single `Wqkv` matrix that is three times wider. Currently, LoRA supports a low-rank approximation to this `Wqkv` matrix.
 - When evaluating with PEFT / LoRA separated weight, just set `pretrained_lora_id_or_path` in `model`(Find an example [here](scripts/eval/yamls/hf_lora_eval.yml#L19)).