ERROR: The function received no value for the required argument: lora_model_name_or_path #2

pereconteur · 2023-04-01T18:39:39Z

Hi,

I would like to combine your model with Alpaca's.
I have installed everything, and made a git clone of the lora model repository at 7B.

Only, despite the path to it no way to continue, am I missing something ?

python vigogne/scripts/export_state_dict_checkpoint.py \
    --base_model_name_or_path <../alpaca.cpp/ggml-alpaca-7b-q4.bin> \
    --lora_model_name_or_path "vigogne-lora-7b" \
    --output_dir ./models/7B
ERROR: The function received no value for the required argument: lora_model_name_or_path
Usage: export_state_dict_checkpoint.py BASE_MODEL_NAME_OR_PATH LORA_MODEL_NAME_OR_PATH OUTPUT_DIR

For detailed information on this command, run:
  export_state_dict_checkpoint.py --help

Tree of Vigogne-Lora :

vigogne-lora-7b
├── README.md
├── adapter_config.json
├── adapter_model.bin
├── runs
│   └── Mar26_00-12-50_koios.zaion.ai
│       ├── 1679785970.575404
│       │   └── events.out.tfevents.1679785970.koios.zaion.ai
│       └── events.out.tfevents.1679785970.koios.zaion.ai
├── tokenizer.model
└── vigogne_logo.png

The text was updated successfully, but these errors were encountered:

bofenghuang · 2023-04-03T07:47:12Z

Hi @pereconteur!

Unfortunately, you cannot merge the Vigogne-LoRA weights with the Alpaca model.

The base_model_name_or_path parameter should include model files for the LLaMA model in Hugging Face's format.

Thank you for asking. I will update the README to clarify this!

pereconteur · 2023-04-04T19:21:21Z

Salut je suppose que tu comprend le francais.

J'ai téléchargé le fichier : llama-7 B-hf, qui à mon avis est le modèle de Llama au format Hugging Face's :

llama-7b-hf
├── LICENSE
├── README.md
├── config.json
├── generation_config.json
├── gitattributes
├── pytorch_model-00001-of-00033.bin
├── pytorch_model-00002-of-00033.bin
├── pytorch_model-00003-of-00033.bin
├── pytorch_model-00004-of-00033.bin
├── pytorch_model-00005-of-00033.bin
├── pytorch_model-00006-of-00033.bin
├── pytorch_model-00007-of-00033.bin
├── pytorch_model-00008-of-00033.bin
├── pytorch_model-00009-of-00033.bin
├── pytorch_model-00010-of-00033.bin
├── pytorch_model-00011-of-00033.bin
├── pytorch_model-00012-of-00033.bin
├── pytorch_model-00013-of-00033.bin
├── pytorch_model-00014-of-00033.bin
├── pytorch_model-00015-of-00033.bin
├── pytorch_model-00016-of-00033.bin
├── pytorch_model-00017-of-00033.bin
├── pytorch_model-00018-of-00033.bin
├── pytorch_model-00019-of-00033.bin
├── pytorch_model-00020-of-00033.bin
├── pytorch_model-00021-of-00033.bin
├── pytorch_model-00022-of-00033.bin
├── pytorch_model-00023-of-00033.bin
├── pytorch_model-00024-of-00033.bin
├── pytorch_model-00025-of-00033.bin
├── pytorch_model-00026-of-00033.bin
├── pytorch_model-00027-of-00033.bin
├── pytorch_model-00028-of-00033.bin
├── pytorch_model-00029-of-00033.bin
├── pytorch_model-00030-of-00033.bin
├── pytorch_model-00031-of-00033.bin
├── pytorch_model-00032-of-00033.bin
├── pytorch_model-00033-of-00033.bin
├── pytorch_model.bin.index.json
├── special_tokens_map.json
├── tokenizer(1).model
└── tokenizer_config.json

Dit moi si c'est bien ca.

Si c'est ca, j'ai effectué la commande suivante :

python export_state_dict_checkpoint.py \
    --base_model_name_or_path <../../llama-7b-hf/> \
    --lora_model_name_or_path "bofenghuang/vigogne-lora-7b" \
    --output_dir vigogne/scripts/./models/7B \
    --checkpoint_size "7b"

et j'ai cette erreur qui survient :

Fatal Python error: init_sys_streams: <stdin> is a directory, cannot continue
Python runtime state: core initialized

Current thread 0x00000001fdebdb40 (most recent call first):
  <no Python frame>

Une idée pour m'aider stp, j'adorerais utilise une version francaise.

Merci ❤️

bofenghuang · 2023-04-06T14:37:04Z

HI @pereconteur ,

Désolé pour la réponse tardive.

Pourrais-tu réssayer avec la commande suivante ?

python export_state_dict_checkpoint.py \
    --base_model_name_or_path ../../llama-7b-hf \
    --lora_model_name_or_path "bofenghuang/vigogne-lora-7b" \
    --output_dir vigogne/scripts/./models/7B \
    --checkpoint_size "7b"

pereconteur · 2023-04-06T14:51:37Z

Salut,

C'est bon j'ai réussi à le faire fonctionner, enfaite je n'avais pas les bons poids pour Llama et pour ceux que ça intéresse j'ai trouvé les poids ici :

https://github.com/cocktailpeanut/dalai

Et il faut que le dossier à 7B correspond à ces hashes :

7935c843a25ae265d60bf4543b90bfd91c4911b728412b5c1d5cff42a3cd5645  ./models/7B/checklist.chk
700df0d3013b703a806d2ae7f1bfb8e59814e3d06ae78be0c66368a50059f33d  ./models/7B/consolidated.00.pth
7e89e242ddc0dd6f060b43ca219ce8b3e8f08959a72cb3c0855df8bb04d46265  ./models/7B/params.json

Après ça tout roule !

Merci pour ton repository !

pereconteur · 2023-04-06T14:53:03Z

Tu pourrais m'expliquer comment t'a fait pour la fine tuner ? Et peut-on fine tuner un modèle avec un fichier .json avec des instructions, input et output ?

esokullu · 2023-04-10T03:20:40Z

Hello! Here's the error I get with export_state_dict_checkpoint.py

on CPU;
Loading checkpoint shards: 100%
...
then: RuntimeError: shape '[32, 2, 64, 4096]' is invalid for input of size 26214400

on GPU;
Loading checkpoint shards: 100%
File "scripts/export_state_dict_checkpoint.py", line 128, in
fire.Fire(main)
File "/opt/conda/lib/python3.8/site-packages/fire/core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/opt/conda/lib/python3.8/site-packages/fire/core.py", line 475, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/opt/conda/lib/python3.8/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "scripts/export_state_dict_checkpoint.py", line 115, in main
new_state_dict[new_k] = unpermute(v)
File "scripts/export_state_dict_checkpoint.py", line 71, in unpermute
return w.view(n_heads, 2, dim // n_heads // 2, dim).transpose(1, 2).reshape(dim, dim)
RuntimeError: shape '[32, 2, 64, 4096]' is invalid for input of size 26214400

this is on 13B.

I get the same error on both. No idea what's going on!?

esokullu · 2023-04-10T03:23:21Z

here's what chatgpt says :)

The error you're encountering is a shape mismatch error when reshaping the tensor in both CPU and GPU environments. The script is trying to reshape the tensor to [32, 2, 64, 4096], which would result in a tensor of size 16,777,216 elements. However, the input tensor has a size of 26,214,400 elements, which is not compatible with the target shape.

Here are a few suggestions to help debug the issue:

Verify the dimensions: Double-check the dimensions and shapes of the tensors you are working with. Make sure that the dimensions of the input tensor are correct and correspond to the expected dimensions for the model you are using.

Inspect the script: Look closely at the export_state_dict_checkpoint.py script, specifically at the unpermute function. Check if there are any hard-coded values that may be causing the shape mismatch. If you find any, update them to the appropriate values based on the model you are using.

Check the model architecture: Make sure you are using the correct model architecture (13B) and that the script is compatible with this specific architecture.

Update the script (if necessary): If the script you are using is outdated or not compatible with the 13B model, try to find an updated version of the script or modify it to handle the model correctly.

Verify the checkpoint: Make sure that the checkpoint file you are loading is not corrupted or mismatched. If you are unsure, try downloading the checkpoint file again.

So I guess it's an issue with 13B? I've completed all steps prior successfully. Will try again with a smaller model, if that is not an error with the script.

bofenghuang · 2023-04-17T09:37:32Z

Hi @esokullu,

It seems like it used the parameters of 7B model to export a 13B. Did you specify the --base_model_size 13B when running export_state_dict_checkpoint.py? For more information please refer to this section.

bofenghuang · 2023-04-17T09:41:55Z

Hi @pereconteur,

Yes, you just need to format your data to a JSON file with instruction, input, and output, then launch the fine-tuning using this command

pereconteur · 2023-04-18T20:53:13Z

Ok et c'est possible de fine-tunner avec un Macbook Pro M1 ? Car j'ai cette erreur lorsque j'essaie ta commande :

ValueError: FP16 Mixed precision training with AMP or APEX (`--fp16`) and FP16 half precision evaluation (`--fp16_full_eval`) can only be used on CUDA devices.

bofenghuang · 2023-04-20T12:33:42Z

I've never tried with M1. But maybe you can take a look at here

pereconteur · 2023-04-20T12:37:15Z

Ok merci, je vais tester ça et je te tiens au courant !

J'ai fait une vidéo Youtube sur ta solution je te mets le lien ci-dessous si tu veux la visionner, en espérant t'avoir fait une belle pub sur tes travaux.

https://www.youtube.com/watch?v=BBf5h0HCFMY&t=292s

bofenghuang · 2023-04-20T12:49:57Z

Merci beaucoup pour cette vidéo ! C'est super clair et précis !

Do you mind if I put the link in the README?

pereconteur · 2023-04-20T12:52:35Z

Non au contraire ! Ça me fait super plaisir de pouvoir contribuer à ton projet !

Je te pose toute ces questions parce que ma prochaine vidéo portera sur la fine-tuning, et j'aimerais en montrer l'exemple avec ton repository. Mais vu que j'ai un mac, et donc pas de carte Nvidia, je ne peux pas utiliser CUDA pour la fine-tuning et donc je cherche une solution pour le faire sur mon mac tu comprend ?

bofenghuang · 2023-04-20T13:35:29Z

Unfortunately, running PyTorch on M1 may be unfeasible because it is primarily optimized for CUDA. Have you considered using a GPU cloud instead?

pereconteur · 2023-05-02T14:07:24Z

Ok,

I have a request. To make my video tutorial on fine-tuning your repo, I need to show how to do it with your finetune.py file, would it be possible for you to film your screen doing the steps, and send them to me, for the purpose of the video?
I would like to show how to modify the json file and then teach it what to put in the json file and then show the result.

For example, I wanted to put in the Json file the instruction: "Donne moi le top 10 du ballon d'or 2016 (football)" and that it gives me the answer that I put in the output. Showing the answer before and after the fine-tuning.

You could accept my request, because I don't have an NVIDIA card and I'm struggling 🥲.

My email : [email protected]

pereconteur · 2023-05-03T11:27:53Z

You would agree ? I would be very grateful and it would allow me to see how it works! @bofenghuang

bofenghuang · 2023-05-03T16:26:29Z

Hi @pereconteur!

Sure, I am happy to help you with your request!

However, I want to let you know that I am currently working on some new features that will affect the structure of this repository. It might be more beneficial for you to wait until these changes are implemented before proceeding with the video tutorial.

pereconteur · 2023-05-03T18:48:13Z

Hi @bofenghuang,

Okay. When do you think it will be ready?
And can you make a step-by-step video with the example I gave for my viewers, so I can explain how to do it? They are very eager to know how to fine-tune!

pereconteur · 2023-05-15T22:01:14Z

Hi @bofenghuang,

I see you updated your repo! I'm very happy that you succeeded!

By the way, I would like to know if you could send me what I asked you? My subscribers are eager to know the steps to finish with your repo, thank you!

FxNion · 2023-08-10T11:17:07Z

Hi @bofenghuang

So much thanks for your job, il really believe that's a good way to play with french llm models.

I follow the wondefull instructions of @pereconteur to tranform vigogne model to llama.cpp models, in order to play with it on MacBook M1

It seems that "bofenghuang/vigogne-lora-7b" is no more available, as the command:

python export_state_dict_checkpoint.py 
             --base_model_name_or_path ../../llama-7B-hf 
             --lora_modele_name_or_path "bofenghuang/vigogne-lora-7b" 
             --output_dir ./models/7B --base_model_size "7B"

No more success with "bofenghuang/vigogne-7b-instruct"

python export_state_dict_checkpoint.py 
             --base_model_name_or_path ../../llama-7B-hf 
             --lora_modele_name_or_path "bofenghuang/vigogne-7b-instruct" 
             --output_dir ./models/7B --base_model_size "7B"

which both terminate by a sadly

Could not consume arg: --lora_modele_name_or_path

I've read you where tranforming (^^) the repo, and may be there is no backward compatibility with the old tutorial ?

Thanks again

pereconteur · 2023-08-10T14:34:59Z

Hi @bofenghuang

So much thanks for your job, il really believe that's a good way to play with french llm models.

I follow the wondefull instructions of @pereconteur to tranform vigogne model to llama.cpp models, in order to play with it on MacBook M1

It seems that "bofenghuang/vigogne-lora-7b" is no more available, as the command:
python export_state_dict_checkpoint.py 

             --base_model_name_or_path ../../llama-7B-hf 

             --lora_modele_name_or_path "bofenghuang/vigogne-lora-7b" 

             --output_dir ./models/7B --base_model_size "7B"
No more success with "bofenghuang/vigogne-7b-instruct"
python export_state_dict_checkpoint.py 

             --base_model_name_or_path ../../llama-7B-hf 

             --lora_modele_name_or_path "bofenghuang/vigogne-7b-instruct" 

             --output_dir ./models/7B --base_model_size "7B"
which both terminate by a sadly
Could not consume arg: --lora_modele_name_or_path
I've read you where tranforming (^^) the repo, and may be there is no backward compatibility with the old tutorial ?

Thanks again

Hi @FxNion !

I have a YouTube channel where I give lots of tutorials, including how to install Vigogne : https://youtu.be/BBf5h0HCFMY

Nevertheless I advise you first to join my Discord, where I update as much as possible all the new way to install, you'll just have to browse the different forum hello : https://discord.gg/xY63gyVfaR

I hope this helps and don't hesitate to give me your feedback!

FxNion · 2023-08-10T14:58:51Z

Salut @pereconteur ;)

C'est suite à ta super vidéo que j'ai atterri ici, pour tenter de résoudre les problèmes que j'ai rencontrés en tentant de marcher dans tes pas.

Et je suis aussi sur ton Discord, ou j'ai posé la même question.

Merci encore, et à bientôt !

pereconteur · 2023-08-10T15:03:09Z

Ok dans ce cas là > Salut @pereconteur ;)

C'est suite à ta super vidéo que j'ai atterri ici, pour tenter de résoudre les problèmes que j'ai rencontrés en tentant

de marcher dans tes pas.

Et je suis aussi sur ton Discord, ou j'ai posé la même question.

Merci encore, et à bientôt !

Merci beaucoup pour ton retour ! Dans ce cas là clique sur ce lien et si il ne marche pas, pose ta question dans la catégorie IA sans entraide : https://discord.com/channels/1092039071435599874/1113928879481032724

FxNion · 2023-08-10T16:19:55Z

Waouh @pereconteur, super boulot avec ce script. Je vais tester ça !

Merci pour ton aide

FxNion · 2023-08-10T16:54:10Z

@pereconteur a priori les urls des adapters_config.json ont changé, ça ne doit pas être grand-chose.

Entry Not Found for url: https://huggingface.co/bofenghuang/vigogne-7b-instruct/resolve/main/adapter_config.json

et

Entry Not Found for url: https://huggingface.co/bofenghuang/vigogne-7b-chat/resolve/main/adapter_config.json.

bofenghuang · 2023-08-16T09:46:55Z

Hi @FxNion,

Thanks for your interest! The repository evolves rapidly, affecting backward compatibility. You could refer to the latest documentation here for reference.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ERROR: The function received no value for the required argument: lora_model_name_or_path #2

ERROR: The function received no value for the required argument: lora_model_name_or_path #2

pereconteur commented Apr 1, 2023 •

edited

Loading

bofenghuang commented Apr 3, 2023

pereconteur commented Apr 4, 2023 •

edited

Loading

bofenghuang commented Apr 6, 2023

pereconteur commented Apr 6, 2023

pereconteur commented Apr 6, 2023

esokullu commented Apr 10, 2023

esokullu commented Apr 10, 2023

bofenghuang commented Apr 17, 2023

bofenghuang commented Apr 17, 2023

pereconteur commented Apr 18, 2023

bofenghuang commented Apr 20, 2023

pereconteur commented Apr 20, 2023

bofenghuang commented Apr 20, 2023

pereconteur commented Apr 20, 2023

bofenghuang commented Apr 20, 2023

pereconteur commented May 2, 2023 •

edited

Loading

pereconteur commented May 3, 2023 •

edited

Loading

bofenghuang commented May 3, 2023

pereconteur commented May 3, 2023

pereconteur commented May 15, 2023

FxNion commented Aug 10, 2023 •

edited

Loading

pereconteur commented Aug 10, 2023

FxNion commented Aug 10, 2023

pereconteur commented Aug 10, 2023

FxNion commented Aug 10, 2023

FxNion commented Aug 10, 2023

bofenghuang commented Aug 16, 2023

ERROR: The function received no value for the required argument: lora_model_name_or_path #2

ERROR: The function received no value for the required argument: lora_model_name_or_path #2

Comments

pereconteur commented Apr 1, 2023 • edited Loading

bofenghuang commented Apr 3, 2023

pereconteur commented Apr 4, 2023 • edited Loading

bofenghuang commented Apr 6, 2023

pereconteur commented Apr 6, 2023

pereconteur commented Apr 6, 2023

esokullu commented Apr 10, 2023

esokullu commented Apr 10, 2023

bofenghuang commented Apr 17, 2023

bofenghuang commented Apr 17, 2023

pereconteur commented Apr 18, 2023

bofenghuang commented Apr 20, 2023

pereconteur commented Apr 20, 2023

bofenghuang commented Apr 20, 2023

pereconteur commented Apr 20, 2023

bofenghuang commented Apr 20, 2023

pereconteur commented May 2, 2023 • edited Loading

pereconteur commented May 3, 2023 • edited Loading

bofenghuang commented May 3, 2023

pereconteur commented May 3, 2023

pereconteur commented May 15, 2023

FxNion commented Aug 10, 2023 • edited Loading

pereconteur commented Aug 10, 2023

FxNion commented Aug 10, 2023

pereconteur commented Aug 10, 2023

FxNion commented Aug 10, 2023

FxNion commented Aug 10, 2023

bofenghuang commented Aug 16, 2023

pereconteur commented Apr 1, 2023 •

edited

Loading

pereconteur commented Apr 4, 2023 •

edited

Loading

pereconteur commented May 2, 2023 •

edited

Loading

pereconteur commented May 3, 2023 •

edited

Loading

FxNion commented Aug 10, 2023 •

edited

Loading