Bug fixes for llava multimodal

[szelok]

Checklist:

• I have read the Contributing guidelines.

[oobabooga]

@szelok could you merge the dev branch check if the changes in this PR are still necessary?

[Victorivus]

@oobabooga

@szelok could you merge the dev branch check if the changes in this PR are still necessary?

I just did, only dev branch does not work:

After modifying modules/models.py with this PR's content, it works:

Note: I didn't touch the openai/completions.py

[randoentity]

@oobabooga
This is still relevant.
On commit a363cdf (origin/dev)
Date: Mon May 27 15:21:30 2024 +0300

Fixed with cherry-pick 3af2cfbc3c198ddd6b351c27af868287c5eb354d from this PR.

One related error is when using --gpu-memory in combination with --load-in-8bit: LlavaLlamaForCausalLM doesn't define from_config. But it works fine when not using bitsandbytes or when not specifying gpu-memory.

[FoxMcloud5655]

Can confirm that what @randoentity said holds true. Applying just the changes to modules/models.py in the PR fixes the issue.