Add support for Gemma chat template

[Haoxiang-Wang]

Description

Supports Gemma chat template for SFT. Currently, when users specify type=gemma in their YAML file. e.g.,

base_model: google/gemma-2b-it
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer

datasets:
  - path: HuggingFaceH4/ultrachat_200k
    conversation: gemma
    type: sharegpt.load_ultrachat
    split: "train_sft"
    train_on_split: "train_sft"

errors will occur, as the current version of axolotl doesn't support Gemma chat template as there is no if self.sep_style == SeparatorStyle.GEMMA in fastchat_conversation_turns.py

I added the following lines to fastchat_conversation_turns.py

    if self.sep_style == SeparatorStyle.GEMMA:
        if self.system_message:
            raise ValueError("Gemma chat template does not support system messages")
        for i, (role, message) in enumerate(self.messages):
            prefix = "<bos>" if i == 0 else ""
            message_str = message if message else ""
            yield prefix + "<start_of_turn>" + role + "\n", message_str + "<end_of_turn>\n"
        return

which will generate texts matching Gemma-instruct's chat template.

Besides, the FastChat latest version 0.2.36 doesn't support Gemma chat template, so I updated the requirements.txt to install the up-to-date FastChat from GitHub.

Motivation and Context

How has this been tested?

I tested it with

base_model: google/gemma-2b-it
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer

datasets:
  - path: HuggingFaceH4/ultrachat_200k
    conversation: gemma
    type: sharegpt.load_ultrachat
    split: "train_sft"
    train_on_split: "train_sft"

and it worked well.

Screenshots (if appropriate)

Types of changes

Social Handles (Optional)

[NanoCode012]

Could you give me an example output from running cli.preprocess with --debug? Does the bos repeat?

[NanoCode012]

Would it be possible to use a version instead of commit?

[winglian]

unfortunately, fastchat hasn't cut a release in a while too.

[NanoCode012]

I am a bit confused about this. I don't think you need to add bos manually like this. Axolotl should prepend it.

[winglian]

since this is Gemma specific, and this function doesn't have access to the tokenizer, I think this is fine for now. in other formats in this function, we have literal "" strings.