Missing Transformers initializer for Falcon models

[martin-gorner]

Repro code:
model5 = keras_hub.models.CausalLM.from_preset("hf://tiiuae/falcon-7b-instruct", dtype="bfloat16")
Result:
ValueError: KerasHub has no converter for huggingface/transformers models with model type 'falcon'

Now that the Falcon model family exists in Keras-hub, this should work.

[mehtamansi29]

Hi @martin-gorner -

Thanks for reporting the issue. You can intialize falcon-7b-instruct model using transformers AutoTokenizer, AutoModelForCausalLM class.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "tiiuae/falcon-7b-instruct"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto") 

And for loading falcon model family(falcon_refinedweb_1b_en) in keras-hub, you can use like this.
model5 = keras_hub.models.CausalLM.from_preset("hf://keras/falcon_refinedweb_1b_en", dtype="bfloat16")

Attached gist here for the reference.

[github-actions]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

[martin-gorner]

Thanks @mehtamansi29 but this issue is filed in keras-hub. The problem is to initialize a Keras-hub model from the safetensor checkpoint, as is possible for Llama, Gemma etc. I'm logging this because cross-compatibility between Keras-hub and Transformers checkpoints is not guaranteed even if the model architecture exists on both sides. A checkpoint translation module is necessary. It is important to keep track of model architectures where this translation module was implemented and where it was not.