how to use flux loras with text encoder keys with diffusers?

[neuron-party]

resulting lora weights from setting train text encoder to true is incompatible with diffusers load_lora_weights. the script networks/convert_flux_lora.py does not convert the text encoder keys either.

[kohya-ss]

Unfortunately, I don't know what keys Diffusers' Text Encoder LoRA has. I would appreciate it if you could share a link to a sample weight file for Diffusers' Text Encoder LoRA.

[neuron-party]

@kohya-ss i'll train one next week and send it to you, but i don't think diffusers also trains t5xxl

[Phonicavi]

Sir, I thought that clip's part (i.e. te1 for sd-scripts, text_encoder_one for diffusers) has been processed in this commit, wish it could be helpful. I've trained LoRA with text encoder (no t5xxl, only clip) and transformer (including both double blocks and single blocks) via kohya-ss, which could be loaded normally using diffusers pipe.load_lora_weights method :)

[cvtower]

load_lora_weights works fine with this code for me.