Info about other tokenizer

[chri002]

Amazing work.
An info about the tokenizer, in the tokenizer and embedded code there are references about Qwen2-0.5B-Instruct and 1.5B; I wanted to know if there have been any tests using them and if so, what precision and fidelity they achieved.
Because using them could greatly reduce the VRAM.

An other info, Is there any chance that an inference with offloading of the various models is planned, for low end PCs?

[lawrence-cj]

Qwen is not tested in Sana. And what do you mean by offloading for low end PCs?

[chri002]

Thanks for the info on qwen, regarding offloading I mean loading and unloading the various parts or layers into RAM so as to fit Sana into a 8 or 6 gb GPU (and maybe even 4gb) besides running for example vae on CPU or even part of gemma. After all gemma also runs on 6gb desktop GPU and android with 8gb RAM.

[bil-ash]

Qwen is not tested in Sana. And what do you mean by offloading for low end PCs?

@lawrence-cj However, the diffusion/model/builder.py file seems to contain code for qwen2(0.5b and 1.5b instruct) as text encoder. Also, train_scripts/train.py file seems to suggest that it is possible to train sana with qwen and t5 also as text encoder. If some version of sana has been trained with qwen2-0.5b-instruct as text encoder, please release that- would love to see the results from such a tiny text encoder.

[lawrence-cj]

We tried qwen once—just a prototype and no release plan for it since didn't see any advantage rather to gemma or T5. So why qwen is necessary here?

[bil-ash]

We tried qwen once—just a prototype and no release plan for it since didn't see any advantage rather to gemma or T5. So why qwen is necessary here?

I thought that using qwen(0.5B instruct) instead of gemma(2B-it) as text encoder would lead to lower resource consumption, that's why was asking

[lawrence-cj]

We tried T5-large once, which contains 400M parameters. But the ability is relatively low. So we haven't tried other LLM or VLM less than 1B parameters. We are also working on a 4-bit Gemma quantization. Once it's done, the whole pipeline will run on 8GB GPUs, so resource issue is not the hightest priority. We may use Qwen, if we have a plan for a small model in the future.

Refer to the revised version of Sana in Openreview: