Stable Diffusion 3

[jchook]

Will it be possible to use kohya to train Stable Diffusion 3 LoRAs?

[AngryOldWang]

Will it be possible to use kohya to train Stable Diffusion 3 LoRAs?

+1

[NeoLoger]

Will it be possible to use kohya to train Stable Diffusion 3 LoRAs?

+1

[cassiebug23]

I'm wondering the same thing, waiting for kohya to support SD3 training!

[DenRakEiw]

+1 :)

[filliptm]

Bumpity

[ximinjie]

+1

[protector131090]

when can i train dreambooth? i need 3 0 training pleeeeeaseeee!

[5KilosOfCheese]

Will it be possible to use kohya to train Stable Diffusion 3 LoRAs?

I think we should wait for kohya-ss to update the script first. Bmaltais only made the GUI.

[oO0]

And we need merger and better captioner. Also motion lora training, VAE training, and converting 2B to 8B

[Charmandrigo]

It's too soon to ask for that dude

[sashaok123]

https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_sd3.md

[Bocchi-Chan2023]

kohya-ss/sd-scripts@e526828

[paboum]

kohya-ss/sd-scripts@e526828

Did you have a chance to run it? There are already some SD3 LoRAs on Civitai... One of which used https://github.com/bghira/simpletuner for training.

[trillgifted]

https://huggingface.co/blog/sd3#dreambooth-and-lora-fine-tuning

[zengjie617789]

https://huggingface.co/blog/sd3#dreambooth-and-lora-fine-tuning

Can you got better results using official trainning scripts? I trained it with my custom datasets, but the results are bad.

[oO0]

does scripts or Simpletuner works on Windows?

[BenDes21]

do we have an estimation for the sd3 update guy's ?

[x-CK-x]

do we have an estimation for the sd3 update guy's ?

This thread: https://huggingface.co/stabilityai/stable-diffusion-3-medium/discussions/98
seems to suggest that it's not quite working with the desired results yet, but it is in the works.

[5KilosOfCheese]

do we have an estimation for the sd3 update guy's ?

Lets just wait for Kohya to get it working proper first. If kohya-ss/sd-scripts#1374 is to be belived it requires quite bit work and rewriting the script.

So this repo will not get it, until Kohya has done it. This system is just a GUI for Kohya's scripts.

Just chill. It's summer, people who got life other than this, they might be on a vacation or busy with other stuff. This is also quite complex thing to pull off, and just making it work is not enough - you need to make it work in a way that produces results worth doing.

Meanwhile go work on your datasets, so you'll have them ready to go.

[bmaltais]

SD3 future is not bright. Most SAI staff have left and civitai has banned the model and any models that might be trained using images produced by SD3... so there might not be much future and interest to finetune SD3 now.

[DenRakEiw]

As I understand it, the Civitai ban isn't permanent; it's just in place until the Civitai lawyers have reviewed the license or until SAI changes a small part in the fine print.
The primary reason for the ban is to protect the community from legal issues.
I do think there's a lot of interest in the community to be able to fine-tune SD3 or at least train LoRAs for SD3.

As for the future of SAI, we'll see what happens there and if they can recover.

[5KilosOfCheese]

SD3 future is not bright. Most SAI staff have left and civitai has banned the model and any models that might be trained using images produced by SD3... so there might not be much future and interest to finetune SD3 now.

Unless people publish the datasets I think trying to enforce the rule of banning all models trained with images from SD3.
However, I think setting up the utilities to do this is still worth the effort. Since there is nothing wrong with using the model for non-commercial purposes.

I been having fun playing around with https://huggingface.co/common-canvas/CommonCanvas-XL-NC It isn't as bad as you'd think. It isn't what SDXL1 or even SD3 can do, but it's just version 0.1. However it's a model which is "legitimate" legally.

But there are interesting alternative architecture and models also available.

But lets not pretend that we didn't see this coming. I predicted this week after SD1.4 released, I'm amazed it took this long for something to happen on the legal front.

[bmaltais]

As I understand it, the Civitai ban isn't permanent; it's just in place until the Civitai lawyers have reviewed the license or until SAI changes a small part in the fine print. The primary reason for the ban is to protect the community from legal issues. I do think there's a lot of interest in the community to be able to fine-tune SD3 or at least train LoRAs for SD3.

As for the future of SAI, we'll see what happens there and if they can recover.

If Kohya add training support to sd-scripts I will certainly try to add support for it in the GUI… but I am not holding my breath.

[5KilosOfCheese]

As I understand it, the Civitai ban isn't permanent; it's just in place until the Civitai lawyers have reviewed the license or until SAI changes a small part in the fine print. The primary reason for the ban is to protect the community from legal issues. I do think there's a lot of interest in the community to be able to fine-tune SD3 or at least train LoRAs for SD3.
As for the future of SAI, we'll see what happens there and if they can recover.

If Kohya add training support to sd-scripts I will certainly try to add support for it in the GUI… but I am not holding my breath.

In the pull request on Kohya which I linked it seems like it is coming. However in kohya-ss/sd-scripts#1370 it would seems that the current knowledge on how to do SD3 LoRA training doesn't yield good results.

So as I emphasised in this discussion earlier. The community is going to need some patience. The fact that the script does get update, added to this QUI, doesn't mean that it produces anything worth the effort. Like it took a while people to realise that you shouldn't mess with the two text encoders in SDXL. It took a fair while for LoRA training to get good on the other models and lets be honest: there are still optimisations and improvements being made.

People really should just chill. Fact is that there is a good established base of models, and information on training which currently will yield superior results, because they been established and figured out. Because even if Kohya updates tomorrow, and you update the gui day after, people will bitch and cry about how the quality is bad, or training is slow, or it takes too much VRAM... until it all gets figured out, optimised and solved.

[BenDes21]

As I understand it, the Civitai ban isn't permanent; it's just in place until the Civitai lawyers have reviewed the license or until SAI changes a small part in the fine print. The primary reason for the ban is to protect the community from legal issues. I do think there's a lot of interest in the community to be able to fine-tune SD3 or at least train LoRAs for SD3.
As for the future of SAI, we'll see what happens there and if they can recover.

If Kohya add training support to sd-scripts I will certainly try to add support for it in the GUI… but I am not holding my breath.

sd3 added in the training scripts :

kohya-ss/sd-scripts@d53ea22