DPMSolverMultistep add rescale_betas_zero_snr

[Beinsezii]

Effectively identical to the implementation for EulerDiscrete in #6024

TL;DR:

• rescale_zero_terminal_snr function copied from DDIM
• Fudge the last cumprod to 2**-24 from 0 when using zsnr so it doesn't produce an inf sigma
• Upcast samples to torch.float32 for the duration of step() because the zsnr sigmas are substantially less numerically stable and the performance hit is negligible.
  • Also has the side effect of producing slightly better images overall
• use_karras_sigmas=True still produces strange results as it does in EulerDiscrete. Might be worth investigating separately?
• Simple true/false UT for the new config value

Demo images

use_karras_sigmas also seems to just generally produce bad/noisy results even without ZSNR so I might not have the config set up correctly?

[yiyixuxu]

oh thanks
left a question about dtype

[yiyixuxu]

do you mean model_output.dtype isn't always the same as sample.dtype before the upcast?

[Beinsezii]

Turns out that was because model_output = self.convert_model_output(model_output, sample=sample) ends up creating a shadowed tensor cast to the sample's dtype. I moved the sample upcast after this call so return_type is no longer needed. Outputs are the same.

[yiyixuxu]

use_karras_sigmas=True still produces strange results as it does in EulerDiscrete. Might be worth investigating separately?

yes separate issue please:)

[yiyixuxu]

thanks!

[yiyixuxu]

can you fix the quality test?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Beinsezii]

Alright I added a single newline.

[yiyixuxu]

thank you!

[lachlan-nicholson]

use_karras_sigmas=True still produces strange results as it does in EulerDiscrete. Might be worth investigating separately?

yes separate issue please:)

Have we created a separate issue for ZeroSNR + Karras?
At a glance the max lambda is very large and could be clipped similar to #3314, potentially via config.sigma_max.
This gives viable results but introduces a new parameter.
I'm left wondering what it means to apply both ZeroSNR and Karras modifiers at the same time.