Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add Adversarial Diffusion Distillation (ADD) Script #6303

Open
wants to merge 58 commits into
base: main
Choose a base branch
from
Open

[WIP] Add Adversarial Diffusion Distillation (ADD) Script #6303

Show file tree
Hide file tree
Changes from 51 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
ecfadbc
Initial commit for ADD SD distillation script.
dg845 Dec 23, 2023
d2316da
make style
dg845 Dec 23, 2023
ca90b96
Fix bug where train_shards_path_or_url arg is duplicated.
dg845 Dec 24, 2023
29706fd
Add SD-XL version of ADD script.
dg845 Dec 24, 2023
1944a1c
Fix bugs in ADD SD distill script.
dg845 Dec 24, 2023
281e72a
make style
dg845 Dec 24, 2023
a925613
apply suggestions from review
dg845 Dec 25, 2023
6d02837
make style
dg845 Dec 25, 2023
100ef46
Merge branch 'main' into add-distill-script
dg845 Dec 26, 2023
60a9ea7
Use DDPMScheduler instead of DDIMScheduler for noise_scheduler since …
dg845 Dec 26, 2023
c706244
Implement SDS weighting and fix bug in exponential weighting function.
dg845 Dec 26, 2023
76fa403
Use teacher CFG estimate as distillation target.
dg845 Dec 27, 2023
840f7dd
make style
dg845 Dec 27, 2023
1a6569f
Add option to also maintain EMA version of student U-Net parameters.
dg845 Dec 27, 2023
af31aa7
make style
dg845 Dec 27, 2023
969823f
apply suggestions from review
dg845 Dec 27, 2023
ddbcd7d
Fix some bugs.
dg845 Dec 27, 2023
9fa6133
make style
dg845 Dec 27, 2023
080e857
Add ema_min_decay argument, which defaults to ema_decay, resulting in…
dg845 Dec 28, 2023
a1913d5
Fix shape bug in DiscriminatorHead.
dg845 Dec 29, 2023
f2e5f6d
Use CLIP pooled outputs instead of last hidden states (prompt_embeds)…
dg845 Dec 29, 2023
97223e1
Fix bug in discriminator R1 penalty implementation.
dg845 Dec 29, 2023
e9a1db8
make style
dg845 Dec 29, 2023
9f85686
Fix bugs in SD-XL ADD script.
dg845 Dec 29, 2023
9cda52e
Add option to use CLIPTextModelWithProjection model in SD ADD script.
dg845 Dec 30, 2023
7d7f98b
make style
dg845 Dec 30, 2023
8675850
Allow Discriminator to optionally take in image conditioning informat…
dg845 Dec 30, 2023
fd39891
make style
dg845 Dec 30, 2023
35e45e6
Change default feature network to DINOv2 ViT-S (see section 4.1/Table…
dg845 Dec 30, 2023
d7459b0
make style
dg845 Dec 30, 2023
266474e
Add option to use image conditioning in discriminator.
dg845 Dec 31, 2023
1773de3
make style
dg845 Dec 31, 2023
d5b9619
In discriminator heads, concatenate conditioning embeddings and map a…
dg845 Dec 31, 2023
b54c664
Fix bug when calculating image conditioning embeddings.
dg845 Dec 31, 2023
f124b03
Merge branch 'main' into add-distill-script
dg845 Jan 1, 2024
275cc8a
Add batched VAE decoding and make VAE encoding/decoding batch size co…
dg845 Jan 1, 2024
aa9cc41
make style
dg845 Jan 1, 2024
da6c5ac
Change validation prompts to example prompts from the ADD paper.
dg845 Jan 1, 2024
db358dd
Fix interpolation type bug.
dg845 Jan 1, 2024
9d46259
[Experimental] add scripts to use ADD to distill a LoRA rather than a…
dg845 Jan 7, 2024
c8c1e83
make style
dg845 Jan 7, 2024
9ff0fa3
Merge branch 'main' into add-distill-script
dg845 Jan 7, 2024
b64b700
Use diffusers.training_utils.resolve_interpolation_mode after PR #642…
dg845 Jan 7, 2024
bb0619f
Add allow_nonzero_terminal_snr argument to disable enforcing zero ter…
dg845 Jan 7, 2024
5cfd137
Use ConfigMixin.from_config with rescale_betas_zero_snr kwarg when cr…
dg845 Jan 8, 2024
acf5175
Allow noise_scheduler to be configured between ddpm and euler.
dg845 Jan 8, 2024
cd82565
Return features from Discriminator and use L2 gradient penalty (haven…
dg845 Jan 14, 2024
ab46142
[not sure if correct] Add tentative fixed implementation of discrimin…
dg845 Jan 15, 2024
5121342
Fix checkpointing bug where discriminator was not being properly save…
dg845 Jan 15, 2024
286b8c5
Propagate discriminator R1 penalty fix to other scripts.
dg845 Jan 15, 2024
686af03
Fix R1 grad penalty L2 norm implementation.
dg845 Jan 15, 2024
807150c
Fix bug when performing validation in the LoRA scripts.
dg845 Jan 21, 2024
8ff02b3
Fix bugs when using image conditioning for the discriminator.
dg845 Jan 21, 2024
7fe135b
Merge branch 'main' into add-distill-script
dg845 Jan 21, 2024
5e97799
Merge branch 'main' into add-distill-script
dg845 Feb 6, 2024
15c292d
Disable SD 1.X/2.X safety checker when doing validation.
dg845 Feb 8, 2024
d9a0379
When rescaling to zero terminal SNR replace last alpha with small pos…
dg845 Feb 10, 2024
bc4b516
For SD 1.X/2.X get last_hidden_state of uncond_prompt_embeds correctl…
dg845 Feb 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions examples/add/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
accelerate>=0.16.0
torchvision
transformers>=4.25.1
ftfy
tensorboard
Jinja2
webdataset
timm
Loading
Loading