Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Differential Diffusion to Auraflow. #9597

Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
eacae10
Modularized the train_lora file
satani99 May 27, 2024
b25aa27
Modularized the train_lora file
satani99 May 27, 2024
b56ce83
Modularized the train_lora file
satani99 May 27, 2024
086ac9d
Modularized the train_lora file
satani99 May 28, 2024
a859ae5
Modularized the train_lora file
satani99 May 28, 2024
d1fd581
Merge branch 'main' into main
sayakpaul May 28, 2024
6d1f24d
Merge branch 'huggingface:main' into main
satani99 May 29, 2024
1cc2503
Merge branch 'huggingface:main' into main
satani99 Jun 27, 2024
18fbec0
Merge branch 'huggingface:main' into main
satani99 Aug 2, 2024
26ce4a0
Merge branch 'huggingface:main' into main
satani99 Aug 13, 2024
175f133
Merge branch 'huggingface:main' into main
satani99 Sep 12, 2024
1af0652
auraflow in community
satani99 Sep 20, 2024
ad96f1b
auraflow in community
satani99 Sep 21, 2024
2fc3c66
auraflow in community
satani99 Sep 23, 2024
32eda7f
added auraflow in community pipeline
satani99 Sep 24, 2024
f03f8ee
make style && make quality in diff diff auraflow
satani99 Sep 24, 2024
81b57c1
Merge branch 'huggingface:main' into Auraflow
satani99 Oct 1, 2024
dda4a59
auraflow debugged
satani99 Oct 2, 2024
87f8ccb
update README.md
satani99 Oct 6, 2024
ca1984d
added AuraFlow differential diffusion pipeline to community
satani99 Oct 6, 2024
1a2c948
Merge branch 'main' into Auraflow
satani99 Oct 6, 2024
fbb3fc0
Merge branch 'main' into Auraflow
satani99 Oct 7, 2024
3bc76bc
resolved auraflow differential pipeline
satani99 Oct 23, 2024
81cadde
Merge branch 'main' into Auraflow
satani99 Oct 23, 2024
c4975e5
Merge branch 'main' into Auraflow
a-r-r-o-w Dec 11, 2024
f5d9a34
Merge branch 'main' into Auraflow
satani99 Dec 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions examples/community/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ Please also check out our [Community Scripts](https://github.com/huggingface/dif
| FRESCO V2V Pipeline | Implementation of [[CVPR 2024] FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation](https://arxiv.org/abs/2403.12962) | [FRESCO V2V Pipeline](#fresco) | - | [Yifan Zhou](https://github.com/SingleZombie) |
| AnimateDiff IPEX Pipeline | Accelerate AnimateDiff inference pipeline with BF16/FP32 precision on Intel Xeon CPUs with [IPEX](https://github.com/intel/intel-extension-for-pytorch) | [AnimateDiff on IPEX](#animatediff-on-ipex) | - | [Dan Li](https://github.com/ustcuna/) |
| HunyuanDiT Differential Diffusion Pipeline | Applies [Differential Diffsuion](https://github.com/exx8/differential-diffusion) to [HunyuanDiT](https://github.com/huggingface/diffusers/pull/8240). | [HunyuanDiT with Differential Diffusion](#hunyuandit-with-differential-diffusion) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1v44a5fpzyr4Ffr4v2XBQ7BajzG874N4P?usp=sharing) | [Monjoy Choudhury](https://github.com/MnCSSJ4x) |
| AuraFlow Differential Diffusion Pipeline | Applies [Differential Diffusion](https://github.com/exx8/differential-diffusion) to [AuraFlow](https://github.com/huggingface/diffusers/pull/8796). | [AuraFlow with Differential Diffusion](#auraflow-with-differential-diffusion) | - | [Nikhil Satani](https://github.com/satani99) |

To load a custom pipeline you just need to pass the `custom_pipeline` argument to `DiffusionPipeline`, as one of the files in `diffusers/examples/community`. Feel free to send a PR with your own pipelines, we will merge them quickly.

Expand Down Expand Up @@ -4324,6 +4325,36 @@ image = pipe(

A colab notebook demonstrating all results can be found [here](https://colab.research.google.com/drive/1v44a5fpzyr4Ffr4v2XBQ7BajzG874N4P?usp=sharing). Depth Maps have also been added in the same colab.

### AuraFlow with Differential Diffusion
#### Usage

```python
import torch
from diffusers.utils import load_image
from pipeline_aura_flow_differential_img2img import AuraFlowDifferentialImg2ImgPipeline
pipe = AuraFlowDifferentialImg2ImgPipeline.from_pretrained(
"fal/AuraFlow", torch_dtype=torch.float32
)
pipe.enable_sequential_cpu_offload()
source_image = load_image(
"https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/differential/20240329211129_4024911930.png"
)
map = load_image(
"https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/differential/gradient_mask_2.png"
)
prompt = "a green pear"
negative_prompt = "blurry"
image = pipe(
prompt=prompt,
negative_prompt=negative_prompt,
image=source_image,
num_inference_steps=28,
guidance_scale=4.5,
strength=1.0,
map=map,
).images[0]
```

# Perturbed-Attention Guidance

[Project](https://ku-cvlab.github.io/Perturbed-Attention-Guidance/) / [arXiv](https://arxiv.org/abs/2403.17377) / [GitHub](https://github.com/KU-CVLAB/Perturbed-Attention-Guidance)
Expand Down
Loading