Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat: Benchmarking Workflow] add stuff for a benchmarking workflow #5839

Merged
merged 118 commits into from
Dec 12, 2023
Merged
Changes from 1 commit
Commits
Show all changes
118 commits
Select commit Hold shift + click to select a range
4c185f0
add poc for benchmarking workflow.
sayakpaul Nov 17, 2023
945ab17
import
sayakpaul Nov 17, 2023
b4debda
fix argument
sayakpaul Nov 17, 2023
22966a1
fix: argument
sayakpaul Nov 17, 2023
12424a3
fix: path
sayakpaul Nov 17, 2023
122d5d9
fix
sayakpaul Nov 17, 2023
c20d254
fix
sayakpaul Nov 17, 2023
43544ed
path
sayakpaul Nov 17, 2023
ada65b3
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Nov 27, 2023
f59e315
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Nov 28, 2023
3c05e41
output csv files.
sayakpaul Nov 28, 2023
24b68fd
workflow cleanup
sayakpaul Nov 28, 2023
8e2088e
append token
sayakpaul Nov 28, 2023
01584c7
add utility to push to hf dataset
sayakpaul Nov 28, 2023
853035b
fix: kw arg
sayakpaul Nov 28, 2023
46aaf96
better reporting
sayakpaul Nov 28, 2023
d626eef
fix: headers
sayakpaul Nov 28, 2023
ab12fe6
better formatting of the numbers.
sayakpaul Nov 28, 2023
1bb531e
better type annotation
sayakpaul Nov 28, 2023
2df4aba
fix: formatting
sayakpaul Nov 28, 2023
939fe5c
moentarily disable check
sayakpaul Nov 28, 2023
3a18e29
push results.
sayakpaul Nov 28, 2023
71279b6
remove disable check
sayakpaul Nov 28, 2023
ea1e57e
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Nov 29, 2023
3c8cc38
introduce base classes.
sayakpaul Nov 29, 2023
9683cd7
img2img class
sayakpaul Nov 29, 2023
274b9e1
add inpainting pipeline
sayakpaul Nov 29, 2023
2b5b8ae
intoduce base benchmark class.
sayakpaul Nov 29, 2023
0b54a6a
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Nov 30, 2023
66b159a
add img2img and inpainting
sayakpaul Nov 30, 2023
01addbd
feat: utility to compare changes
sayakpaul Dec 1, 2023
c30cab6
fix
sayakpaul Dec 1, 2023
689b9f7
fix import
sayakpaul Dec 1, 2023
d046a25
add args
sayakpaul Dec 1, 2023
71f6bd9
basepath
sayakpaul Dec 1, 2023
295cf30
better exception handling
sayakpaul Dec 1, 2023
b5e2371
better path handling
sayakpaul Dec 1, 2023
e7aed9e
fix
sayakpaul Dec 1, 2023
8eb8baf
fix
sayakpaul Dec 1, 2023
aac35c3
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 1, 2023
3cb02f8
remove
sayakpaul Dec 1, 2023
60c980c
ifx
sayakpaul Dec 1, 2023
1e7db92
Merge branch 'main' into feat/ci-benchmarking
patrickvonplaten Dec 1, 2023
cd91b62
fix
sayakpaul Dec 1, 2023
38b8708
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 2, 2023
aeefb55
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 4, 2023
1782d5a
add: support for controlnet.
sayakpaul Dec 4, 2023
df5dead
image_url -> url
sayakpaul Dec 4, 2023
c6c545c
move images to huggingface hub
sayakpaul Dec 4, 2023
b358c87
correct urls.
sayakpaul Dec 4, 2023
93b491b
root_ckpt
sayakpaul Dec 4, 2023
131bfce
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 4, 2023
13a86dc
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 4, 2023
748f6dc
flush before benchmarking
sayakpaul Dec 4, 2023
5d5d5fd
don't install accelerate from source
sayakpaul Dec 4, 2023
4651082
add runner
sayakpaul Dec 4, 2023
8e80579
simplify Diffusers Benchmarking step
sayakpaul Dec 4, 2023
d49ad65
change runner
sayakpaul Dec 4, 2023
7c7846b
fix: subprocess call.
sayakpaul Dec 4, 2023
5dbcbf5
filter percentage values
sayakpaul Dec 4, 2023
cb8572a
fix controlnet benchmark
sayakpaul Dec 4, 2023
6dec96c
add t2i adapters.
sayakpaul Dec 4, 2023
86d597f
fix filter columns
sayakpaul Dec 4, 2023
fa7bfe1
fix t2i adapter benchmark
sayakpaul Dec 4, 2023
59df524
fix init.
sayakpaul Dec 4, 2023
3cd0f59
fix
sayakpaul Dec 4, 2023
8583db8
remove safetensors flag
sayakpaul Dec 4, 2023
6b9bf4a
fix args print
sayakpaul Dec 4, 2023
38160f1
fix
sayakpaul Dec 4, 2023
e6116b0
feat: run_command
sayakpaul Dec 4, 2023
d98fbe1
add adapter resolution mapping
sayakpaul Dec 4, 2023
c93278d
benchmark t2i adapter fix.
sayakpaul Dec 4, 2023
924096f
fix adapter input
sayakpaul Dec 4, 2023
628591d
fix
sayakpaul Dec 4, 2023
0f4ae4e
convert to L.
sayakpaul Dec 4, 2023
de739fa
add flush() add appropriate places
sayakpaul Dec 4, 2023
cb9f9c6
better filtering
sayakpaul Dec 4, 2023
d7aee28
okay
sayakpaul Dec 4, 2023
385ffbb
get env for torch
sayakpaul Dec 4, 2023
611ae13
convert to float
sayakpaul Dec 4, 2023
b3a91d8
fix
sayakpaul Dec 4, 2023
e55913e
filter out nans.
sayakpaul Dec 5, 2023
dc3063a
better coment
sayakpaul Dec 5, 2023
63aee79
sdxl
sayakpaul Dec 5, 2023
9a9d5ea
sdxl for other benchmarks.
sayakpaul Dec 5, 2023
3d66747
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 5, 2023
c8f6eef
fix: condition
sayakpaul Dec 5, 2023
4a67437
fix: condition for inpainting
sayakpaul Dec 5, 2023
e94b895
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 5, 2023
eedf218
fix: mapping for resolution
sayakpaul Dec 5, 2023
e300038
fix
sayakpaul Dec 5, 2023
60614f5
include kandinsky and wuerstchen
sayakpaul Dec 5, 2023
b394168
fix: Wuerstchen
sayakpaul Dec 5, 2023
70f3556
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 5, 2023
b7eb3fb
Empty-Commit
sayakpaul Dec 5, 2023
63a61bd
Merge branch 'main' into feat/ci-benchmarking
DN6 Dec 7, 2023
821726d
[Community] AnimateDiff + Controlnet Pipeline (#5928)
a-r-r-o-w Dec 7, 2023
3dc2362
EulerDiscreteScheduler add `rescale_betas_zero_snr` (#6024)
Beinsezii Dec 7, 2023
26a8c00
Revert "[Community] AnimateDiff + Controlnet Pipeline (#5928)"
sayakpaul Dec 7, 2023
8db59d7
Revert "EulerDiscreteScheduler add `rescale_betas_zero_snr` (#6024)"
sayakpaul Dec 7, 2023
f76ba5b
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 7, 2023
4e7fb4d
add SDXL turbo
sayakpaul Dec 7, 2023
e2df761
add lcm lora to the mix as well.
sayakpaul Dec 7, 2023
2588853
fix
sayakpaul Dec 7, 2023
81d56de
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 7, 2023
a7fd2c3
increase steps to 2 when running turbo i2i
sayakpaul Dec 7, 2023
191ebf6
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 8, 2023
b878a29
debug
sayakpaul Dec 8, 2023
1389d0e
debug
sayakpaul Dec 8, 2023
b2d35be
debug
sayakpaul Dec 8, 2023
d78609d
fix for good
sayakpaul Dec 8, 2023
b3897f8
fix and isolate better
sayakpaul Dec 8, 2023
8289baa
fuse lora so that torch compile works with peft
sayakpaul Dec 8, 2023
dd54366
fix: LCMLoRA
sayakpaul Dec 8, 2023
d6966b4
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 8, 2023
51acace
better identification for LCM
sayakpaul Dec 8, 2023
65b97e8
Merge branch 'main' into feat/ci-benchmarking
sayakpaul Dec 9, 2023
80e8311
change to cron job
sayakpaul Dec 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix
sayakpaul committed Dec 1, 2023
commit cd91b622c8b9de5a42ca44b5b3b2b01993c0ee85
4 changes: 2 additions & 2 deletions benchmarks/base_classes.py
Original file line number Diff line number Diff line change
@@ -88,7 +88,7 @@ class ImageToImageBenchmark(TextToImageBenchmark):
image = load_image(url).convert("RGB")

def __init__(self, args):
super.__init__(args)
super().__init__(args)
self.image = self.image.resize(RESOLUTION_MAPPING[args.ckpt])

def run_inference(self, pipe, args):
@@ -106,7 +106,7 @@ class InpaintingBenchmark(ImageToImageBenchmark):
mask = load_image(mask_url).convert("RGB")

def __init__(self, args):
super.__init__(args)
super().__init__(args)
self.image = self.image.resize(RESOLUTION_MAPPING[args.ckpt])
self.mask = self.mask.resize(RESOLUTION_MAPPING[args.ckpt])