Flax pipeline pndm #583
Merged
Flax pipeline pndm #583
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This may be a temporary solution until #567 is addressed.
This deals with the weights, not the model.
Otherwise jitting/parallelization don't work properly as they don't know how to deal with traced objects. I temporarily removed `step_prk`.
PNDMScheduler uses it to reserve space, other schedulers will just ignore it.
|
The documentation is not available anymore as the PR was closed or merged. |
patil-suraj
reviewed
Sep 20, 2022
src/diffusers/pipelines/stable_diffusion/pipeline_flax_stable_diffusion.py
Outdated
Show resolved
Hide resolved
Values returned still not verified for correctness.
pcuenca
requested review from
patil-suraj,
kashif,
mishig25 and
patrickvonplaten
|
This is ready for review, I just synced with I tested a generation with the current Flax pipeline using the |
patil-suraj
approved these changes
Sep 22, 2022
There was a problem hiding this comment.
Very cool @pcuenca , just left tiny nits.
And yes, we can set this as default scheduler, maybe in a follow-up PR.
Thanks a lot for working on this :)
Co-authored-by: Suraj Patil <[email protected]>
Co-authored-by: Suraj Patil <[email protected]>
|
Should we merge this one? |
I think I addressed all the comments, so it should be ok to merge. We still need to verify the numerical outputs of the PRK steps (not used in Stable Diffusion afaik). I'll try to do it today. |
|
Merging then! Think we don't need to verify prk as it's Flax. Let's only dive into it if people ask for it |
prathikr
pushed a commit
to prathikr/diffusers
that referenced
this pull request
Oct 26, 2022
* WIP: flax FlaxDiffusionPipeline & FlaxStableDiffusionPipeline * todo comment * Fix imports * Fix imports * add dummies * Fix empty init * make pipeline work * up * Allow dtype to be overridden on model load. This may be a temporary solution until huggingface#567 is addressed. * Convert params to bfloat16 or fp16 after loading. This deals with the weights, not the model. * Use Flax schedulers (typing, docstring) * PNDM: replace control flow with jax functions. Otherwise jitting/parallelization don't work properly as they don't know how to deal with traced objects. I temporarily removed `step_prk`. * Pass latents shape to scheduler set_timesteps() PNDMScheduler uses it to reserve space, other schedulers will just ignore it. * Wrap model imports inside availability checks. * Optionally return state in from_config. Useful for Flax schedulers. * Do not convert model weights to dtype. * Re-enable PRK steps with functional implementation. Values returned still not verified for correctness. * Remove left over has_state var. * make style * Apply suggestion list -> tuple Co-authored-by: Suraj Patil <[email protected]> * Apply suggestion list -> tuple Co-authored-by: Suraj Patil <[email protected]> * Remove unused comments. * Use zeros instead of empty. Co-authored-by: Mishig Davaadorj <[email protected]> Co-authored-by: Mishig Davaadorj <[email protected]> Co-authored-by: Patrick von Platen <[email protected]> Co-authored-by: Suraj Patil <[email protected]>
yoonseokjin
pushed a commit
to yoonseokjin/diffusers
that referenced
this pull request
Dec 25, 2023
* WIP: flax FlaxDiffusionPipeline & FlaxStableDiffusionPipeline * todo comment * Fix imports * Fix imports * add dummies * Fix empty init * make pipeline work * up * Allow dtype to be overridden on model load. This may be a temporary solution until huggingface#567 is addressed. * Convert params to bfloat16 or fp16 after loading. This deals with the weights, not the model. * Use Flax schedulers (typing, docstring) * PNDM: replace control flow with jax functions. Otherwise jitting/parallelization don't work properly as they don't know how to deal with traced objects. I temporarily removed `step_prk`. * Pass latents shape to scheduler set_timesteps() PNDMScheduler uses it to reserve space, other schedulers will just ignore it. * Wrap model imports inside availability checks. * Optionally return state in from_config. Useful for Flax schedulers. * Do not convert model weights to dtype. * Re-enable PRK steps with functional implementation. Values returned still not verified for correctness. * Remove left over has_state var. * make style * Apply suggestion list -> tuple Co-authored-by: Suraj Patil <[email protected]> * Apply suggestion list -> tuple Co-authored-by: Suraj Patil <[email protected]> * Remove unused comments. * Use zeros instead of empty. Co-authored-by: Mishig Davaadorj <[email protected]> Co-authored-by: Mishig Davaadorj <[email protected]> Co-authored-by: Patrick von Platen <[email protected]> Co-authored-by: Suraj Patil <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
WIP for PNDM scheduler support in Flax pipeline. Includes #577 and #581 (which I need for testing).