Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

output doesnt have any changes applied. is ther any tutorial or documentation? #6

Open
runner22k opened this issue Jun 10, 2023 · 3 comments

Comments

@runner22k
Copy link

runner22k commented Jun 10, 2023

I was able to get the extension up and running after some tries, as it needed a package to be compiled and user should have Windows SDK too.

Now I am trying to test it. but I don't see any effect on the video. even art style specified in the prompt is not applied. Dear developer, is there any documentation or tutorial available? How to use controlnet with this extension?

@dchatel
Copy link
Owner

dchatel commented Jun 10, 2023

The extension is pretty simple.
It extracts the faces of the video and align them (so it even works great with upside down faces). It put the extracted faces in batches of batch_size to be processed at once, in a similar way to img2img. You can use controlnet, simply by activating as you normally would, but if you do, you need to set batch_size to 1. Controlnet changed something since I've created this extension, and now controlnet doesn't handle batches anymore.
If you have issues making it work, tell me, I'll probably make a quick tutorial video on youtube.

@runner22k
Copy link
Author

runner22k commented Jun 10, 2023

The extension is pretty simple. It extracts the faces of the video and align them (so it even works great with upside down faces). It put the extracted faces in batches of batch_size to be processed at once, in a similar way to img2img. You can use controlnet, simply by activating as you normally would, but if you do, you need to set batch_size to 1. Controlnet changed something since I've created this extension, and now controlnet doesn't handle batches anymore. If you have issues making it work, tell me, I'll probably make a quick tutorial video on youtube.

Oh! thanks for the reply. I will change batch size and check. Lets hope. A tutorial is always welcome. :) Is lora supported too?

And one more question, is it possible for you to add an option to swap the face and then apply user selected art style? May be we need to pass the image back to controlnet and generate. which might double the time. But, Your extension's feature to regenerate frames will still keep the entire process at roughly around 50% than sd-cn animation. Only thing missing is applying art style to the entire video.

is it possible to save setting to a txt file, so that it will be good to reuse same setting again if need.

@runner22k
Copy link
Author

runner22k commented Jun 10, 2023

something is going wrong, I can see a randomly bouncing rounded square when I apply controlnet. please upload a tutorial if possible. Thanks in advance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants