Ability to change the strength of safety_checker

[suzukimain]

What does this PR do?

Fixes #9003

"""
About safety_Level.
`int` or `float` or one of the following
'WEAK',
'MEDIUM',
'NOMAL',
'STRONG',
'MAX'.
"""

#To see the filter strength.
pipe.filter_level() # 0.0 (default)

#--------------
#If you want to change the intensity.
pipe.safety_checker_level("STRONG")
pipe.filter_level() # 1.0

#--------------
# If numbers are used
pipe.safety_checker_level(3.0)
pipe.filter_level() # 3.0

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[suzukimain]

However, I could not solve the following two problems by myself.

1. regarding the warning message when safety_checker is weakened, the message when safety_checker=None is passed is used as is.
(I was going to put in a modified message, but I was worried, so I decided it was better not to change it and copied it as is.) Resolved in #9404.

---

2. for the number to replace when a string such as 'NOMAL' is entered, I have set the following, but I am not sure if this is the best value. Fixed.

"WEAK": -1.0
"MEDIUM": -0.5
"NOMAL": 0.0
"STRONG": 0.5
"MAX": 1.0

Please let me know if there are other problems.
thank you

[suzukimain]

Hello, @yiyixuxu
Is there any problem?
I hope to not have caused any inconvenience. thank you for your cooperation.

[yiyixuxu]

Hi @suzukimain
what would be a use case to adjust the strength of safety_checker?

[suzukimain]

Hi, @yiyixuxu

It is intended to be used in classes such as StableDiffusionPipelineSafe and StableDiffusionImg2ImgPipeline, which inherit from DiffusionPipeline and take safety_checker as an argument.
Also, the module name may need to be changed to something more descriptive,
although I couldn't come up with a good idea myself.

pip install git+https://github.com/suzukimain/diffusers@safety_checker

from diffusers import StableDiffusionPipeline
import torch

pipe = StableDiffusionPipeline.from_pretrained(
    "stable-diffusion-v1-5/stable-diffusion-v1-5",
    torch_dtype=torch.float16
    ).to("cuda")

#To see the filter strength.
print(f"Default filter strength: {pipe.filter_level()}") # 0.0

pipe.safety_checker_level("STRONG")

print(f"Filter strength: {pipe.filter_level()}") # 0.5

img=pipe("An image of a squirrel in Picasso style").images[0]
img

[yiyixuxu]

thanks I'm trying to understand whether it would be a common/meaningful use case that people need this feature -
could you explain a little bit?

[suzukimain]

thanks I'm trying to understand whether it would be a common/meaningful use case that people need this feature - could you explain a little bit?

Currently, the safety checker only allows for enabling or disabling its functionality.
This means that when the filtering strength feels too weak, there's no way to increase it, and conversely, when the filtering strength feels too strong, the only option is to disable it.
This can be inconvenient, so it would be beneficial to provide users with the ability to adjust the filtering strength flexibly.
Additionally, it might be helpful to refer to #5623

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[asomoza]

IMO to have this option is not bad but I really don't understand why we added the safety checker as part of diffusers, this is better handled outside of diffusers (on the app or library) so they can use whatever they want to filter the outputs, probably this is what all the services are doing right now.

[hlky]

Agree with @asomoza not sure why the safety checker was added, I don't believe it's actually part of the license/usage restrictions and the safety checker is so bad nobody has ever really used it which defeats the purpose.
Anyway changes look good other than the comments.

[suzukimain]

hi @hlky, fixed. Thank you.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[suzukimain]

Hello @hlky, Some corrections have been made.

[suzukimain]

I apologize for the inconvenience. I have corrected it again.

[hlky]

@suzukimain could you run some tests to demonstrate the oversensitivity of the safety checker and find what level of adjustment works well? Choose some (safe) prompt(s), find seed(s) that trigger the safety checker, then set filter strength until the image is allowed through.