Experimental hand improvement feature

[davidiwharper]

Thank you Fooocus team for making AI generation comprehensible. It's an amazing tool.

Despite what I gather was the promise of better hands in SDXL, I have found this element of portraiture to be shockingly bad. Just awful. Muggle kryptonite (to mix my metaphors). Given that Fooocus is intended to be for general audiences who aren't likely to work their way through a bunch of LoRAs and arcane prompts, I wonder whether it would be worth doing something about this issue.
 
The wrong LoRA was created to improve SDXL outputs, It's the best SDXL LoRA for hands I have found thus far. The original author pulled the safetensors version about a month ago, and it appears that the remaining pickletensor file is incompatible with Fooocus. Luckily, however, there is a third party safetensors version available via Civitai that I have tested successfully.

I've found that this LoRA can improve hand quality quite nicely - sometimes directly and sometimes through obscurity (artistic blurring or clever positioning) - and doesn't seem to have made anything especially worse. Even when other things go wrong anatomically (e.g. extra arms), the hands are usually passable, with the right number of fingers and little to no immediately obvious defects (long fingers, weird arrangements of fingers, etc.). I think that the chances of an image with decent hands goes from unlikely to maybe 50%.

The downsides that I have found thus far:

• The LoRA struggles mightily when trying to deal with multiple people
• Going above a 1.0 weight seems counterproductive
• Not much luck combining 'wrong' with other LoRAs relating to hands (possibly because they aren't much good)

On the issue of multiple people, I have found that adding the following negative prompt (based on Ramesh & Gangnani, 2023) can be of some help:

bad anatomy, bad hands, three hands, three legs, bad arms, missing arms, missing fingers, extra fingers, ugly fingers, long fingers, horn, amputation, disconnected limbs

These keywords seem to reign in SDXL's worst instincts, and two-person outputs are noticeably less bizarre. However, I seem to get a passable image about 25% of the time - still not bad compared to the starting point.

Based on all of this, I would like to suggest that an Improve hands (experimental) checkbox be added to automate some sort of basic fix for end users frustrated with the weird stuff that vanilla SDXL seems insistent on pushing out:

1. Load the 'wrong' LoRA with a strength of 1, and insert 'wrong' into the negative prompt
2. Insert a bunch of useful keywords automagically, e.g. as above
3. Display a note to the user in red (once clicked) confirming that the feature is experimental, maybe with a link to more information about the need for patience

I have read that more sophisticated technology may become available in the future to address this issue, so hopefully the user-facing portion of the proposed feature can remain in-tact once something better arrives on the scene in the future. In the meantime, I hope that my notes are of some assistance,

[barepixels]

Solving hands is the number one issue with Stable Diffussion. Am crossing fingers and trying it out now. Thank you for alerting us of this LoRA

UPDATE: Just tried on a batch of 50 images and it does seem an improvement. Not perfect but better than without. I will be using this all the time now. Thanks @davidiwharper Please keep us updated when you find a newer and better solution

[lllyasviel]

show examples

[davidiwharper]

With the 'badhands' LoRA 0.5

With 'wrong' LoRA only at 1.0

With 'wrong' LoRA at 1.0 and the various prompts noted

[davidiwharper]

'wrong' @ 1.0 only

'wrong' @ 1.0 plus prompts

[davidiwharper]

All with LoRA & prompts (I've added 'bad bands' to the negative list for consistency)