Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Extension]: Error when trying to use Adetailer with DPM 2M solver #2638

Closed
1 task done
driqeks opened this issue Dec 20, 2023 · 4 comments
Closed
1 task done

[Extension]: Error when trying to use Adetailer with DPM 2M solver #2638

driqeks opened this issue Dec 20, 2023 · 4 comments
Labels
extension Extension is not working correctly upstream Fix is required in upstream library

Comments

@driqeks
Copy link

driqeks commented Dec 20, 2023 •
edited
Loading

Issue Description

When trying to generate any image containing a face, and having Adetailer with face detection active, there is an error in the log and Adetailer is skipped. This was working before I updated SD.Next to the latest version. Using SDXL.

                         {'ram': {'used': 11.92, 'total': 125.7}, 'gpu':        
                         {'used': 9.14, 'total': 23.63}, 'retries': 0, 'oom': 0}
Progress  5.91it/s █████████████████████████████████ 100% 20/20 00:03 00:00 Base

0: 640x640 1 face, 5.6ms
Speed: 2.2ms preprocess, 5.6ms inference, 21.6ms postprocess per image at shape (1, 3, 640, 640)
Progress  6.22it/s ██████████████████████████▍        80% 16/20 00:02 00:00 Base
01:29:34-914465 ERROR    Processing: args={'prompt_embeds': tensor([[[-3.4355,  
                         -2.3594,  3.8535,  ...,  0.1772,  0.3999, -0.3267],    
                                  [-1.3213, -0.8647,  1.3848,  ...,  0.1418,    
                         -0.4170, -0.1592],                                     
                                  [-0.2786,  0.6079,  0.3472,  ...,  0.7002,    
                         0.3311, -0.7188],                                      
                                  ...,                                          
                                  [-0.7061,  0.0975,  0.0967,  ...,  0.2585,    
                         0.0869,  0.5269],                                      
                                  [-0.7021,  0.0947,  0.0977,  ...,  0.2291,    
                         0.0122,  0.5103],                                      
                                  [-0.7036,  0.1326,  0.1235,  ...,  0.3547,    
                         0.1252,  0.5674]]], device='cuda:0',                   
                         dtype=torch.float16), 'pooled_prompt_embeds':          
                         tensor([[-1.2246,  0.5894,  0.2168,  ...,  0.3433,     
                         0.6899, -0.2537]], device='cuda:0',                    
                         dtype=torch.float16), 'negative_prompt_embeds':        
                         tensor([[[-3.4355, -2.3594,  3.8535,  ...,  0.1772,    
                         0.3999, -0.3267],                                      
                                  [-0.8232, -0.5801,  0.9434,  ...,  0.3511,    
                         -0.5752,  0.6289],                                     
                                  [-0.2571, -0.6147, -0.4995,  ..., -0.3555,    
                         0.0533,  0.0647],                                      
                                  ...,                                          
                                  [-0.2053, -0.2067, -0.5938,  ...,  0.3477,    
                         0.1062,  0.6802],                                      
                                  [-0.2073, -0.2185, -0.5908,  ...,  0.3237,    
                         -0.0105,  0.6709],                                     
                                  [-0.2037, -0.1109, -0.5430,  ...,  0.4497,    
                         0.0469,  0.7676]]], device='cuda:0',                   
                         dtype=torch.float16), 'negative_pooled_prompt_embeds': 
                         tensor([[-0.4348,  0.3589, -1.0137,  ..., -0.8911,     
                         -0.8936,  0.8726]], device='cuda:0',                   
                         dtype=torch.float16), 'guidance_scale': 6, 'generator':
                         [<torch._C.Generator object at 0x7f47dc2c2090>],       
                         'output_type': 'latent', 'callback_on_step_end':       
                         <function process_diffusers.<locals>.diffusers_callback
                         at 0x7f46e6fe3760>,                                    
                         'callback_on_step_end_tensor_inputs': ['latents'],     
                         'num_inference_steps': 50, 'eta': 1.0,                 
                         'guidance_rescale': 0.7, 'denoising_start': None,      
                         'denoising_end': None, 'image': [<PIL.Image.Image image
                         mode=RGB size=1024x1024 at 0x7F47DC284640>],           
                         'mask_image': <PIL.Image.Image image mode=L            
                         size=1024x1024 at 0x7F47DC284370>, 'strength': 0.4,    
                         'height': 1024, 'width': 1024} a Tensor with 2 elements
                         cannot be converted to Scalar                          
01:29:35-982983 ERROR    Processing: RuntimeError                               
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/user/StableDiffusion/SDNext/automatic/modules/processing_diffusers.py: │
│ 516 in process_diffusers                                                     │
│                                                                              │
│   515 │   │   t0 = time.time()                                               │
│ ❱ 516 │   │   output = shared.sd_model(**base_args) # pylint: disable=not-ca │
│   517 │   │   if shared.cmd_opts.profile:                                    │
│                                                                              │
│ /home/user/StableDiffusion/SDNext/automatic/venv/lib/python3.10/site-package │
│ s/torch/utils/_contextlib.py:115 in decorate_context                         │
│                                                                              │
│   114 │   │   with ctx_factory():                                            │
│ ❱ 115 │   │   │   return func(*args, **kwargs)                               │
│   116                                                                        │
│                                                                              │
│ /home/user/StableDiffusion/SDNext/automatic/venv/lib/python3.10/site-package │
│ s/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl_inpai │
│ nt.py:1655 in __call__                                                       │
│                                                                              │
│   1654 │   │   │   │   │   │   noise_timestep = timesteps[i + 1]             │
│ ❱ 1655 │   │   │   │   │   │   init_latents_proper = self.scheduler.add_nois │
│   1656 │   │   │   │   │   │   │   init_latents_proper, noise, torch.tensor( │
│                                                                              │
│ /home/user/StableDiffusion/SDNext/automatic/venv/lib/python3.10/site-package │
│ s/diffusers/schedulers/scheduling_dpmsolver_multistep.py:898 in add_noise    │
│                                                                              │
│   897 │   │                                                                  │
│ ❱ 898 │   │   step_indices = [(schedule_timesteps == t).nonzero().item() for │
│   899                                                                        │
│                                                                              │
│ /home/user/StableDiffusion/SDNext/automatic/venv/lib/python3.10/site-package │
│ s/diffusers/schedulers/scheduling_dpmsolver_multistep.py:898 in <listcomp>   │
│                                                                              │
│   897 │   │                                                                  │
│ ❱ 898 │   │   step_indices = [(schedule_timesteps == t).nonzero().item() for │
│   899                                                                        │
╰──────────────────────────────────────────────────────────────────────────────╯
RuntimeError: a Tensor with 2 elements cannot be converted to Scalar
01:29:36-253679 INFO     Processed: images=0 time=4.88 its=0.00 memory={'ram':  
                         {'used': 17.28, 'total': 125.7}, 'gpu': {'used': 9.75, 
                         'total': 23.63}, 'retries': 0, 'oom': 0}               
01:29:36-386738 ERROR    Running script postprocess image:                      
                         extensions/adetailer/scripts/!adetailer.py: IndexError 
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/user/StableDiffusion/SDNext/automatic/modules/scripts.py:584 in        │
│ postprocess_image                                                            │
│                                                                              │
│   583 │   │   │   │   args = p.per_script_args.get(script.title(), p.script_ │
│ ❱ 584 │   │   │   │   script.postprocess_image(p, pp, *args)                 │
│   585 │   │   │   except Exception as e:                                     │
│                                                                              │
│ /home/user/StableDiffusion/SDNext/automatic/extensions/adetailer/adetailer/t │
│ raceback.py:159 in wrapper                                                   │
│                                                                              │
│   158 │   │   │   │   error = RuntimeError(output)                           │
│ ❱ 159 │   │   │   raise error from None                                      │
│   160                                                                        │
╰──────────────────────────────────────────────────────────────────────────────╯
IndexError: 
╭──────────────────────────────────────────────────────────────────────────╮
│                               System info                                │
│ ┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ │
│ ┃             ┃ Value                                                  ┃ │
│ ┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │
│ │    Platform │ Linux-6.2.0-39-generic-x86_64-with-glibc2.35           │ │
│ │      Python │ 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]     │ │
│ │     Version │ Unknown (too old or vladmandic)                        │ │
│ │      Commit │ Unknown                                                │ │
│ │ Commandline │ ['launch.py']                                          │ │
│ │   Libraries │ {'torch': '2.1.0+cu121', 'torchvision':                │ │
│ │             │ '0.16.0+cu121', 'ultralytics': '8.0.222', 'mediapipe': │ │
│ │             │ '0.10.8'}                                              │ │
│ └─────────────┴────────────────────────────────────────────────────────┘ │
│                   Inputs                                                 │
│ ┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓                               │
│ ┃                 ┃ Value                ┃                               │
│ ┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩                               │
│ │          prompt │ photo of a women     │                               │
│ │ negative_prompt │                      │                               │
│ │          n_iter │ 1                    │                               │
│ │      batch_size │ 1                    │                               │
│ │           width │ 1024                 │                               │
│ │          height │ 1024                 │                               │
│ │    sampler_name │ DPM++ 2M             │                               │
│ │       enable_hr │ False                │                               │
│ │     hr_upscaler │ None                 │                               │
│ │      checkpoint │ albedobaseXL_v13     │                               │
│ │             vae │ sdxl_vae.safetensors │                               │
│ │            unet │ ------               │                               │
│ └─────────────────┴──────────────────────┘                               │
│                 ADetailer                                                │
│ ┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓                                │
│ ┃                     ┃ Value           ┃                                │
│ ┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩                                │
│ │             version │ 23.11.1         │                                │
│ │            ad_model │ face_yolov8n.pt │                                │
│ │           ad_prompt │ detailed face   │                                │
│ │  ad_negative_prompt │                 │                                │
│ │ ad_controlnet_model │ None            │                                │
│ │              is_api │ False           │                                │
│ └─────────────────────┴─────────────────┘                                │
│ ╭───────────────── Traceback (most recent call last) ──────────────────╮ │
│ │ /home/user/StableDiffusion/SDNext/automatic/extensions/adetailer/ade │ │
│ │ tailer/traceback.py:139 in wrapper                                   │ │
│ │                                                                      │ │
│ │   138 │   │   try:                                                   │ │
│ │ ❱ 139 │   │   │   return func(*args, **kwargs)                       │ │
│ │   140 │   │   except Exception as e:                                 │ │
│ │                                                                      │ │
│ │ /home/user/StableDiffusion/SDNext/automatic/extensions/adetailer/scr │ │
│ │ ipts/!adetailer.py:750 in postprocess_image                          │ │
│ │                                                                      │ │
│ │   749 │   │   │   │   │   continue                                   │ │
│ │ ❱ 750 │   │   │   │   is_processed |= self._postprocess_image_inner( │ │
│ │   751                                                                │ │
│ │                                                                      │ │
│ │ /home/user/StableDiffusion/SDNext/automatic/extensions/adetailer/scr │ │
│ │ ipts/!adetailer.py:722 in _postprocess_image_inner                   │ │
│ │                                                                      │ │
│ │   721 │   │   │   p2 = copy(i2i)                                     │ │
│ │ ❱ 722 │   │   │   p2.init_images = [processed.images[0]]             │ │
│ │   723                                                                │ │
│ ╰──────────────────────────────────────────────────────────────────────╯ │
│ IndexError: list index out of range                                      │
╰──────────────────────────────────────────────────────────────────────────╯

01:29:43-295868 INFO     Processed: images=1 time=20.46 its=0.98 memory={'ram': 
                         {'used': 17.28, 'total': 125.7}, 'gpu': {'used': 9.75, 
                         'total': 23.63}, 'retries': 0, 'oom': 0}

Version Platform Description

01:24:18-569034 INFO     Logger:                                                
                         file="/home/user/StableDiffusion/SDNext/automatic/sdnex
                         t.log" level=INFO size=1623929 mode=append             
01:24:18-570131 INFO     Python 3.10.12 on Linux                                
01:24:18-586578 INFO     Version: app=sd.next updated=2023-12-04 hash=93f35ccf  
                         url=https://github.com/vladmandic/automatic/tree/master
01:24:20-045591 INFO     Latest published version:                              
                         00281e15d2f4d50b53fa892bae4e1cfa56e7ecd6               
                         2023-12-19T23:26:51Z                                   
01:24:20-049884 INFO     Platform: arch=x86_64 cpu=x86_64 system=Linux          
                         release=6.2.0-39-generic python=3.10.12                
01:24:20-051377 INFO     nVidia CUDA toolkit detected: nvidia-smi present       
01:24:22-183622 INFO     Extensions: disabled=['sd-webui-controlnet']           
01:24:22-184607 INFO     Extensions: enabled=['sd-extension-chainner', 'Lora',  
                         'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'stable-diffusion-webui-images-browser',               
                         'stable-diffusion-webui-rembg'] extensions-builtin     
01:24:22-185955 INFO     Extensions: enabled=['adetailer', 'sd-dynamic-prompts']
                         extensions                                             
01:24:22-187002 INFO     Startup: standard                                      
01:24:22-187558 INFO     Verifying requirements                                 
01:24:22-200583 INFO     Verifying packages                                     
01:24:22-202038 INFO     Verifying submodules                                   
01:24:30-112412 INFO     Extensions enabled: ['sd-extension-chainner', 'Lora',  
                         'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'stable-diffusion-webui-images-browser',               
                         'stable-diffusion-webui-rembg', 'adetailer',           
                         'sd-dynamic-prompts']                                  
01:24:30-113532 INFO     Verifying requirements                                 
01:24:30-131882 INFO     Extension preload: {'extensions-builtin': 0.0,         
                         'extensions': 0.0}                                     
01:24:30-133120 INFO     Command line args: []                                  
01:24:32-896209 INFO     Load packages: torch=2.1.0+cu121 diffusers=0.24.0      
                         gradio=3.43.2                                          
01:24:34-544936 INFO     Engine: backend=Backend.DIFFUSERS compute=cuda         
                         mode=no_grad device=cuda cross-optimization="xFormers" 
01:24:34-615294 INFO     Device: device=NVIDIA GeForce RTX 4090 n=1 arch=sm_90  
                         cap=(8, 9) cuda=12.1 cudnn=8902 driver=535.129.03

URL link of the extension

https://github.com/Bing-su/adetailer

URL link of the issue reported in the extension repository

No response

Acknowledgements

  • I have read the above and searched for existing issues
@driqeks driqeks added the extension Extension is not working correctly label Dec 20, 2023
@vladmandic
Copy link
Owner

just to confirm, try using any other scheduler.
core issue seems to be a bug in diffusers implementation of that specific scheduler with exact settings
huggingface/diffusers#6232

@vladmandic vladmandic added the upstream Fix is required in upstream library label Dec 20, 2023
@eadnams22
Copy link
Contributor

Can confirm this, and the workaround. I use a different sampler/scheduler for adetailer inpainting because of this.

@driqeks
Copy link
Author

driqeks commented Dec 22, 2023

It does work when using the UniPC sampler

@vladmandic vladmandic changed the title [Extension]: Error when trying to use Adetailer [Extension]: Error when trying to use Adetailer with DPM 2M solver Feb 15, 2024
@vladmandic
Copy link
Owner

fixed upstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
extension Extension is not working correctly upstream Fix is required in upstream library
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@vladmandic @eadnams22 @driqeks