autocast_devcice fix #571

yesbroc · 2023-10-08T11:46:16Z

yesbroc
Oct 8, 2023

first encountered this bug when trying out image prompts, stopped a batch mid generation to retry and got this error message:

  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\async_worker.py", line 552, in worker
    handler(task)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\async_worker.py", line 270, in handler
    t['c'] = pipeline.clip_encode(texts=t['positive'], pool_top_k=positive_top_k)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\default_pipeline.py", line 177, in clip_encode
    cond, pooled = clip_encode_single(final_clip, text)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\default_pipeline.py", line 154, in clip_encode_single
    result = clip.encode_from_tokens(tokens, return_pooled=True)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd.py", line 120, in encode_from_tokens
    cond, pooled = self.cond_stage_model.encode_token_weights(tokens)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sdxl_clip.py", line 56, in encode_token_weights
    g_out, g_pooled = self.clip_g.encode_token_weights(token_weight_pairs_g)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\patch.py", line 246, in encode_token_weights_patched_with_a1111_method
    out, pooled = self.encode(to_encode)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd1_clip.py", line 179, in encode
    return self(tokens)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\accelerate\hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd1_clip.py", line 150, in forward
    with precision_scope(model_management.get_autocast_device(device), torch.float32):
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\amp\autocast_mode.py", line 201, in __init__
    raise RuntimeError('User specified autocast device_type must be \'cuda\' or \'cpu\'')
RuntimeError: User specified autocast device_type must be 'cuda' or 'cpu'
Total time: 9.18 seconds

easy fix, just add autocast_instance.device_type = 'cuda' # Debug: Set device_type to:

   @functools.wraps(func)
   def decorate_autocast(*args, **kwargs):
       with autocast_instance:
       # Setting device_type for autocast_instance
        autocast_instance.device_type = 'cuda'  # Debug: Set device_type
       print('Debug: device_type set to:', autocast_instance.device_type)
   decorate_autocast.__script_unsupported = '@autocast() decorator is not supported in script mode'  # type: ignore[attr-defined]
   return decorate_autocast```

lllyasviel · 2023-10-08T11:54:07Z

lllyasviel
Oct 8, 2023
Maintainer

I tried 4 windows devices and 3 linux devices and never have problem like this
how to reproduce the problem?

5 replies

yesbroc Oct 8, 2023
Author

image prompt>advanced>pyracanny
generate>stop
generate (2nd)

the "fix" didn't work 😔

yesbroc Oct 8, 2023
Author

To create a public link, set `share=True` in `launch()`.
model_type EPS
adm 2560
Refiner model loaded: C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\models\checkpoints\sd_xl_refiner_1.0_0.9vae.safetensors
model_type EPS
adm 2816
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
missing {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.text_projection'}
Base model loaded: C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\models\checkpoints\sd_xl_base_1.0_0.9vae.safetensors
LoRAs loaded: [('sd_xl_offset_example-lora_1.0.safetensors', 0.5), ('None', 0.5), ('None', 0.5), ('None', 0.5), ('None', 0.5)]
Fooocus Expansion engine loaded for cuda:0, use_fp16 = True.
loading new
App started successful. Use the app with http://127.0.0.1:7860/ or 127.0.0.1:7860
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 7.0
[Fooocus] Image processing ...
[Fooocus] Downloading control models ...
[Fooocus] Loading control models ...
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 20
[Fooocus] Initializing ...
[Fooocus] Loading models ...
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] New suffix: intricate, retrofuturistic, beautiful, cinematic, artstation, william bouguereau, alphonse mucha, greg rutkowski, 4 k
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] New suffix: extremely high detail, photo realistic, cinematic lighting, atmospheric, 8 k, octane render, post - processing, portrait, extremely hyper - detailed, intricate, epic composition, lifelike attributes, cinematic lighting, masterpiece, trending on artstation, very very detailed, stunning
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Fooocus] Image processing ...
Preparation time: 4.88 seconds
loading new
loading in lowvram mode 1724.2115392684937
Moving model to GPU: 133.33 seconds
  0%|                                                                                          | 0/30 [00:00<?, ?it/s]C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torchsde\_brownian\brownian_interval.py:594: UserWarning: Should have tb<=t1 but got tb=14.614643096923828 and t1=14.614643.
  warnings.warn(f"Should have {tb_name}<=t1 but got {tb_name}={tb} and t1={self._end}.")
  7%|█████▍                                                                            | 2/30 [00:14<03:27,  7.42s/it]
User stopped
loading new
Total time: 155.12 seconds
loading in lowvram mode 256.0
loading in lowvram mode 256.0
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 7.0
[Fooocus] Image processing ...
[Fooocus] Downloading control models ...
[Fooocus] Loading control models ...
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 20
[Fooocus] Initializing ...
[Fooocus] Loading models ...
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] New suffix: intricate, highly detailed 8 k, artgerm, bouguereau
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] New suffix: extremely high quality artwork
[Fooocus] Encoding positive #1 ...
Traceback (most recent call last):
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\async_worker.py", line 552, in worker
    handler(task)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\async_worker.py", line 270, in handler
    t['c'] = pipeline.clip_encode(texts=t['positive'], pool_top_k=positive_top_k)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\default_pipeline.py", line 177, in clip_encode
    cond, pooled = clip_encode_single(final_clip, text)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\default_pipeline.py", line 154, in clip_encode_single
    result = clip.encode_from_tokens(tokens, return_pooled=True)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd.py", line 120, in encode_from_tokens
    cond, pooled = self.cond_stage_model.encode_token_weights(tokens)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sdxl_clip.py", line 56, in encode_token_weights
    g_out, g_pooled = self.clip_g.encode_token_weights(token_weight_pairs_g)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\patch.py", line 246, in encode_token_weights_patched_with_a1111_method
    out, pooled = self.encode(to_encode)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd1_clip.py", line 179, in encode
    return self(tokens)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\accelerate\hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd1_clip.py", line 150, in forward
    with precision_scope(model_management.get_autocast_device(device), torch.float32):
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\amp\autocast_mode.py", line 203, in __init__
    raise RuntimeError('User specified autocast device_type must be \'cuda\' or \'cpu\'')
RuntimeError: User specified autocast device_type must be 'cuda' or 'cpu'
Total time: 0.75 seconds

yesbroc Oct 8, 2023
Author

tried adding device_type = 'cuda' to the top of the code, no issues so far
pyracanny😭

lllyasviel Oct 8, 2023
Maintainer

give full log

yesbroc Oct 8, 2023
Author

i've already sent it, but here it is anyways

To create a public link, set `share=True` in `launch()`.
model_type EPS
adm 2560
Refiner model loaded: C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\models\checkpoints\sd_xl_refiner_1.0_0.9vae.safetensors
model_type EPS
adm 2816
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
missing {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.text_projection'}
Base model loaded: C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\models\checkpoints\sd_xl_base_1.0_0.9vae.safetensors
LoRAs loaded: [('sd_xl_offset_example-lora_1.0.safetensors', 0.5), ('None', 0.5), ('None', 0.5), ('None', 0.5), ('None', 0.5)]
Fooocus Expansion engine loaded for cuda:0, use_fp16 = True.
loading new
App started successful. Use the app with http://127.0.0.1:7860/ or 127.0.0.1:7860
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 7.0
[Fooocus] Image processing ...
[Fooocus] Downloading control models ...
[Fooocus] Loading control models ...
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 20
[Fooocus] Initializing ...
[Fooocus] Loading models ...
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] New suffix: intricate, retrofuturistic, beautiful, cinematic, artstation, william bouguereau, alphonse mucha, greg rutkowski, 4 k
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] New suffix: extremely high detail, photo realistic, cinematic lighting, atmospheric, 8 k, octane render, post - processing, portrait, extremely hyper - detailed, intricate, epic composition, lifelike attributes, cinematic lighting, masterpiece, trending on artstation, very very detailed, stunning
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Fooocus] Image processing ...
Preparation time: 4.88 seconds
loading new
loading in lowvram mode 1724.2115392684937
Moving model to GPU: 133.33 seconds
  0%|                                                                                          | 0/30 [00:00<?, ?it/s]C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torchsde\_brownian\brownian_interval.py:594: UserWarning: Should have tb<=t1 but got tb=14.614643096923828 and t1=14.614643.
  warnings.warn(f"Should have {tb_name}<=t1 but got {tb_name}={tb} and t1={self._end}.")
  7%|█████▍                                                                            | 2/30 [00:14<03:27,  7.42s/it]
User stopped
loading new
Total time: 155.12 seconds
loading in lowvram mode 256.0
loading in lowvram mode 256.0
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 7.0
[Fooocus] Image processing ...
[Fooocus] Downloading control models ...
[Fooocus] Loading control models ...
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 20
[Fooocus] Initializing ...
[Fooocus] Loading models ...
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] New suffix: intricate, highly detailed 8 k, artgerm, bouguereau
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] New suffix: extremely high quality artwork
[Fooocus] Encoding positive #1 ...
Traceback (most recent call last):
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\async_worker.py", line 552, in worker
    handler(task)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\async_worker.py", line 270, in handler
    t['c'] = pipeline.clip_encode(texts=t['positive'], pool_top_k=positive_top_k)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\default_pipeline.py", line 177, in clip_encode
    cond, pooled = clip_encode_single(final_clip, text)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\default_pipeline.py", line 154, in clip_encode_single
    result = clip.encode_from_tokens(tokens, return_pooled=True)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd.py", line 120, in encode_from_tokens
    cond, pooled = self.cond_stage_model.encode_token_weights(tokens)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sdxl_clip.py", line 56, in encode_token_weights
    g_out, g_pooled = self.clip_g.encode_token_weights(token_weight_pairs_g)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\patch.py", line 246, in encode_token_weights_patched_with_a1111_method
    out, pooled = self.encode(to_encode)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd1_clip.py", line 179, in encode
    return self(tokens)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\accelerate\hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd1_clip.py", line 150, in forward
    with precision_scope(model_management.get_autocast_device(device), torch.float32):
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\amp\autocast_mode.py", line 203, in __init__
    raise RuntimeError('User specified autocast device_type must be \'cuda\' or \'cpu\'')
RuntimeError: User specified autocast device_type must be 'cuda' or 'cpu'
Total time: 0.75 seconds

yesbroc · 2023-10-09T11:57:47Z

yesbroc
Oct 9, 2023
Author

got a "new" one on the cuda 12.1 version:


C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus>.\python_embeded\python.exe -s Fooocus\entry_with_update.py
Already up-to-date
Update succeeded.
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Fooocus version: 2.1.27
Inference Engine exists and URL is correct.
Inference Engine checkout finished for c16f5744e306fb042363767d771da68929f088d8.
Total VRAM 4096 MB, total RAM 15791 MB
Trying to enable lowvram mode because your GPU seems to have 4GB or less. If you don't want this use: --normalvram
Set vram state to: LOW_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3050 Ti Laptop GPU : native
VAE dtype: torch.bfloat16
Using pytorch cross attention
Running on local URL:  http://127.0.0.1:7861

To create a public link, set `share=True` in `launch()`.
model_type EPS
adm 2560
Refiner model loaded: C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\models\checkpoints\sd_xl_refiner_1.0_0.9vae.safetensors
model_type EPS
adm 2816
making attention of type 'vanilla-pytorch' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-pytorch' with 512 in_channels
missing {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.text_projection'}
Base model loaded: C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\models\checkpoints\sd_xl_base_1.0_0.9vae.safetensors
LoRAs loaded: [('sd_xl_offset_example-lora_1.0.safetensors', 0.5), ('None', 0.5), ('None', 0.5), ('None', 0.5), ('None', 0.5)]
Fooocus Expansion engine loaded for cuda:0, use_fp16 = True.
loading new
loading in lowvram mode 256.0
App started successful. Use the app with http://127.0.0.1:7861/ or 127.0.0.1:7861
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 7.0
[Fooocus] Downloading control models ...
[Fooocus] Loading control models ...
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 20
[Fooocus] Initializing ...
[Fooocus] Loading models ...
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
Fooocus Expansion loaded by itself.
[Prompt Expansion] New suffix: intricate, elegant, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration, art by artgerm and greg rutkowski and alphonse mucha 8 k
[Fooocus] Preparing Fooocus text #2 ...
Fooocus Expansion loaded by itself.
[Prompt Expansion] New suffix: extremely high quality artwork
[Fooocus] Encoding positive #1 ...
loading new
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Fooocus] Image processing ...
Preparation time: 24.49 seconds
loading new
loading in lowvram mode 1724.2115392684937
Moving model to GPU: 145.27 seconds
  0%|                                                                                          | 0/30 [00:05<?, ?it/s]
User stopped
loading new
Total time: 331.01 seconds
loading in lowvram mode 256.0
loading in lowvram mode 256.0
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 7.0
[Fooocus] Downloading control models ...
[Fooocus] Loading control models ...
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 20
[Fooocus] Initializing ...
[Fooocus] Loading models ...
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
Fooocus Expansion loaded by itself.
[Prompt Expansion] New suffix: intricate, elegant, digital painting, anime, detailed face, atmospheric lighting, detailed and intricate environment
[Fooocus] Preparing Fooocus text #2 ...
Fooocus Expansion loaded by itself.
[Prompt Expansion] New suffix: extremely high quality artwork, very detailed, trending on artstation, high quality artwork, very realistic, hyperrealistic, digital art, trending on deviantart, in the style of michael whelan, high definition portait photo, path traced, house background, no extra space, sharp focus, front view
[Fooocus] Encoding positive #1 ...
Traceback (most recent call last):
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\async_worker.py", line 551, in worker
    handler(task)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\async_worker.py", line 269, in handler
    t['c'] = pipeline.clip_encode(texts=t['positive'], pool_top_k=positive_top_k)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\default_pipeline.py", line 177, in clip_encode
    cond, pooled = clip_encode_single(final_clip, text)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\default_pipeline.py", line 154, in clip_encode_single
    result = clip.encode_from_tokens(tokens, return_pooled=True)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd.py", line 120, in encode_from_tokens
    cond, pooled = self.cond_stage_model.encode_token_weights(tokens)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sdxl_clip.py", line 56, in encode_token_weights
    g_out, g_pooled = self.clip_g.encode_token_weights(token_weight_pairs_g)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\patch.py", line 246, in encode_token_weights_patched_with_a1111_method
    out, pooled = self.encode(to_encode)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd1_clip.py", line 179, in encode
    return self(tokens)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\accelerate\hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd1_clip.py", line 150, in forward
    with precision_scope(model_management.get_autocast_device(device), torch.float32):
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\amp\autocast_mode.py", line 268, in __init__
    raise RuntimeError(
RuntimeError: User specified an unsupported autocast device_type 'meta'
Total time: 2.66 seconds

0 replies

yesbroc · 2023-10-10T15:17:58Z

yesbroc
Oct 10, 2023
Author

new one, what's happening 😭


C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus>.\python_embeded\python.exe -s Fooocus\entry_with_update.py
Fast-forward merge
Update succeeded.
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Fooocus version: 2.1.37
Inference Engine exists and URL is correct.
Inference Engine checkout finished for d1a0abd40b86f3f079b0cc71e49f9f4604831457.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Total VRAM 4096 MB, total RAM 15791 MB
Trying to enable lowvram mode because your GPU seems to have 4GB or less. If you don't want this use: --normalvram
Set vram state to: LOW_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3050 Ti Laptop GPU : native
VAE dtype: torch.bfloat16
Using pytorch cross attention
model_type EPS
adm 2560
Refiner model loaded: C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\models\checkpoints\sd_xl_refiner_1.0_0.9vae.safetensors
model_type EPS
adm 2816
making attention of type 'vanilla-pytorch' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-pytorch' with 512 in_channels
missing {'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
Base model loaded: C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\models\checkpoints\sd_xl_base_1.0_0.9vae.safetensors
LoRAs loaded: [('sd_xl_offset_example-lora_1.0.safetensors', 0.5), ('None', 0.5), ('None', 0.5), ('None', 0.5), ('None', 0.5)]
Fooocus Expansion engine loaded for cuda:0, use_fp16 = True.
loading new
App started successful. Use the app with http://127.0.0.1:7860/ or 127.0.0.1:7860
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 7.0
[Fooocus] Downloading control models ...
[Fooocus] Loading control models ...
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 20
[Fooocus] Initializing ...
[Fooocus] Loading models ...
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] New suffix: intricate, mossy, trending on cgsociety, concept art by tooth wu and wlop and beeple and greg rutkowski, octane render, 8 k, unreal engine. dnd digital art
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] New suffix: extremely high quality artwork
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Fooocus] Image processing ...
Preparation time: 8.19 seconds
loading new
loading in lowvram mode 1724.2115392684937
Moving model to GPU: 160.13 seconds
  0%|                                                                                          | 0/30 [00:05<?, ?it/s]
User stopped
loading new
Total time: 217.13 seconds
loading in lowvram mode 256.0
loading in lowvram mode 256.0
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 7.0
[Fooocus] Downloading control models ...
[Fooocus] Loading control models ...
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 20
[Fooocus] Initializing ...
[Fooocus] Loading models ...
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
Fooocus Expansion loaded by itself.
[Prompt Expansion] New suffix: intricate, elegant, highly detailed animal monster, digital painting, artstation, concept art, smooth, sharp focus, illustration, art by artgerm and greg rutkowski and alphonse mucha, 8 k
[Fooocus] Preparing Fooocus text #2 ...
Fooocus Expansion loaded by itself.
[Prompt Expansion] New suffix: extremely high quality artwork
[Fooocus] Encoding positive #1 ...
C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\amp\autocast_mode.py:293: UserWarning: In CPU autocast, but the target dtype is not supported. Disabling autocast.
CPU Autocast only supports dtype of torch.bfloat16 currently.
  warnings.warn(error_message)
Traceback (most recent call last):
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\async_worker.py", line 565, in worker
    handler(task)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\async_worker.py", line 269, in handler
    t['c'] = pipeline.clip_encode(texts=t['positive'], pool_top_k=positive_top_k)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\default_pipeline.py", line 177, in clip_encode
    cond, pooled = clip_encode_single(final_clip, text)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\default_pipeline.py", line 154, in clip_encode_single
    result = clip.encode_from_tokens(tokens, return_pooled=True)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd.py", line 120, in encode_from_tokens
    cond, pooled = self.cond_stage_model.encode_token_weights(tokens)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sdxl_clip.py", line 56, in encode_token_weights
    g_out, g_pooled = self.clip_g.encode_token_weights(token_weight_pairs_g)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\modules\patch.py", line 246, in encode_token_weights_patched_with_a1111_method
    out, pooled = self.encode(to_encode)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd1_clip.py", line 179, in encode
    return self(tokens)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\accelerate\hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\sd1_clip.py", line 150, in forward
    with precision_scope(model_management.get_autocast_device(device), torch.float32):
  File "C:\Users\orijp\OneDrive\Desktop\chatgpts\fooocus\python_embeded\lib\site-packages\torch\amp\autocast_mode.py", line 356, in __enter__
    torch.set_autocast_cpu_dtype(self.fast_dtype)  # type: ignore[arg-type]
RuntimeError: Currently, AutocastCPU only support Bfloat16 as the autocast_cpu_dtype

1 reply

yesbroc Oct 10, 2023
Author

temporary fix, switching the base model seems to set device_type properly and avoid the bug, reproduction:
image input>image prompt>advanced>pyracanny
generate>stop
generate(2nd, causes bug)
switch base model
image prompt>advanced>CPDS or Image prompt or Pyracanny
generate(3rd)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

autocast_devcice fix #571

{{title}}

Replies: 3 comments 6 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

autocast_devcice fix #571

yesbroc Oct 8, 2023

Replies: 3 comments · 6 replies

lllyasviel Oct 8, 2023 Maintainer

yesbroc Oct 8, 2023 Author

yesbroc Oct 8, 2023 Author

yesbroc Oct 8, 2023 Author

lllyasviel Oct 8, 2023 Maintainer

yesbroc Oct 8, 2023 Author

yesbroc Oct 9, 2023 Author

yesbroc Oct 10, 2023 Author

yesbroc Oct 10, 2023 Author

yesbroc
Oct 8, 2023

Replies: 3 comments 6 replies

lllyasviel
Oct 8, 2023
Maintainer

yesbroc Oct 8, 2023
Author

yesbroc Oct 8, 2023
Author

yesbroc Oct 8, 2023
Author

lllyasviel Oct 8, 2023
Maintainer

yesbroc Oct 8, 2023
Author

yesbroc
Oct 9, 2023
Author

yesbroc
Oct 10, 2023
Author

yesbroc Oct 10, 2023
Author