Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FluxTrainLoop Overlapped entries: 'latents_128x96.npy' (possible zip bomb) #123

Closed
liangzoey opened this issue Jan 21, 2025 · 0 comments
Closed

Comments

@liangzoey
Copy link

Hello KJ, I have encountered some issues while running the program. As a newcomer, I am not sure how to solve them and I need help.

ComfyUI Error Report

Error Details

  • Node ID: 4
  • Node Type: FluxTrainLoop
  • Exception Type: zipfile.BadZipFile
  • Exception Message: Overlapped entries: 'latents_128x96.npy' (possible zip bomb)

Stack Trace

  File "F:\ComfyUI\execution.py", line 327, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "F:\ComfyUI\execution.py", line 202, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "F:\ComfyUI\execution.py", line 174, in _map_node_over_list
    process_inputs(input_dict, i)

  File "F:\ComfyUI\execution.py", line 163, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "F:\ComfyUI\custom_nodes\ComfyUI-FluxTrainer\nodes.py", line 905, in train
    steps_done = training_loop(
                 ^^^^^^^^^^^^^^

  File "F:\ComfyUI\custom_nodes\ComfyUI-FluxTrainer\train_network.py", line 1172, in training_loop
    for step, batch in enumerate(skipped_dataloader or train_dataloader):

  File "F:\ComfyUI\venv\Lib\site-packages\accelerate\data_loader.py", line 563, in __iter__
    next_batch = next(dataloader_iter)
                 ^^^^^^^^^^^^^^^^^^^^^

  File "F:\ComfyUI\venv\Lib\site-packages\torch\utils\data\dataloader.py", line 701, in __next__
    data = self._next_data()
           ^^^^^^^^^^^^^^^^^

  File "F:\ComfyUI\venv\Lib\site-packages\torch\utils\data\dataloader.py", line 757, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "F:\ComfyUI\venv\Lib\site-packages\torch\utils\data\_utils\fetch.py", line 52, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
            ~~~~~~~~~~~~^^^^^

  File "F:\ComfyUI\venv\Lib\site-packages\torch\utils\data\dataset.py", line 350, in __getitem__
    return self.datasets[dataset_idx][sample_idx]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^

  File "F:\ComfyUI\custom_nodes\ComfyUI-FluxTrainer\library\train_util.py", line 1505, in __getitem__
    self.latents_caching_strategy.load_latents_from_disk(image_info.latents_npz, image_info.bucket_reso)

  File "F:\ComfyUI\custom_nodes\ComfyUI-FluxTrainer\library\strategy_flux.py", line 219, in load_latents_from_disk
    return self._default_load_latents_from_disk(8, npz_path, bucket_reso)  # support multi-resolution
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "F:\ComfyUI\custom_nodes\ComfyUI-FluxTrainer\library\strategy_base.py", line 538, in _default_load_latents_from_disk
    latents = npz["latents" + key_reso_suffix]
              ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "F:\ComfyUI\venv\Lib\site-packages\numpy\lib\npyio.py", line 251, in __getitem__
    bytes = self.zip.open(key)
            ^^^^^^^^^^^^^^^^^^

  File "C:\Users\liangzoey\Miniconda3\Lib\zipfile\__init__.py", line 1652, in open
    raise BadZipFile(f"Overlapped entries: {zinfo.orig_filename!r} (possible zip bomb)")

System Information

  • ComfyUI Version: 0.3.12
  • Arguments: F:\ComfyUI\main.py --auto-launch --preview-method auto --disable-cuda-malloc --fast
  • OS: nt
  • Python Version: 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 15:03:56) [MSC v.1929 64 bit (AMD64)]
  • Embedded Python: false
  • PyTorch Version: 2.5.1+cu124

Devices

  • Name: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
    • Type: cuda
    • VRAM Total: 25756696576
    • VRAM Free: 11095542228
    • Torch VRAM Total: 15804137472
    • Torch VRAM Free: 2893580756

Logs

2025-01-21T16:50:29.451601 -  2025-01-21T16:50:29.452601 -                 2025-01-21T16:50:29.452601 - 
2025-01-21T16:50:29.452601 -                     2025-01-21T16:50:29.452601 -          2025-01-21T16:50:29.452601 - o.latents_npz,                   2025-01-21T16:50:29.452601 -  2025-01-21T16:50:29.452601 -                 2025-01-21T16:50:29.452601 - 
2025-01-21T16:50:29.452601 -                     2025-01-21T16:50:29.452601 -          2025-01-21T16:50:29.452601 - image_info.bucket_reso2025-01-21T16:50:29.452601 - )2025-01-21T16:50:29.452601 -           2025-01-21T16:50:29.452601 -  2025-01-21T16:50:29.452601 -                 2025-01-21T16:50:29.452601 - 
2025-01-21T16:50:29.452601 -                     2025-01-21T16:50:29.453601 -          2025-01-21T16:50:29.453601 -   File                           2025-01-21T16:50:29.453601 -  2025-01-21T16:50:29.453601 -                 2025-01-21T16:50:29.453601 - 
2025-01-21T16:50:29.453601 -                     2025-01-21T16:50:29.453601 -          2025-01-21T16:50:29.453601 - "F:\ComfyUI\custom_nodes\ComfyUI-2025-01-21T16:50:29.453601 -  2025-01-21T16:50:29.453601 -                 2025-01-21T16:50:29.453601 - 
2025-01-21T16:50:29.453601 -                     2025-01-21T16:50:29.454601 -          2025-01-21T16:50:29.454601 - FluxTrainer\library\strategy_flux2025-01-21T16:50:29.454601 -  2025-01-21T16:50:29.454601 -                 2025-01-21T16:50:29.454601 - 
2025-01-21T16:50:29.454601 -                     2025-01-21T16:50:29.454601 -          2025-01-21T16:50:29.454601 - .py"2025-01-21T16:50:29.454601 - , line 2025-01-21T16:50:29.454601 - 2192025-01-21T16:50:29.454601 - , in               2025-01-21T16:50:29.454601 -  2025-01-21T16:50:29.454601 -                 2025-01-21T16:50:29.454601 - 
2025-01-21T16:50:29.454601 -                     2025-01-21T16:50:29.454601 -          2025-01-21T16:50:29.454601 - load_latents_from_disk           2025-01-21T16:50:29.454601 -  2025-01-21T16:50:29.454601 -                 2025-01-21T16:50:29.454601 - 
2025-01-21T16:50:29.454601 -                     2025-01-21T16:50:29.455601 -          2025-01-21T16:50:29.455601 -     return                       2025-01-21T16:50:29.455601 -  2025-01-21T16:50:29.455601 -                 2025-01-21T16:50:29.455601 - 
2025-01-21T16:50:29.455601 -                     2025-01-21T16:50:29.455601 -          2025-01-21T16:50:29.455601 - self._default_load_latents_from_d2025-01-21T16:50:29.455601 -  2025-01-21T16:50:29.455601 -                 2025-01-21T16:50:29.455601 - 
2025-01-21T16:50:29.455601 -                     2025-01-21T16:50:29.455601 -          2025-01-21T16:50:29.455601 - isk2025-01-21T16:50:29.455601 - (2025-01-21T16:50:29.456601 - 82025-01-21T16:50:29.456601 - , npz_path, bucket_reso2025-01-21T16:50:29.456601 - )2025-01-21T16:50:29.456601 -   # 2025-01-21T16:50:29.456601 -  2025-01-21T16:50:29.456601 -                 2025-01-21T16:50:29.456601 - 
2025-01-21T16:50:29.456601 -                     2025-01-21T16:50:29.456601 -          2025-01-21T16:50:29.456601 - support multi-resolution         2025-01-21T16:50:29.456601 -  2025-01-21T16:50:29.456601 -                 2025-01-21T16:50:29.456601 - 
2025-01-21T16:50:29.456601 -                     2025-01-21T16:50:29.456601 -          2025-01-21T16:50:29.457601 -            ^^^^^^^^^^^^^^^^^^^^^^2025-01-21T16:50:29.457601 -  2025-01-21T16:50:29.457601 -                 2025-01-21T16:50:29.457601 - 
2025-01-21T16:50:29.457601 -                     2025-01-21T16:50:29.457601 -          2025-01-21T16:50:29.457601 - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^2025-01-21T16:50:29.457601 -  2025-01-21T16:50:29.457601 -                 2025-01-21T16:50:29.457601 - 
2025-01-21T16:50:29.457601 -                     2025-01-21T16:50:29.457601 -          2025-01-21T16:50:29.457601 - ^^^^^^^                          2025-01-21T16:50:29.457601 -  2025-01-21T16:50:29.457601 -                 2025-01-21T16:50:29.458601 - 
2025-01-21T16:50:29.458601 -                     2025-01-21T16:50:29.458601 -          2025-01-21T16:50:29.458601 -   File                           2025-01-21T16:50:29.458601 -  2025-01-21T16:50:29.458601 -                 2025-01-21T16:50:29.458601 - 
2025-01-21T16:50:29.458601 -                     2025-01-21T16:50:29.458601 -          2025-01-21T16:50:29.458601 - "F:\ComfyUI\custom_nodes\ComfyUI-2025-01-21T16:50:29.458601 -  2025-01-21T16:50:29.458601 -                 2025-01-21T16:50:29.458601 - 
2025-01-21T16:50:29.458601 -                     2025-01-21T16:50:29.458601 -          2025-01-21T16:50:29.458601 - FluxTrainer\library\strategy_base2025-01-21T16:50:29.458601 -  2025-01-21T16:50:29.458601 -                 2025-01-21T16:50:29.459745 - 
2025-01-21T16:50:29.459745 -                     2025-01-21T16:50:29.459745 -          2025-01-21T16:50:29.459745 - .py"2025-01-21T16:50:29.459745 - , line 2025-01-21T16:50:29.459745 - 5382025-01-21T16:50:29.459745 - , in               2025-01-21T16:50:29.459745 -  2025-01-21T16:50:29.459745 -                 2025-01-21T16:50:29.459745 - 
2025-01-21T16:50:29.459745 -                     2025-01-21T16:50:29.459745 -          2025-01-21T16:50:29.459745 - _default_load_latents_from_disk  2025-01-21T16:50:29.459745 -  2025-01-21T16:50:29.459745 -                 2025-01-21T16:50:29.459745 - 
2025-01-21T16:50:29.460745 -                     2025-01-21T16:50:29.460745 -          2025-01-21T16:50:29.460745 -     latents = npz2025-01-21T16:50:29.460745 - [2025-01-21T16:50:29.460745 - "latents"2025-01-21T16:50:29.460745 -  +    2025-01-21T16:50:29.460745 -  2025-01-21T16:50:29.460745 -                 2025-01-21T16:50:29.460745 - 
2025-01-21T16:50:29.460745 -                     2025-01-21T16:50:29.460745 -          2025-01-21T16:50:29.460745 - key_reso_suffix2025-01-21T16:50:29.460745 - ]2025-01-21T16:50:29.460745 -                  2025-01-21T16:50:29.460745 -  2025-01-21T16:50:29.460745 -                 2025-01-21T16:50:29.460745 - 
2025-01-21T16:50:29.461745 -                     2025-01-21T16:50:29.461745 -          2025-01-21T16:50:29.461745 -               ~~~^^^^^^^^^^^^^^^^2025-01-21T16:50:29.461745 -  2025-01-21T16:50:29.461745 -                 2025-01-21T16:50:29.461745 - 
2025-01-21T16:50:29.461745 -                     2025-01-21T16:50:29.461745 -          2025-01-21T16:50:29.461745 - ^^^^^^^^^^^^^                    2025-01-21T16:50:29.461745 -  2025-01-21T16:50:29.461745 -                 2025-01-21T16:50:29.461745 - 
2025-01-21T16:50:29.461745 -                     2025-01-21T16:50:29.461745 -          2025-01-21T16:50:29.461745 -   File                           2025-01-21T16:50:29.461745 -  2025-01-21T16:50:29.462745 -                 2025-01-21T16:50:29.462745 - 
2025-01-21T16:50:29.462745 -                     2025-01-21T16:50:29.462745 -          2025-01-21T16:50:29.462745 - "F:\ComfyUI\venv\Lib\site-package2025-01-21T16:50:29.462745 -  2025-01-21T16:50:29.462745 -                 2025-01-21T16:50:29.462745 - 
2025-01-21T16:50:29.462745 -                     2025-01-21T16:50:29.462745 -          2025-01-21T16:50:29.462745 - s\numpy\lib\npyio.py"2025-01-21T16:50:29.462745 - , line 2025-01-21T16:50:29.462745 - 2512025-01-21T16:50:29.462745 - , 2025-01-21T16:50:29.462745 -  2025-01-21T16:50:29.463745 -                 2025-01-21T16:50:29.463745 - 
2025-01-21T16:50:29.463745 -                     2025-01-21T16:50:29.463745 -          2025-01-21T16:50:29.463745 - in __getitem__                   2025-01-21T16:50:29.463745 -  2025-01-21T16:50:29.463745 -                 2025-01-21T16:50:29.463745 - 
2025-01-21T16:50:29.463745 -                     2025-01-21T16:50:29.463745 -          2025-01-21T16:50:29.463745 -     bytes = 2025-01-21T16:50:29.463745 - self.zip.open2025-01-21T16:50:29.463745 - (2025-01-21T16:50:29.463745 - key2025-01-21T16:50:29.463745 - )2025-01-21T16:50:29.464745 -    2025-01-21T16:50:29.464745 -  2025-01-21T16:50:29.464745 -                 2025-01-21T16:50:29.464745 - 
2025-01-21T16:50:29.464745 -                     2025-01-21T16:50:29.464745 -          2025-01-21T16:50:29.464745 -             ^^^^^^^^^^^^^^^^^^   2025-01-21T16:50:29.464745 -  2025-01-21T16:50:29.464745 -                 2025-01-21T16:50:29.464745 - 
2025-01-21T16:50:29.464745 -                     2025-01-21T16:50:29.464745 -          2025-01-21T16:50:29.464745 -   File                           2025-01-21T16:50:29.464745 -  2025-01-21T16:50:29.464745 -                 2025-01-21T16:50:29.464745 - 
2025-01-21T16:50:29.465745 -                     2025-01-21T16:50:29.465745 -          2025-01-21T16:50:29.465745 - "C:\Users\liangzoey\Miniconda3\Li2025-01-21T16:50:29.465745 -  2025-01-21T16:50:29.465745 -                 2025-01-21T16:50:29.465745 - 
2025-01-21T16:50:29.465745 -                     2025-01-21T16:50:29.465745 -          2025-01-21T16:50:29.465745 - b\zipfile\__init__.py"2025-01-21T16:50:29.465745 - , line     2025-01-21T16:50:29.465745 -  2025-01-21T16:50:29.465745 -                 2025-01-21T16:50:29.465745 - 
2025-01-21T16:50:29.465745 -                     2025-01-21T16:50:29.465745 -          2025-01-21T16:50:29.465745 - 16522025-01-21T16:50:29.466745 - , in open                    2025-01-21T16:50:29.466745 -  2025-01-21T16:50:29.466745 -                 2025-01-21T16:50:29.466745 - 
2025-01-21T16:50:29.466745 -                     2025-01-21T16:50:29.466745 -          2025-01-21T16:50:29.466745 -     raise 2025-01-21T16:50:29.466745 - BadZipFile2025-01-21T16:50:29.466745 - (2025-01-21T16:50:29.466745 - f"Overlapped2025-01-21T16:50:29.466745 -  2025-01-21T16:50:29.467424 -                 2025-01-21T16:50:29.467424 - 
2025-01-21T16:50:29.467424 -                     2025-01-21T16:50:29.467424 -          2025-01-21T16:50:29.467424 - entries: 2025-01-21T16:50:29.467424 - {2025-01-21T16:50:29.467424 - zinfo.orig_filename!r2025-01-21T16:50:29.467424 - }2025-01-21T16:50:29.467424 -  2025-01-21T16:50:29.467990 -  2025-01-21T16:50:29.467990 -                 2025-01-21T16:50:29.467990 - 
2025-01-21T16:50:29.467990 -                     2025-01-21T16:50:29.467990 -          2025-01-21T16:50:29.467990 - (2025-01-21T16:50:29.467990 - possible zip bomb2025-01-21T16:50:29.467990 - )2025-01-21T16:50:29.467990 - "2025-01-21T16:50:29.468545 - )2025-01-21T16:50:29.468545 -             2025-01-21T16:50:29.468545 -  2025-01-21T16:50:29.468545 -                 2025-01-21T16:50:29.468545 - 
2025-01-21T16:50:29.468545 -                     2025-01-21T16:50:29.468545 -          2025-01-21T16:50:29.468545 - zipfile.BadZipFile: Overlapped   2025-01-21T16:50:29.469128 -  2025-01-21T16:50:29.469128 -                 2025-01-21T16:50:29.469128 - 
2025-01-21T16:50:29.469128 -                     2025-01-21T16:50:29.469128 -          2025-01-21T16:50:29.469128 - entries: 2025-01-21T16:50:29.469128 - 'latents_128x96.npy'2025-01-21T16:50:29.469128 -     2025-01-21T16:50:29.469128 -  2025-01-21T16:50:29.469128 -                 2025-01-21T16:50:29.469128 - 
2025-01-21T16:50:29.469128 -                     2025-01-21T16:50:29.469128 -          2025-01-21T16:50:29.469128 - (2025-01-21T16:50:29.469128 - possible zip bomb2025-01-21T16:50:29.469128 - )2025-01-21T16:50:29.469693 -               2025-01-21T16:50:29.469693 -  2025-01-21T16:50:29.469693 -                 2025-01-21T16:50:29.469693 - 
2025-01-21T16:50:29.469693 -                     2025-01-21T16:50:29.469693 -          2025-01-21T16:50:29.469693 -                                  2025-01-21T16:50:29.469693 -  2025-01-21T16:50:29.469693 -                 2025-01-21T16:50:29.469693 - 
2025-01-21T16:50:29.473119 -                    2025-01-21T16:50:29.473636 -  2025-01-21T16:50:29.473636 - INFO    2025-01-21T16:50:29.473636 -  2025-01-21T16:50:29.473636 - Prompt executed in 2025-01-21T16:50:29.473636 - 63.352025-01-21T16:50:29.473636 -  seconds      2025-01-21T16:50:29.473636 -  2025-01-21T16:50:29.474156 - main.py2025-01-21T16:50:29.474156 - :2025-01-21T16:50:29.474156 - 1862025-01-21T16:50:29.474156 - 

Attached Workflow

Please make sure that workflow does not contain any sensitive information such as API keys or passwords.

Workflow too large. Please manually upload the workflow from local file system.

Additional Context

(Please add any additional context or steps to reproduce the error here)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant