Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NotImplementedError: Cannot copy out of meta tensor; no data! Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to() when moving module from meta to a different device. #480

Closed
starinskycc opened this issue Aug 23, 2024 · 1 comment

Comments

@starinskycc
Copy link

一直报这个错误是什么意思,使用的是L20的显卡,另外,我测试xlabai的训练代码也是报这个错误。
override steps. steps for 10 epochs is / 指定エポックまでのステップ数: 700
Traceback (most recent call last):
File "/root/autodl-tmp/lora-scripts/./sd-scripts/flux_train_network.py", line 411, in
trainer.train(args)
File "/root/autodl-tmp/lora-scripts/sd-scripts/train_network.py", line 570, in train
unet = accelerator.prepare(unet)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/accelerate/accelerator.py", line 1311, in prepare
result = tuple(
^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/accelerate/accelerator.py", line 1312, in
self._prepare_one(obj, first_pass=True, device_placement=d) for obj, d in zip(args, device_placement)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/accelerate/accelerator.py", line 1188, in _prepare_one
return self.prepare_model(obj, device_placement=device_placement)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/accelerate/accelerator.py", line 1435, in prepare_model
model = model.to(self.device)
^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1173, in to
return self._apply(convert)
^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/nn/modules/module.py", line 779, in _apply
module._apply(fn)
File "/root/miniconda3/lib/python3.12/site-packages/torch/nn/modules/module.py", line 804, in _apply
param_applied = fn(param)
^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1166, in convert
raise NotImplementedError(
NotImplementedError: Cannot copy out of meta tensor; no data! Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to() when moving module from meta to a different device.
17:18:40-513209 ERROR Training failed / 训练失败

@Akegarasu
Copy link
Owner

#477

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants