Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KeyError: 'zero123-unified-guidance-cache' #3

Open
altava-sgp opened this issue Apr 18, 2024 · 17 comments
Open

KeyError: 'zero123-unified-guidance-cache' #3

altava-sgp opened this issue Apr 18, 2024 · 17 comments

Comments

@altava-sgp
Copy link

I run this command.

python launch.py --config configs/stable-zero123_hash3d.yaml --train --gpu 0 data.image_path=load/images/dog1_rgba.png

I got this error.

$ python launch.py --config configs/stable-zero123_hash3d.yaml --train --gpu 0 data.image_path=load/images/dog1_rgba.png
Global seed set to 0
find:  single-image-datamodule
find:  zero123-system
find:  implicit-volume
find:  diffuse-with-point-light-material
find:  solid-color-background
find:  nerf-volume-renderer
[INFO] ModelCheckpoint(save_last=True, save_top_k=-1, monitor=None) will duplicate the last checkpoint saved.
[INFO] GPU available: True (cuda), used: True
[INFO] TPU available: False, using: 0 TPU cores
[INFO] IPU available: False, using: 0 IPUs
[INFO] HPU available: False, using: 0 HPUs
[INFO] You are using a CUDA device ('NVIDIA GeForce RTX 4090') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
[INFO] single image dataset: load image load/images/dog1_rgba.png torch.Size([1, 128, 128, 3])
[INFO] single image dataset: load image load/images/dog1_rgba.png torch.Size([1, 128, 128, 3])
[INFO] LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
[INFO] 
  | Name       | Type                          | Params
-------------------------------------------------------------
0 | geometry   | ImplicitVolume                | 12.6 M
1 | material   | DiffuseWithPointLightMaterial | 0     
2 | background | SolidColorBackground          | 0     
3 | renderer   | NeRFVolumeRenderer            | 0     
-------------------------------------------------------------
12.6 M    Trainable params
0         Non-trainable params
12.6 M    Total params
50.450    Total estimated model params size (MB)
[INFO] Validation results will be saved to outputs/zero123-sai-hash3d/[64, 128, 256]_dog1_rgba.png@20240418-085259/save
find:  zero123-unified-guidance-cache
Traceback (most recent call last):
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/launch.py", line 301, in <module>
    main(args, extras)
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/launch.py", line 244, in main
    trainer.fit(system, datamodule=dm, ckpt_path=cfg.resume)
  File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 532, in fit
    call._call_and_handle_interrupt(
  File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 43, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 571, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 961, in _run
    call._call_lightning_module_hook(self, "on_fit_start")
  File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 146, in _call_lightning_module_hook
    output = fn(*args, **kwargs)
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/threestudio/systems/zero123.py", line 40, in on_fit_start
    self.guidance = threestudio.find(self.cfg.guidance_type)(self.cfg.guidance)
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/threestudio/__init__.py", line 33, in find
    return __modules__[name]
KeyError: 'zero123-unified-guidance-cache'

What's the problem ?
@Adamdad
Copy link
Owner

Adamdad commented Apr 18, 2024

Hello, can you check how to you set the path to your threestudio package. I am sure that the class of zero123-unified-guidance-cache is inside the code but it seems that you cannot access this class. Can you please check?

hash3D/threestudio-hash3d/threestudio/models/guidance/zero123_unified_guidance_cache.py

Line 34 in d705863

@threestudio.register("zero123-unified-guidance-cache")

@altava-sgp
Copy link
Author

@Adamdad The structure of folders and files is here.
image

@Adamdad
Copy link
Owner

Adamdad commented Apr 18, 2024

Can you run the code again, and set export PYTHONPATH="$PWD" before running your script?

@altava-sgp
Copy link
Author

It pauses for a little time after this...

$ pwd
/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d
$ 
$ export PYTHONPATH="$PWD"
$ 
$ python launch.py --config configs/stable-zero123_hash3d.yaml --train --gpu 0 data.image_path=load/images/dog1_rgba.png
Global seed set to 0
find:  single-image-datamodule
find:  zero123-system
find:  implicit-volume
find:  diffuse-with-point-light-material
find:  solid-color-background
find:  nerf-volume-renderer
[INFO] ModelCheckpoint(save_last=True, save_top_k=-1, monitor=None) will duplicate the last checkpoint saved.
[INFO] GPU available: True (cuda), used: True
[INFO] TPU available: False, using: 0 TPU cores
[INFO] IPU available: False, using: 0 IPUs
[INFO] HPU available: False, using: 0 HPUs
[INFO] You are using a CUDA device ('NVIDIA GeForce RTX 4090') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
[INFO] single image dataset: load image load/images/dog1_rgba.png torch.Size([1, 128, 128, 3])
[INFO] single image dataset: load image load/images/dog1_rgba.png torch.Size([1, 128, 128, 3])
[INFO] LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

I am waiting...

@altava-sgp
Copy link
Author

I run command in docker terminal.
image

@altava-sgp
Copy link
Author

Eventually I got this error again.
image

$ pwd
/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d
$ 
$ export PYTHONPATH="$PWD"
$ 
$ python launch.py --config configs/stable-zero123_hash3d.yaml --train --gpu 0 data.image_path=load/images/dog1_rgba.png
Global seed set to 0
find:  single-image-datamodule
find:  zero123-system
find:  implicit-volume
find:  diffuse-with-point-light-material
find:  solid-color-background
find:  nerf-volume-renderer
[INFO] ModelCheckpoint(save_last=True, save_top_k=-1, monitor=None) will duplicate the last checkpoint saved.
[INFO] GPU available: True (cuda), used: True
[INFO] TPU available: False, using: 0 TPU cores
[INFO] IPU available: False, using: 0 IPUs
[INFO] HPU available: False, using: 0 HPUs
[INFO] You are using a CUDA device ('NVIDIA GeForce RTX 4090') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
[INFO] single image dataset: load image load/images/dog1_rgba.png torch.Size([1, 128, 128, 3])
[INFO] single image dataset: load image load/images/dog1_rgba.png torch.Size([1, 128, 128, 3])
[INFO] LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
[INFO] 
  | Name       | Type                          | Params
-------------------------------------------------------------
0 | geometry   | ImplicitVolume                | 12.6 M
1 | material   | DiffuseWithPointLightMaterial | 0     
2 | background | SolidColorBackground          | 0     
3 | renderer   | NeRFVolumeRenderer            | 0     
-------------------------------------------------------------
12.6 M    Trainable params
0         Non-trainable params
12.6 M    Total params
50.450    Total estimated model params size (MB)
[INFO] Validation results will be saved to outputs/zero123-sai-hash3d/[64, 128, 256]_dog1_rgba.png@20240418-092826/save
find:  zero123-unified-guidance-cache
Traceback (most recent call last):
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/launch.py", line 301, in <module>
    main(args, extras)
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/launch.py", line 244, in main
    trainer.fit(system, datamodule=dm, ckpt_path=cfg.resume)
  File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 532, in fit
    call._call_and_handle_interrupt(
  File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 43, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 571, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 961, in _run
    call._call_lightning_module_hook(self, "on_fit_start")
  File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 146, in _call_lightning_module_hook
    output = fn(*args, **kwargs)
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/threestudio/systems/zero123.py", line 40, in on_fit_start
    self.guidance = threestudio.find(self.cfg.guidance_type)(self.cfg.guidance)
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/threestudio/__init__.py", line 33, in find
    return __modules__[name]
KeyError: 'zero123-unified-guidance-cache'

@Adamdad
Copy link
Owner

Adamdad commented Apr 18, 2024

can git pull again? I just push one line of code

hash3D/threestudio-hash3d/threestudio/models/guidance/__init__.py

Lines 1 to 12 in ea45d58

from . import (
controlnet_guidance,
deep_floyd_guidance,
instructpix2pix_guidance,
stable_diffusion_guidance,
stable_diffusion_unified_guidance,
stable_diffusion_vsd_guidance,
stable_zero123_guidance,
zero123_guidance,
zero123_unified_guidance,
zero123_unified_guidance_cache,
stable_diffusion_guidance_cache

@altava-sgp
Copy link
Author

I got this error.

$ pwd
/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d
$ 
$ python launch.py --config configs/stable-zero123_hash3d.yaml --train --gpu 0 data.image_path=load/images/dog1_rgba.png
Traceback (most recent call last):
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/launch.py", line 301, in <module>
    main(args, extras)
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/launch.py", line 135, in main
    import threestudio
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/threestudio/__init__.py", line 55, in <module>
    from . import data, models, systems
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/threestudio/models/__init__.py", line 1, in <module>
    from . import (
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/threestudio/models/guidance/__init__.py", line 1, in <module>
    from . import (
  File "/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/threestudio/models/guidance/zero123_unified_guidance_cache.py", line 31, in <module>
    from threestudio.utils.hash_table import GridBasedHashTable, GridBasedHashTable_Sim, AdaptiveGridBasedHashTable, GridBasedHashTable_Key
ImportError: cannot import name 'GridBasedHashTable_Key' from 'threestudio.utils.hash_table' (/home/dreamer/threestudio/custom/hash3D/threestudio-hash3d/threestudio/utils/hash_table.py)

@Adamdad
Copy link
Owner

Adamdad commented Apr 18, 2024

Sorry, just remove this. Can pull again. GridBasedHashTable_Key is an ablated version of my full method.

@altava-sgp
Copy link
Author

It is working!
image

Let's see what will happen

@altava-sgp
Copy link
Author

It continues...
image

@altava-sgp
Copy link
Author

I got it !

image
image

it1000-test.mp4

@altava-sgp
Copy link
Author

altava-sgp commented Apr 18, 2024 •
edited
Loading

So can I use this ckpt file to proceed next steps ?

image

@Adamdad
Copy link
Owner

Adamdad commented Apr 18, 2024

Yes, threestudio can help you to convert this ckpt to mesh .obj and can be applied to other applications.

@altava-sgp
Copy link
Author

@Adamdad Are you in threestudio discord server ?

@Adamdad
Copy link
Owner

Adamdad commented Apr 18, 2024

Yes I am in, but I never send any message there.

@Adamdad
Copy link
Owner

Adamdad commented Apr 18, 2024

Can you give a star on the project? Thank you so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Adamdad @altava-sgp