Lift3D: Synthesize 3D Training Data by Lifting 2D GAN to 3D Generative Radiance Field

[Paper] [Project Page]

Lift3D: Synthesize 3D Training Data by Lifting 2D GAN to 3D Generative Radiance Field
Leheng Li, Qing Lian, Luozhou Wang, Ningning Ma, Ying-Cong Chen
Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2023

Installation

1. Clone repository

git clone https://github.com/Len-Li/Lift3D.git
cd Lift3D

2. Set up conda environment or use your existing one

conda create --name Lift3D python=3.8
conda activate Lift3D

3. Install the key requirement

pip install torch torchvision torchaudio
pip install configargparse munch pillow

3. Download the checkpoint and object latents

• Onedrive link

Please download lift3d_ckp.pt and obj_latent.pth, then put them in the folder ckp

Inference

python infer.py

The rendered images and semantic mask are saved in the folder test_out.

Acknowledgment

Additionally, we express our gratitude to the authors of the following opensource projects:

• EG3D (tri-plane inplementation)
• StyleSDF (NeRF training framework)

BibTeX

@InProceedings{lift3D2023CVPR, 
	author = {Leheng Li and Qing Lian and Luozhou Wang and Ningning Ma and Ying-Cong Chen}, 
	title = {Lift3D: Synthesize 3D Training Data by Lifting 2D GAN to 3D Generative Radiance Field}, 
	booktitle = {Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)}, 
	year = {2023}, 
}