Help！Why is the resulting mesh of such poor quality

[520jz]

I use my own dataset, the dataset is in llff format, and below is my dataset format

Below is an RGB image and mask display of my dataset

This is my config file

This is a bad result

Is there a problem with my dataset or with my config file settings?

[jmunkberg]

Hello,
Please see the discussion here: #84

• Use a sufficient number of views. I would recommend 50+ views, nicely distributed around the object.
• Verify the accuracy of the camera poses. I assume you used colmap? We sometimes check the poses by creating a NeRF using https://github.com/NVlabs/instant-ngp as a sanity check. We assume perfect poses, and do not correct for inaccuracies, so if the poses are bad, nvdiffrec will fail.
• Accuracy of masks. Even a few pixel differences, as shown above, may make the optimization harder. If you have the possibility to create masks with higher accuracy, that will help.
• Is the lighting constant in all images, and is the object fully static across all views?

[520jz]

Hello, Please see the discussion here: #84

• Use a sufficient number of views. I would recommend 50+ views, nicely distributed around the object.
• Verify the accuracy of the camera poses. I assume you used colmap? We sometimes check the poses by creating a NeRF using https://github.com/NVlabs/instant-ngp as a sanity check. We assume perfect poses, and do not correct for inaccuracies, so if the poses are bad, nvdiffrec will fail.
• Accuracy of masks. Even a few pixel differences, as shown above, may make the optimization harder. If you have the possibility to create masks with higher accuracy, that will help.
• Is the lighting constant in all images, and is the object fully static across all views?

Hi, thank you for reply! I appreciate your help. I'm going to take a look at #84
1、My number of views is 66. I feel like my number of views should be adequate.
2、This dataset was downloaded from the Internet, and the dataset itself has camera poses. But the camera pose should also be colmap generated. I thought I should Verify the accuracy of the camera poses.
3、Masks may be less accurate. Original image with white background, I used the following code to generate the mask.
cv2.threshold(gray, 250, 255, cv2. THRESH_BINARY_INV)
4、yes

Some other problems：
Dear author, forgive me for my ignorance.
1、Does the resolution of the input RGB image have any effect on the generated mesh? My original image is relatively large in resolution, so I lowered the resolution of the picture.
2、Do the input RGB images need an alpha channel? That is, must the background of the image be transparent?

[jmunkberg]

Resolution:
Higher is better for reconstruction quality (better texture detail), but requires more memory. For the paper, we usually trained in 800x800 or 1024x1024 resolution with a batch size of 4-8 using GPUs with 32-48GB of memory

Alpha:
If you are using the llff reader, the color images are expected to be three-channel RGB without any alpha channel or transparency. See the code here: https://github.com/NVlabs/nvdiffrec/blob/main/dataset/dataset_llff.py#L83
You can easily modify that to your needs.