Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about evaluation on NeRF Synthetic #60

Open
francescodisario opened this issue Jun 11, 2024 · 1 comment
Open

Question about evaluation on NeRF Synthetic #60

francescodisario opened this issue Jun 11, 2024 · 1 comment

Comments

@francescodisario
Copy link

Hi and first of all thanks for your amazing work!
I have a question regarding your evaluation method for NeRF Synthetic. In the paper you reported 33.68 db PSNR on NeRF Synthethic and you compared it with the vanilla GS which is around 33.20.
To achieve such good results, did you set the background color to white? I believe that from your code you keept the background black, while in vanilla-gs (and il all previous works), the bg color i white.

In fact, if I try to execute your code with bg white, I can't reproduce your result (I am around 0.6db less), while vanilla-gs with bg color black is around 34db PSNR.

I don't think it's a big problem, but I just wanted to be sure that everything is working correctly and point out that the bg color may really affect accuracies results. So it would have been better to compare them with the same bg color.

@inspirelt
Copy link
Collaborator

Hi, we evaluate nerfsynthetic with the default black background. And I believe the gap in results may caused by some minor updates in recent months, like the density control strategies. You can adopt your current reproduced results in your research as long as clarifying experiment settings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants