3D Gaussian Splatting (3DGS) has shown its impressive power in novel view synthesis. However, creating relightable 3D assets, especially for objects with ill-defined shapes (e.g., fur), is still a challenging task. For these scenes, the decomposition between the light, geometry, and material is more ambiguous, as neither the surface constraints nor the analytical shading model hold. To address this issue, we propose RNG, a novel representation of relightable neural Gaussians, enabling the relighting of objects with both hard surfaces or fluffy boundaries. We avoid any assumptions in the shading model but maintain feature vectors, which can be further decoded by an MLP into colors, in each Gaussian point. Following prior work, we utilize a point light to reduce the ambiguity and introduce a shadow-aware condition to the network. We additionally propose a depth refinement network to help the shadow computation under the 3DGS framework, leading to better shadow effects under point lights. Furthermore, to avoid the blurriness brought by the alpha-blending in 3DGS, we design a hybrid forward-deferred optimization strategy. As a result, we achieve about 20× faster in training and about 600× faster in rendering than prior work based on neural radiance fields, with 60 frames per second on an RTX4090.
3D 高斯分布（3D Gaussian Splatting, 3DGS）在新视角合成中展示了其强大的能力。然而，创建可重光照的三维资产，尤其是针对形状不明确的物体（如毛发），仍然是一项具有挑战性的任务。在这些场景中，光照、几何和材质之间的分解更加模糊，因为无论是表面约束还是解析光照模型都难以适用。为了解决这个问题，我们提出了RNG，一种新颖的可重光照神经高斯表示方法，能够对具有硬表面或柔软边界的物体进行重光照处理。我们避免了对光照模型的假设，但在每个高斯点上保留了可以通过MLP解码为颜色的特征向量。遵循以往的工作，我们使用点光源来减少模糊性，并引入了一个对阴影敏感的条件到网络中。我们还提出了一个深度优化网络，以帮助在3DGS框架下计算阴影，从而在点光源下实现更好的阴影效果。此外，为了避免3DGS中的alpha混合带来的模糊问题，我们设计了一种混合前向-延迟优化策略。最终，我们实现了比基于神经辐射场的先前工作快约20倍的训练速度和快约600倍的渲染速度，在RTX4090上可实现60帧每秒的渲染速度。