We consider the problem of physically-based inverse rendering using 3D Gaussian Splatting (3DGS) representations. While recent 3DGS methods have achieved remarkable results in novel view synthesis (NVS), accurately capturing high-fidelity geometry, physically interpretable materials and lighting remains challenging, as it requires precise geometry modeling to provide accurate surface normals, along with physically-based rendering (PBR) techniques to ensure correct material and lighting disentanglement. Previous 3DGS methods resort to approximating surface normals, but often struggle with noisy local geometry, leading to inaccurate normal estimation and suboptimal material-lighting decomposition. In this paper, we introduce GeoSplatting, a novel hybrid representation that augments 3DGS with explicit geometric guidance and differentiable PBR equations. Specifically, we bridge isosurface and 3DGS together, where we first extract isosurface mesh from a scalar field, then convert it into 3DGS points and formulate PBR equations for them in a fully differentiable manner. In GeoSplatting, 3DGS is grounded on the mesh geometry, enabling precise surface normal modeling, which facilitates the use of PBR frameworks for material decomposition. This approach further maintains the efficiency and quality of NVS from 3DGS while ensuring accurate geometry from the isosurface. Comprehensive evaluations across diverse datasets demonstrate the superiority of GeoSplatting, consistently outperforming existing methods both quantitatively and qualitatively.
我们研究基于物理的逆向渲染问题，使用三维高斯分裂（3DGS）表示。尽管最新的3DGS方法在新视角合成（NVS）中取得了显著成果，但要精确捕捉高保真几何、物理可解释的材质和光照仍然具有挑战性，因为这需要精确的几何建模以提供准确的表面法线，并且需要基于物理的渲染（PBR）技术来确保材质和光照的正确解耦。现有的3DGS方法通常通过近似表面法线来解决该问题，但在处理噪声较大的局部几何时往往会遇到困难，导致法线估计不准和次优的材质光照分解。在本文中，我们提出了一种名为GeoSplatting的新型混合表示方法，通过显式几何引导和可微PBR方程扩展3DGS。具体而言，我们将等值面与3DGS相结合，首先从标量场中提取等值面网格，然后将其转换为3DGS点，并为其构建全可微的PBR方程。在GeoSplatting中，3DGS基于网格几何，使得表面法线建模更加精确，从而支持PBR框架用于材质分解。此方法在保持3DGS的NVS效率和质量的同时，确保了来自等值面的精确几何表现。在多样化数据集上的全面评估表明，GeoSplatting在定量和定性上均显著优于现有方法。