Recently, 3D Gaussian splatting has gained attention for its capability to generate high-fidelity rendering results. At the same time, most applications such as games, animation, and AR/VR use mesh-based representations to represent and render 3D scenes. We propose a novel approach that integrates mesh representation with 3D Gaussian splats to perform high-quality rendering of reconstructed real-world scenes. In particular, we introduce a distance-based Gaussian splatting technique to align the Gaussian splats with the mesh surface and remove redundant Gaussian splats that do not contribute to the rendering. We consider the distance between each Gaussian splat and the mesh surface to distinguish between tightly-bound and loosely-bound Gaussian splats. The tightly-bound splats are flattened and aligned well with the mesh geometry. The loosely-bound Gaussian splats are used to account for the artifacts in reconstructed 3D meshes in terms of rendering. We present a training strategy of binding Gaussian splats to the mesh geometry, and take into account both types of splats. In this context, we introduce several regularization techniques aimed at precisely aligning tightly-bound Gaussian splats with the mesh surface during the training process. We validate the effectiveness of our method on large and unbounded scene from mip-NeRF 360 and Deep Blending datasets. Our method surpasses recent mesh-based neural rendering techniques by achieving a 2dB higher PSNR, and outperforms mesh-based Gaussian splatting methods by 1.3 dB PSNR, particularly on the outdoor mip-NeRF 360 dataset, demonstrating better rendering quality. We provide analyses for each type of Gaussian splat and achieve a reduction in the number of Gaussian splats by 30% compared to the original 3D Gaussian splatting.
最近，3D高斯散射因其生成高保真渲染结果的能力而受到关注。同时，大多数应用（如游戏、动画和AR/VR）使用基于网格的表示来表达和渲染3D场景。我们提出了一种新颖的方法，将网格表示与3D高斯散射相结合，以对重建的真实世界场景进行高质量渲染。具体来说，我们引入了一种基于距离的高斯散射技术，将高斯点与网格表面对齐，并移除对渲染无贡献的冗余高斯点。我们考虑每个高斯点与网格表面之间的距离，以区分紧密绑定的高斯点和松散绑定的高斯点。紧密绑定的高斯点被压平并与网格几何形状对齐，而松散绑定的高斯点则用于修正重建3D网格中的渲染瑕疵。我们提出了一种将高斯点绑定到网格几何形状的训练策略，并同时考虑两种类型的高斯点。在此背景下，我们引入了几种正则化技术，旨在训练过程中精确对齐紧密绑定的高斯点与网格表面。我们在mip-NeRF 360和Deep Blending数据集上的大规模和无界场景中验证了我们方法的有效性。我们的方法相较于最近的基于网格的神经渲染技术，在PSNR上提升了2dB，并且在mip-NeRF 360的户外数据集上，较基于网格的高斯散射方法提升了1.3dB PSNR，展现了更好的渲染质量。我们对每种类型的高斯点进行了分析，并将高斯点的数量相比原始3D高斯散射减少了30%。