MixedGaussianAvatar: Realistically and Geometrically Accurate Head Avatar via Mixed 2D-3D Gaussian Splatting
Reconstructing high-fidelity 3D head avatars is crucial in various applications such as virtual reality. The pioneering methods reconstruct realistic head avatars with Neural Radiance Fields (NeRF), which have been limited by training and rendering speed. Recent methods based on 3D Gaussian Splatting (3DGS) significantly improve the efficiency of training and rendering. However, the surface inconsistency of 3DGS results in subpar geometric accuracy; later, 2DGS uses 2D surfels to enhance geometric accuracy at the expense of rendering fidelity. To leverage the benefits of both 2DGS and 3DGS, we propose a novel method named MixedGaussianAvatar for realistically and geometrically accurate head avatar reconstruction. Our main idea is to utilize 2D Gaussians to reconstruct the surface of the 3D head, ensuring geometric accuracy. We attach the 2D Gaussians to the triangular mesh of the FLAME model and connect additional 3D Gaussians to those 2D Gaussians where the rendering quality of 2DGS is inadequate, creating a mixed 2D-3D Gaussian representation. These 2D-3D Gaussians can then be animated using FLAME parameters. We further introduce a progressive training strategy that first trains the 2D Gaussians and then fine-tunes the mixed 2D-3D Gaussians. We demonstrate the superiority of MixedGaussianAvatar through comprehensive experiments.
重建高保真3D头部头像在虚拟现实等多种应用中至关重要。先前的方法使用神经辐射场（Neural Radiance Fields, NeRF）重建逼真的头部头像，但受限于训练和渲染速度。基于3D高斯点云（3D Gaussian Splatting, 3DGS）的最新方法显著提升了训练和渲染效率。然而，3DGS的表面不一致性导致几何精度不足；后续的2DGS通过2D点云提高了几何精度，但以渲染质量为代价。为同时利用2DGS和3DGS的优势，我们提出了一种名为 MixedGaussianAvatar 的新方法，用于实现逼真且几何准确的头部头像重建。 我们的核心思路是利用2D高斯点云重建3D头部表面，以确保几何精度。具体而言，我们将2D高斯点云附着在FLAME模型的三角网格上，并在2DGS渲染质量不足的地方附加额外的3D高斯点云，形成混合的2D-3D高斯表示。这些2D-3D高斯点云可以通过FLAME参数进行动画化。 此外，我们引入了一种渐进式训练策略：首先训练2D高斯点云，然后对混合的2D-3D高斯点云进行微调。综合实验结果表明，MixedGaussianAvatar 在真实感和几何精度上均表现出色，优于现有方法。