Despite the substantial progress of novel view synthesis, existing methods, either based on the Neural Radiance Fields (NeRF) or more recently 3D Gaussian Splatting (3DGS), suffer significant degradation when the input becomes sparse. Numerous efforts have been introduced to alleviate this problem, but they still struggle to synthesize satisfactory results efficiently, especially in the large scene. In this paper, we propose SCGaussian, a Structure Consistent Gaussian Splatting method using matching priors to learn 3D consistent scene structure. Considering the high interdependence of Gaussian attributes, we optimize the scene structure in two folds: rendering geometry and, more importantly, the position of Gaussian primitives, which is hard to be directly constrained in the vanilla 3DGS due to the non-structure property. To achieve this, we present a hybrid Gaussian representation. Besides the ordinary non-structure Gaussian primitives, our model also consists of ray-based Gaussian primitives that are bound to matching rays and whose optimization of their positions is restricted along the ray. Thus, we can utilize the matching correspondence to directly enforce the position of these Gaussian primitives to converge to the surface points where rays intersect. Extensive experiments on forward-facing, surrounding, and complex large scenes show the effectiveness of our approach with state-of-the-art performance and high efficiency.
尽管新颖视图合成取得了显著进展，现有方法（无论是基于神经辐射场（NeRF）还是最近的3D Gaussian Splatting（3DGS））在输入稀疏时仍会出现显著性能退化。尽管已经提出了许多改进措施来缓解这一问题，但在大场景中高效生成令人满意的结果依然具有挑战性。在本文中，我们提出了一种名为 SCGaussian 的结构一致性高斯分布（Structure Consistent Gaussian Splatting）方法，通过匹配先验学习3D一致的场景结构。考虑到高斯属性之间的高度相关性，我们从两方面优化场景结构：渲染几何以及更重要的高斯基元的位置。由于标准3DGS方法的非结构特性，高斯基元位置难以直接约束。为此，我们提出了一种混合高斯表示。除了普通的非结构高斯基元外，我们的模型还包括基于射线的高斯基元，这些基元绑定于匹配的射线，其位置优化限制在射线方向上。因此，我们能够利用匹配对应关系，直接强制这些高斯基元的位置收敛到射线与表面交点处。广泛的实验表明，在前向视图、环绕场景以及复杂大场景中，我们的方法表现出了高效的性能和领先的效果。