Novel-view synthesis aims to generate novel views of a scene from multiple input images or videos, and recent advancements like 3D Gaussian splatting (3DGS) have achieved notable success in producing photorealistic renderings with efficient pipelines. However, generating high-quality novel views under challenging settings, such as sparse input views, remains difficult due to insufficient information in under-sampled areas, often resulting in noticeable artifacts. This paper presents 3DGS-Enhancer, a novel pipeline for enhancing the representation quality of 3DGS representations. We leverage 2D video diffusion priors to address the challenging 3D view consistency problem, reformulating it as achieving temporal consistency within a video generation process. 3DGS-Enhancer restores view-consistent latent features of rendered novel views and integrates them with the input views through a spatial-temporal decoder. The enhanced views are then used to fine-tune the initial 3DGS model, significantly improving its rendering performance. Extensive experiments on large-scale datasets of unbounded scenes demonstrate that 3DGS-Enhancer yields superior reconstruction performance and high-fidelity rendering results compared to state-of-the-art methods.
新视角合成旨在从多个输入图像或视频中生成场景的新视角。近年来，3D高斯点云（3DGS）等方法在生成高效的写实渲染方面取得了显著进展。然而，在稀疏输入视角等具有挑战性的场景下生成高质量的新视角仍然困难，因欠采样区域信息不足，常导致明显的伪影问题。本文提出了一种新颖的增强3DGS表示质量的流程，称为3DGS-Enhancer。我们利用2D视频扩散先验来解决具有挑战性的3D视角一致性问题，将其重新表述为视频生成过程中的时间一致性问题。3DGS-Enhancer恢复了渲染的新视角的视角一致性潜在特征，并通过时空解码器将其与输入视角整合。增强后的视角用于微调初始的3DGS模型，从而显著提高其渲染性能。在大规模无边界场景数据集上的大量实验表明，3DGS-Enhancer相较于最先进的方法，能提供更优越的重建性能和高保真的渲染结果。