Recently, image-to-3D approaches have significantly advanced the generation quality and speed of 3D assets based on large reconstruction models, particularly 3D Gaussian reconstruction models. Existing large 3D Gaussian models directly map 2D image to 3D Gaussian parameters, while regressing 2D image to 3D Gaussian representations is challenging without 3D priors. In this paper, we propose a large Point-to-Gaussian model, that inputs the initial point cloud produced from large 3D diffusion model conditional on 2D image to generate the Gaussian parameters, for image-to-3D generation. The point cloud provides initial 3D geometry prior for Gaussian generation, thus significantly facilitating image-to-3D Generation. Moreover, we present the Attention mechanism, Projection mechanism, and Point feature extractor, dubbed as APP block, for fusing the image features with point cloud features. The qualitative and quantitative experiments extensively demonstrate the effectiveness of the proposed approach on GSO and Objaverse datasets, and show the proposed method achieves state-of-the-art performance.
最近，基于大型重建模型的图像到三维（image-to-3D）方法在生成质量和速度方面取得了显著进展，尤其是在三维高斯重建模型方面。现有的大型三维高斯模型直接将二维图像映射到三维高斯参数上，而在没有三维先验的情况下，将二维图像回归到三维高斯表示是具有挑战性的。在本文中，我们提出了一种大型 Point-to-Gaussian 模型，该模型输入由大型三维扩散模型在二维图像条件下生成的初始点云，以生成高斯参数，用于图像到三维的生成。点云提供了高斯生成的初始三维几何先验，因此显著促进了图像到三维的生成。此外，我们提出了注意力机制、投影机制和点特征提取器，统称为 APP 块，用于融合图像特征和点云特征。定性和定量实验广泛证明了该方法在 GSO 和 Objaverse 数据集上的有效性，并显示该方法达到了当前最先进的性能。