This study presents RSGaussian, an innovative novel view synthesis (NVS) method for aerial remote sensing scenes that incorporate LiDAR point cloud as constraints into the 3D Gaussian Splatting method, which ensures that Gaussians grow and split along geometric benchmarks, addressing the overgrowth and floaters issues occurs. Additionally, the approach introduces coordinate transformations with distortion parameters for camera models to achieve pixel-level alignment between LiDAR point clouds and 2D images, facilitating heterogeneous data fusion and achieving the high-precision geo-alignment required in aerial remote sensing. Depth and plane consistency losses are incorporated into the loss function to guide Gaussians towards real depth and plane representations, significantly improving depth estimation accuracy. Experimental results indicate that our approach has achieved novel view synthesis that balances photo-realistic visual quality and high-precision geometric estimation under aerial remote sensing datasets. Finally, we have also established and open-sourced a dense LiDAR point cloud dataset along with its corresponding aerial multi-view images, AIR-LONGYAN.
本研究提出了RSGaussian，这是一种创新的新颖视图合成（Novel View Synthesis，NVS）方法，适用于航空遥感场景。该方法将LiDAR点云作为约束引入三维高斯点云（3D Gaussian Splatting）方法，确保高斯点云沿几何基准增长和分裂，从而解决了过度增长和漂浮点的问题。此外，该方法引入了带有畸变参数的坐标变换，用于相机模型，以实现LiDAR点云与二维图像之间的像素级对齐，促进异构数据融合，并实现航空遥感中所需的高精度地理对齐。深度和平面一致性损失被纳入损失函数中，以引导高斯点云朝向真实的深度和平面表示，显著提高了深度估计的准确性。实验结果表明，我们的方法在航空遥感数据集下实现了在照片真实视觉质量和高精度几何估计之间的平衡的新颖视图合成。最后，我们还建立并开源了一个密集的LiDAR点云数据集及其对应的航空多视图图像，命名为AIR-LONGYAN。