Spike cameras, as an innovative neuromorphic camera that captures scenes with the 0-1 bit stream at 40 kHz, are increasingly employed for the 3D reconstruction task via Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (3DGS). Previous spike-based 3D reconstruction approaches often employ a casecased pipeline: starting with high-quality image reconstruction from spike streams based on established spike-to-image reconstruction algorithms, then progressing to camera pose estimation and 3D reconstruction. However, this cascaded approach suffers from substantial cumulative errors, where quality limitations of initial image reconstructions negatively impact pose estimation, ultimately degrading the fidelity of the 3D reconstruction. To address these issues, we propose a synergistic optimization framework, \textbf{USP-Gaussian}, that unifies spike-based image reconstruction, pose correction, and Gaussian splatting into an end-to-end framework. Leveraging the multi-view consistency afforded by 3DGS and the motion capture capability of the spike camera, our framework enables a joint iterative optimization that seamlessly integrates information between the spike-to-image network and 3DGS. Experiments on synthetic datasets with accurate poses demonstrate that our method surpasses previous approaches by effectively eliminating cascading errors. Moreover, we integrate pose optimization to achieve robust 3D reconstruction in real-world scenarios with inaccurate initial poses, outperforming alternative methods by effectively reducing noise and preserving fine texture details.
尖峰相机作为一种创新的类脑神经形态相机，以 40 kHz 的速率捕获场景并生成 0-1 比特流，正逐步应用于通过神经辐射场（NeRF）或三维高斯点（3DGS）进行三维重建任务。以往基于尖峰相机的三维重建方法通常采用一个级联的处理流程：首先通过现有的尖峰流到图像的重建算法生成高质量的图像，然后进行相机位姿估计和三维重建。然而，这种级联方法存在显著的累积误差问题，初始图像重建质量的限制会对位姿估计产生负面影响，从而最终降低三维重建的精度。 为了解决这些问题，我们提出了一种协同优化框架，称为 USP-Gaussian，将基于尖峰的图像重建、位姿校正和高斯点绘制统一到一个端到端的框架中。通过利用 3DGS 提供的多视图一致性和尖峰相机的运动捕获能力，该框架实现了尖峰流到图像网络和 3DGS 之间信息的联合迭代优化。在合成数据集上的实验表明，即使初始位姿非常准确，我们的方法仍能通过有效消除级联误差而优于现有方法。此外，在真实场景中面对不准确的初始位姿时，我们集成了位姿优化，能够实现稳健的三维重建，相较于其他方法，我们的方法能够有效降低噪声并保留细腻的纹理细节。