We introduce OmniRe, a holistic approach for efficiently reconstructing high-fidelity dynamic urban scenes from on-device logs. Recent methods for modeling driving sequences using neural radiance fields or Gaussian Splatting have demonstrated the potential of reconstructing challenging dynamic scenes, but often overlook pedestrians and other non-vehicle dynamic actors, hindering a complete pipeline for dynamic urban scene reconstruction. To that end, we propose a comprehensive 3DGS framework for driving scenes, named OmniRe, that allows for accurate, full-length reconstruction of diverse dynamic objects in a driving log. OmniRe builds dynamic neural scene graphs based on Gaussian representations and constructs multiple local canonical spaces that model various dynamic actors, including vehicles, pedestrians, and cyclists, among many others. This capability is unmatched by existing methods. OmniRe allows us to holistically reconstruct different objects present in the scene, subsequently enabling the simulation of reconstructed scenarios with all actors participating in real-time (~60Hz). Extensive evaluations on the Waymo dataset show that our approach outperforms prior state-of-the-art methods quantitatively and qualitatively by a large margin. We believe our work fills a critical gap in driving reconstruction.
我们介绍了 OmniRe，这是一种全面的方法，用于高效重建高保真动态城市场景。近期使用神经辐射场或高斯点云进行驾驶序列建模的方法展示了重建复杂动态场景的潜力，但常常忽视行人和其他非车辆动态角色，阻碍了动态城市场景重建的完整流程。为此，我们提出了一个全面的 3DGS 框架，名为 OmniRe，它允许在驾驶日志中准确、完整地重建各种动态物体。OmniRe 基于高斯表示构建动态神经场景图，并构建多个本地典型空间，以建模包括车辆、行人和骑自行车者在内的各种动态角色。这一能力在现有方法中无可比拟。OmniRe 使我们能够全面重建场景中存在的不同物体，进而实现所有参与者实时（~60Hz）模拟重建场景。在 Waymo 数据集上的广泛评估表明，我们的方法在定量和定性方面均大幅超越了现有的最先进方法。我们相信我们的工作填补了驾驶重建中的一个关键空白。