Skip to content

Latest commit

 

History

History
5 lines (3 loc) · 1.78 KB

2404.03613.md

File metadata and controls

5 lines (3 loc) · 1.78 KB

Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting

As 3D Gaussian Splatting (3DGS) provides fast and high-quality novel view synthesis, it is a natural extension to deform a canonical 3DGS to multiple frames. However, previous works fail to accurately reconstruct dynamic scenes, especially 1) static parts moving along nearby dynamic parts, and 2) some dynamic areas are blurry. We attribute the failure to the wrong design of the deformation field, which is built as a coordinate-based function. This approach is problematic because 3DGS is a mixture of multiple fields centered at the Gaussians, not just a single coordinate-based framework. To resolve this problem, we define the deformation as a function of per-Gaussian embeddings and temporal embeddings. Moreover, we decompose deformations as coarse and fine deformations to model slow and fast movements, respectively. Also, we introduce an efficient training strategy for faster convergence and higher quality.

由于三维高斯喷溅(3DGS)提供了快速且高质量的新视角合成,将标准的3DGS变形应用于多帧是一个自然的扩展。然而,以往的工作未能准确重建动态场景,特别是1) 静态部分沿着附近的动态部分移动,以及2) 一些动态区域模糊不清。我们将这一失败归因于变形场设计错误,该设计构建为基于坐标的函数。这种方法存在问题,因为3DGS是多个以高斯为中心的场的混合,而不仅仅是一个基于单一坐标的框架。为了解决这个问题,我们将变形定义为每个高斯嵌入和时间嵌入的函数。此外,我们将变形分解为粗变形和细变形,分别模拟慢速和快速运动。同时,我们引入了一种高效的训练策略,以实现更快的收敛和更高的质量。