Skip to content

Latest commit

 

History

History
7 lines (5 loc) · 2.36 KB

2411.16768.md

File metadata and controls

7 lines (5 loc) · 2.36 KB

GAST: Sequential Gaussian Avatars with Hierarchical Spatio-temporal Context

3D human avatars, through the use of canonical radiance fields and per-frame observed warping, enable high-fidelity rendering and animating. However, existing methods, which rely on either spatial SMPL(-X) poses or temporal embeddings, respectively suffer from coarse rendering quality or limited animation flexibility. To address these challenges, we propose GAST, a framework that unifies 3D human modeling with 3DGS by hierarchically integrating both spatial and temporal information. Specifically, we design a sequential conditioning framework for the non-rigid warping of the human body, under whose guidance more accurate 3D Gaussians can be obtained in the observation space. Moreover, the explicit properties of Gaussians allow us to embed richer sequential information, encompassing both the coarse sequence of human poses and finer per-vertex motion details. These sequence conditions are further sampled across different temporal scales, in a coarse-to-fine manner, ensuring unbiased inputs for non-rigid warping. Experimental results demonstrate that our method combined with hierarchical spatio-temporal modeling surpasses concurrent baselines, delivering both high-quality rendering and flexible animating capabilities.

通过使用规范辐射场和逐帧观察到的形变,3D 人体化身能够实现高保真的渲染和动画。然而,现有方法依赖空间上的 SMPL(-X) 姿态或时间嵌入,分别面临渲染质量粗糙或动画灵活性受限的问题。 为了解决这些挑战,我们提出了 GAST,一个将 3D 人体建模与 3D 高斯投影(3DGS)相统一的框架,通过层次化整合空间和时间信息实现高效建模。具体来说,我们设计了一种顺序条件框架,用于非刚体的人体形变,在其引导下,可以在观测空间中获得更精确的 3D 高斯。此外,高斯的显式属性允许我们嵌入更丰富的序列信息,涵盖人体姿态的粗略序列以及更细粒度的逐顶点运动细节。 这些序列条件以粗到细的方式在不同时间尺度上进行采样,从而确保非刚体形变的输入不带偏差。实验结果表明,我们结合层次化时空建模的方法,超越了现有的同期基线,实现了高质量渲染和灵活的动画能力,显著提升了 3D 人体建模的表现力和实用性。