SSD-Poser: Avatar Pose Estimation with State Space Duality from Sparse Observations
Authors:
Preprint
Abstract
The growing applications of AR/VR increase the demand for real-time full-body pose estimation from Head-Mounted Displays (HMDs). Although HMDs provide joint signals from the head and hands, reconstructing a full-body pose remains challenging due to the unconstrained lower body. Recent advancements often rely on conventional neural networks and generative models to improve performance in this task, such as Transformers and diffusion models. However, these approaches struggle to strike a balance between achieving precise pose reconstruction and maintaining fast inference speed. To overcome these challenges, a lightweight and efficient model, SSD-Poser, is designed for robust full-body motion estimation from sparse observations. SSD-Poser incorporates a well-designed hybrid encoder, State Space Attention Encoders, to adapt the state space duality to complex motion poses and enable real-time realistic pose reconstruction. Moreover, a Frequency-Aware Decoder is introduced to mitigate jitter caused by variable-frequency motion signals, remarkably enhancing the motion smoothness. Comprehensive experiments on the AMASS dataset demonstrate that SSD-Poser achieves exceptional accuracy and computational efficiency, showing outstanding inference efficiency compared to state-of-the-art methods.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation2024
- Enhancing Motion Reconstruction From Sparse Tracking Inputs With Kinematic ConstraintsIEEE Transactions on Automation Science and Engineering, 2024
- AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensing2022
- Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensors2022
- HuMoR: 3D Human Motion Model for Robust Pose Estimation2021
- LoBSTr: Real‐time Lower‐body Pose Prediction from Sparse Upper‐body Tracking SignalsComputer Graphics Forum, 2021
- Total Capture: 3D Human Pose Estimation Fusing Video and Inertial Sensors2017