Pose-Aware Attention Network for Flexible Motion Retargeting by Body Part
- 19 May 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 30 (8) , 4792-4808
Abstract
Motion retargeting is a fundamental problem in computer graphics and computer vision. Existing approaches usually have many strict requirements, such as the source-target skeletons needing to have the same number of joints or share the same topology. To tackle this problem, we note that skeletons with different structure may have some common body parts despite the differences in joint numbers. Following this observation, we propose a novel, flexible motion retargeting framework. The key idea of our method is to regard the body part as the basic retargeting unit rather than directly retargeting the whole body motion. To enhance the spatial modeling capability of the motion encoder, we introduce a pose-aware attention network (PAN) in the motion encoding phase. The PAN is pose-aware since it can dynamically predict the joint weights within each body part based on the input pose, and then construct a shared latent space for each body part by feature pooling. Extensive experiments show that our approach can generate better motion retargeting results both qualitatively and quantitatively than state-of-the-art methods. Moreover, we also show that our framework can generate reasonable results even for a more challenging retargeting scenario, like retargeting between bipedal and quadrupedal skeletons because of the body part retargeting strategy and PAN.Keywords
All Related Versions
This publication has 47 references indexed in Scilit:
- Skeleton-Free Pose Transfer for Stylized 3D Characters2022
- End-to-End Human Pose and Mesh Reconstruction with Transformers2021
- Flow Guided Transformable Bottleneck Networks for Motion Retargeting2021
- Decoupled Spatial-Temporal Attention Network for Skeleton-Based Action-Gesture Recognition2021
- Contact and Human Dynamics from Monocular Video2020
- A variational U‐Net for motion retargetingComputer Animation and Virtual Worlds, 2020
- Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition2019
- Neural Kinematic Networks for Unsupervised Motion Retargetting2018
- Enriching a motion database by analogous combination of partial human motionsThe Visual Computer, 2007
- Online motion retargettingThe Journal of Visualization and Computer Animation, 2000