A Twist Representation and Shape Refinement Method for Human Mesh Recovery
- 25 December 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 1821-1834
Abstract
3D human mesh recovery from single RGB images or monocular videos is a challenging task. The twist representation utilized in existing inverse kinematics-based methods fails to accurately describe the twisting posture when the estimated bone direction is imprecise. Additionally, supervising SMPL shape parameters has the issue of shape estimation overfitting due to limited training data. This often results in compromised bone lengths that subsequently impair the precision of joint positions. To address these issues, we propose a framework that breaks down both human pose and shape into finer components, effectively managing and minimizing errors within each component. The proposed framework integrates two key advancements: the advanced Ortho-Twist and Swing Representation (OTSR) and the Skeleton-Focused Shape Refinement (SFSR). OTSR offers a more sophisticated representation for limb rotations compared to the traditional twist angle and swing representation to enhance the accuracy of twisting posture estimation. SFSR refines the estimated SMPL shape parameters by fitting bone lengths using the estimated joint positions, thereby significantly mitigating shape overfitting and enhancing joint position accuracy in the recovered mesh.We conduct experiments on the Human3.6M and 3DPW datasets. The results demonstrate the superiority of the proposed framework in both single-image and video scenarios. Additionally, the ablation studies confirm the effectiveness of our proposed modules, and further generalizability experiments demonstrate that our two key advancements can serve as plugand- play modules to enhance existing methods.Keywords
Funding Information
- National Key Research and Development Project (2023YFC3321600)
- Shenzhen Technology (JCYJ20220531095810023)
- Open Fund of Key Laboratory of Urban Land Resources Monitoring and Simulation
- Ministry of Natural Resources (KF-2023-08-22)
- Guangdong Major Project of Basic and Applied Basic Research (2023B0303000016)
This publication has 51 references indexed in Scilit:
- CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation2022
- Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers2022
- 3D Human Motion Estimation via Motion Compression and Refinement2021
- I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image2020
- Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose2020
- Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera2018
- Cascaded Pyramid Network for Multi-person Pose Estimation2018
- Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision2017
- Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image2016
- Microsoft COCO: Common Objects in Context2014