End-to-End 3D Human Pose Estimation Network With Multi-Layer Feature Fusion
Open Access
- 24 June 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 89124-89134
Abstract
The 3D human pose estimation is a technique used to determine the position of the human body in a three-dimensional space. This involves identifying body rotations, joint angles, and other pose-related information from image or video data. In this paper, we propose an end-to-end 3D human pose estimation network that is based on multi-level feature fusion.The network is composed of two main components. The first component utilizes the deepest features extracted by the backbone network. These features undergo initial data encoding and are then processed by the Semantic Information Extraction Module, which primarily consists of a multi-head self-attention mechanism. This module extracts deeper features, resulting in primary human body feature data. The second component focuses on the shallowest features and inputs them into the Global Information Processing Module, which performs global feature extraction.The features extracted from both components, along with the Bbox info (bounding box information), are collectively fed into the Iterative Regression Module. This module generates human pose data, which is then utilized to reconstruct and generate the human body using a human pose model. To evaluate the performance of our method, we train and test it on well-known benchmark datasets such as 3DPW, AGORA and MPII. Our method demonstrates exceptional performance, as it achieves a reduction of approximately 5.3% on the PA-MPJPE metric and approximately 5.1% on the MPJPE metric compared to the best model we referenced.Keywords
Funding Information
- youth program of national natural science foundation of china (62106083)
- Natural Science Foundation of Fujian Province of China (2021J01868, 2022HZ022019, 2022J05162)
This publication has 50 references indexed in Scilit:
- Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera2018
- Integral Human Pose Regression2018
- Single-Shot Multi-person 3D Pose Estimation from Monocular RGB2018
- Human Pose Estimation with Iterative Error Feedback2016
- 3D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Information2016
- Bayesian Image Based 3D Pose Estimation2016
- Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video2016
- Deep Residual Learning for Image Recognition2016
- A Dual-Source Approach for 3D Pose Estimation from a Single Image2016
- 2D Human Pose Estimation: New Benchmark and State of the Art Analysis2014