Weakly Supervised Adversarial Learning for 3D Human Pose Estimation from Point Clouds
- 13 February 2020
- journal article
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 26 (5) , 1851-1859
Abstract
Point clouds-based 3D human pose estimation that aims to recover the 3D locations of human skeleton joints plays an important role in many AR/VR applications. The success of existing methods is generally built upon large scale data annotated with 3D human joints. However, it is a labor-intensive and error-prone process to annotate 3D human joints from input depth images or point clouds, due to the self-occlusion between body parts as well as the tedious annotation process on 3D point clouds. Meanwhile, it is easier to construct human pose datasets with 2D human joint annotations on depth images. To address this problem, we present a weakly supervised adversarial learning framework for 3D human pose estimation from point clouds. Compared to existing 3D human pose estimation methods from depth images or point clouds, we exploit both the weakly supervised data with only annotations of 2D human joints and fully supervised data with annotations of 3D human joints. In order to relieve the human pose ambiguity due to weak supervision, we adopt adversarial learning to ensure the recovered human pose is valid. Instead of using either 2D or 3D representations of depth images in previous methods, we exploit both point clouds and the input depth image. We adopt 2D CNN to extract 2D human joints from the input depth image, 2D human joints aid us in obtaining the initial 3D human joints and selecting effective sampling points that could reduce the computation cost of 3D human pose regression using point clouds network. The used point clouds network can narrow down the domain gap between the network input i.e. point clouds and 3D joints. Thanks to weakly supervised adversarial learning framework, our method can achieve accurate 3D human pose from point clouds. Experiments on the ITOP dataset and EVAL dataset demonstrate that our method can achieve state-of-the-art performance efficiently.Keywords
Funding Information
- Natural Science Foundation of Beijing Municipality (L182052)
- National Key R&D Program of China (2016YFB1001201)
- National Natural Science Foundation of China (61772499, 61473276)
- Distinguished Young Researcher Program
- Institute of Software, Chinese Academy of Sciences
This publication has 29 references indexed in Scilit:
- Human Pose Estimation from Depth Images via Inference Embedded Multi-task Learning2016
- Towards Viewpoint Invariant 3D Human Pose Estimation2016
- VoxNet: A 3D Convolutional Neural Network for real-time object recognition2015
- DeepPose: Human Pose Estimation via Deep Neural Networks2014
- Real-Time Human Pose Tracking from Range Data2012
- Efficient regression of general-activity human poses from depth images2011
- Real-time human pose recognition in parts from single depth images2011
- Object Detection with Discriminatively Trained Part-Based ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009
- Semi-Supervised Learning (Chapelle, O. et al., Eds.; 2006) [Book reviews]IEEE Transactions on Neural Networks, 2009
- Noise-tolerant learning, the parity problem, and the statistical query modelJournal of the ACM, 2003