Adversarial Learning of Structure-Aware Fully Convolutional Networks for Landmark Localization
- 26 February 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 42 (7) , 1654-1669
Abstract
Landmark/pose estimation in single monocular images has received much effort in computer vision due to its important applications. It remains a challenging task when input images come with severe occlusions caused by, e.g., adverse camera views. Under such circumstances, biologically implausible pose predictions may be produced. In contrast, human vision is able to predict poses by exploiting geometric constraints of landmark point inter-connectivity. To address the problem, by incorporating priors about the structure of pose components, we propose a novel structure-aware fully convolutional network to implicitly take such priors into account during training of the deep network. Explicit learning of such constraints is typically challenging. Instead, inspired by how human identifies implausible poses, we design discriminators to distinguish the real poses from the fake ones (such as biologically implausible ones). If the pose generator G generates results that the discriminator fails to distinguish from real ones, the network successfully learns the priors. Training of the network follows the strategy of conditional Generative Adversarial Networks (GANs). The effectiveness of the proposed network is evaluated on three pose-related tasks: 2D human pose estimation, 2D facial landmark estimation and 3D human pose estimation. The proposed approach significantly outperforms several state-of-the-art methods and almost always generates plausible pose predictions, demonstrating the usefulness of implicit learning of structures using GANs.Keywords
All Related Versions
Funding Information
- National Science Fund of China (#91420201, 61472187, 61502235, 61233011, 61373063)
- Australian Research Council
This publication has 89 references indexed in Scilit:
- L2,1-based regression and prediction accumulation across views for robust facial landmark detectionImage and Vision Computing, 2016
- DeepPose: Human Pose Estimation via Deep Neural Networks2014
- Face Alignment by Explicit Shape RegressionInternational Journal of Computer Vision, 2013
- Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural EnvironmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- 2D Articulated Human Pose Estimation and Retrieval in (Almost) Unconstrained Still ImagesInternational Journal of Computer Vision, 2012
- Interactive Facial Feature Localization2012
- Reconstructing 3D Human Pose from 2D Image Landmarks2012
- Upper Body Detection and Tracking in Extended Signing SequencesInternational Journal of Computer Vision, 2011
- Cascaded Models for Articulated Pose Estimation2010
- Determination of 3D human body postures from a single viewComputer Vision, Graphics, and Image Processing, 1985