DeepHuman: 3D Human Reconstruction from a Single Image
Authors:
Preprint
Abstract
We propose DeepHuman, an image-guided volume-to-volume translation CNN for 3D human reconstruction from a single RGB image. To reduce the ambiguities associated with the surface geometry reconstruction, even for the reconstruction of invisible areas, we propose and leverage a dense semantic representation generated from SMPL model as an additional input. One key feature of our network is that it fuses different scales of image features into the 3D space through volumetric feature transformation, which helps to recover accurate surface geometry. The visible surface details are further refined through a normal refinement network, which can be concatenated with the volume generation network using our proposed volumetric normal projection layer. We also contribute THuman, a 3D real-world human model dataset containing about 7000 models. The network is trained using training data generated from the dataset. Overall, due to the specific design of our network and the diversity in our dataset, our method enables 3D human model estimation given only a single image and outperforms state-of-the-art approaches.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Towards Fully Mobile 3D Face, Body, and Environment Capture Using Only Head-worn CamerasIEEE Transactions on Visualization and Computer Graphics, 2018
- Detailed Human Avatars from Monocular Video2018
- 3D Human Pose Estimation in the Wild by Adversarial Learning2018
- Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision2017
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human MotionInternational Journal of Computer Vision, 2009
- A Point-Cloud-Based Multiview Stereo Algorithm for Free-Viewpoint VideoIEEE Transactions on Visualization and Computer Graphics, 2009
- Surface Capture for Performance-Based AnimationIEEE Computer Graphics and Applications, 2007
- Scalable 3D video of dynamic scenesThe Visual Computer, 2005
- Virtualized reality: constructing virtual worlds from real scenesIEEE MultiMedia, 1997