Human Mesh Reconstruction with Generative Adversarial Networks from Single RGB Images
Open Access
Abstract
Applications related to smart cities require virtual cities in the experimental development stage. To build a virtual city that are close to a real city, a large number of various types of human models need to be created. To reduce the cost of acquiring models, this paper proposes a method to reconstruct 3D human meshes from single images captured using a normal camera. It presents a method for reconstructing the complete mesh of the human body from a single RGB image and a generative adversarial network consisting of a newly designed shape–pose-based generator (based on deep convolutional neural networks) and an enhanced multi-source discriminator. Using a machine learning approach, the reliance on multiple sensors is reduced and 3D human meshes can be recovered using a single camera, thereby reducing the cost of building smart cities. The proposed method achieves an accuracy of 92.1% in body shape recovery; it can also process 34 images per second. The method proposed in this paper approach significantly improves the performance compared with previous state-of-the-art approaches. Given a single view image of various humans, our results can be used to generate various 3D human models, which can facilitate 3D human modeling work to simulate virtual cities. Since our method can also restore the poses of the humans in the image, it is possible to create various human poses by given corresponding images with specific human poses.Keywords
Funding Information
- National Research Foundation of Korea (2018R1A2B2007934)
- Dongguk University (Dongguk University Research Fund of 2020)
This publication has 20 references indexed in Scilit:
- GANimation: Anatomically-Aware Facial Animation from a Single Image2018
- VNectACM Transactions on Graphics, 2017
- Learning from Synthetic Humans2017
- Direct, Dense, and Deformable: Template-Based Non-rigid 3D Reconstruction from RGB Video2015
- MoShACM Transactions on Graphics, 2014
- Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural EnvironmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Markerless Motion Capture of Multiple Characters Using Multiview Image SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human MotionInternational Journal of Computer Vision, 2009
- A Statistical Model of Human Pose and Body ShapeComputer Graphics Forum, 2009
- SCAPEACM Transactions on Graphics, 2005