Masked GAN for Unsupervised Depth and Pose Prediction With Scale Consistency
- 25 December 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 32 (12) , 5392-5403
Abstract
Previous work has shown that adversarial learning can be used for unsupervised monocular depth and visual odometry (VO) estimation, in which the adversarial loss and the geometric image reconstruction loss are utilized as the mainly supervisory signals to train the whole unsupervised framework. However, the performance of the adversarial framework and image reconstruction is usually limited by occlusions and the visual field changes between the frames. This article proposes a masked generative adversarial network (GAN) for unsupervised monocular depth and ego-motion estimations. The MaskNet and Boolean mask scheme are designed in this framework to eliminate the effects of occlusions and impacts of visual field changes on the reconstruction loss and adversarial loss, respectively. Furthermore, we also consider the scale consistency of our pose network by utilizing a new scale-consistency loss, and therefore, our pose network is capable of providing the full camera trajectory over a long monocular sequence. Extensive experiments on the KITTI data set show that each component proposed in this article contributes to the performance, and both our depth and trajectory predictions achieve competitive performance on the KITTI and Make3D data sets.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (61988101, 61673176)
- Program of Shanghai Academic Research Leader (20XD1401300)
- Programme of Introducing Talents of Discipline to Universities (B17017)
This publication has 35 references indexed in Scilit:
- Unsupervised Learning of Depth and Ego-Motion from Video2017
- ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D CamerasIEEE Transactions on Robotics, 2017
- Direct Sparse OdometryIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation2016
- PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization2015
- Vision meets robotics: The KITTI datasetThe International Journal of Robotics Research, 2013
- Make3D: Learning 3D Scene Structure from a Single Still ImageIEEE Transactions on Pattern Analysis and Machine Intelligence, 2008
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004