Unsupervised Learning of Depth Estimation and Camera Pose With Multi-Scale GANs
- 26 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (10) , 17039-17047
Abstract
Unsupervised learning methods have achieved remarkable performance in monocular depth estimation and camera pose, which mostly solve the multi-task learning problem by using their inner geometry consistency as the self-supervision signal. While most existing approaches mostly adopt the generative model to obtain the depth map prediction, so in the resolution of depth map there is room for improvement. To this end, we present our unsupervised learning architecture based on adversarial learning model, which is used for unsupervised learning of high-resolution single view depth and camera pose. Specifically, we present a multi-scale deep convolutional Generative Adversarial Network (GAN) based learning system, which consists of three networks (pose estimation network PCNN, Generator-D and Discriminator-D for depth map prediction). Furthermore, in order to generate high-resolution depth map, we propose a multi-scale GAN model (MSGAN) to decompose the hard high-quality image generation problem into more manageable sub-problems through a coarse-to-fine process. Then, we modify the overall generation architecture of GAN model by changing the down-sampling and up-sampling components to improve the quality and accuracy of the depth map prediction. Finally, in order to improve the rate of convergence, we use the Least Square Error to increase the penalty for outliers. Detailed quantitative and qualitative evaluations of the proposed framework on the KITTI dataset show that the proposed method provides better results for both pose estimation and depth recovery.Keywords
Funding Information
- National Natural Science Foundation of China (61573019, 61751302, 61627810, 61320106010)
- Key Special Projects of National Key Research and Development Plan (2018YFB2003501)
This publication has 33 references indexed in Scilit:
- DeepVO: Towards end-to-end visual odometry with deep Recurrent Convolutional Neural Networks2017
- Generative Image Modeling Using Style and Structure Adversarial Networks2016
- Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue2016
- Generative Visual Manipulation on the Natural Image Manifold2016
- HyperDepth: Learning Depth from Structured Light without Matching2016
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation2016
- Learning Depth from Single Monocular Images Using Deep Convolutional Neural FieldsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture2015
- ORB-SLAM: A Versatile and Accurate Monocular SLAM SystemIEEE Transactions on Robotics, 2015
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004