Learning Depth With Very Sparse Supervision
- 14 July 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 5 (4) , 5542-5549
Abstract
Motivated by the astonishing capabilities of natural intelligent agents and inspired by theories from psychology, this paper explores the idea that perception gets coupled to 3D properties of the world via interaction with the environment. Existing works for depth estimation require either massive amounts of annotated training data or some form of hard-coded geometrical constraint. This paper explores a new approach to learning depth perception requiring neither of those. Specifically, we propose a novel global-local network architecture that can be trained with the data observed by a robot exploring an environment: images and extremely sparse depth measurements, down to even a single pixel per image. From a pair of consecutive images, the proposed network outputs a latent representation of the camera’s and scene’s parameters, and a dense depth map. Experiments on several datasets show that, when ground truth is available even for just one of the image pixels, the proposed network can learn monocular dense depth estimation up to 22.5% more accurately than state-of-the-art approaches. We believe that this work, in addition to its scientific interest, lays the foundations to learn depth with extremely sparse supervision, which can be valuable to all robotic systems acting under severe bandwidth or sensing constraints.Keywords
All Related Versions
Funding Information
- Intel Corporation
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
- SNSF-ERC Starting Grant
- Swiss National Center of Competence Research (NCCR) Robotics
This publication has 36 references indexed in Scilit:
- ORB-SLAM: A Versatile and Accurate Monocular SLAM SystemIEEE Transactions on Robotics, 2015
- SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels2013
- A benchmark for the evaluation of RGB-D SLAM systems2012
- DTAM: Dense tracking and mapping in real-time2011
- Evaluation of Stereo Matching Costs on Images with Radiometric DifferencesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2008
- Line Geometry and Camera AutocalibrationJournal of Mathematical Imaging and Vision, 2008
- Bundle Adjustment — A Modern Synthesis2000
- TriangulationComputer Vision and Image Understanding, 1997
- Optic flowVision Research, 1986
- A computer algorithm for reconstructing a scene from two projectionsNature, 1981