ADfM-Net: An Adversarial Depth-From-Motion Network Based on Cross Attention and Motion Enhanced
- 20 April 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (8) , 4879-4886
Abstract
The temporal consistent and accurate depth estimation for consecutive images is essential for many downstream applications. However, most existing methods only infer depth from a single image, ignoring the temporal information and important depth cues from motion in the sequence. Additionally, the depths of adjacent frames are estimated separately without any constraint. In this paper, we promote the temporal consistency and accuracy of depth results from the aforementioned two aspects: multi-frame framework and consistency constraint. Firstly, a framework with cross-frame attention and motion enhancement module is proposed for better temporal consistency and depth precision. Secondly, an adversarial metric learning strategy is introduced to further constrain the consistency of adjacent depth results, without any additional computation and memory cost. The experiments on KITTI and Cityscapes datasets demonstrate the effectiveness of our framework. Furthermore, noting that the traditional metrics can not reveal the consistency of the depth results, a new temporal consistency metric is proposed, which would facilitate further research.Keywords
This publication has 40 references indexed in Scilit:
- Unsupervised Monocular Depth Estimation with Left-Right Consistency2017
- Unsupervised Learning of Depth and Ego-Motion from Video2017
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture2015
- JOINT 3D ESTIMATION OF VEHICLES AND SCENE FLOWISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2015
- Depth Transfer: Depth Extraction from Video Using Non-Parametric SamplingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Vision meets robotics: The KITTI datasetThe International Journal of Robotics Research, 2013
- Consistent Depth Maps Recovery from a Video SequenceIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving ObjectsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1985