Self-supervised Monocular Depth Estimation: Let’s Talk About The Weather
Authors:
- 1 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8873-8883
Abstract
Current, self-supervised depth estimation architectures rely on clear and sunny weather scenes to train deep neural networks. However, in many locations, this assumption is too strong. For example in the UK (2021), 149 days consisted of rain. For these architectures to be effective in real-world applications, we must create models that can generalise to all weather conditions, times of the day and image qualities. Using a combination of computer graphics and generative models, one can augment existing sunny-weather data in a variety of ways that simulate adverse weather effects. While it is tempting to use such data augmentations for self-supervised depth, in the past this was shown to degrade performance instead of improving it. In this paper, we put forward a method that uses augmentations to remedy this problem. By exploiting the correspondence between unaugmented and augmented data we introduce a pseudo-supervised loss for both depth and pose estimation. This brings back some of the benefits of supervised learning while still not requiring any labels. We also make a series of practical recommendations which collectively offer a reliable, efficient framework for weather-related augmentation of self-supervised depth from monocular video. We present extensive testing to show that our method, Robust-Depth, achieves SotA performance on the KITTI dataset while significantly surpassing SotA on challenging, adverse condition data such as DrivingStereo, Foggy CityScape and NuScenes-Night. The project website can be found at https://kieran514.github.io/Robust-Depth-Project/.Keywords
All Related Versions
This publication has 43 references indexed in Scilit:
- RA-Depth: Resolution Adaptive Self-supervised Monocular Depth Estimation2022
- MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer2022
- Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation2021
- Unsupervised Scale-Consistent Depth Learning from VideoInternational Journal of Computer Vision, 2021
- Rain Rendering for Evaluating and Improving Robustness to Bad WeatherInternational Journal of Computer Vision, 2020
- Semantic Foggy Scene Understanding with Synthetic DataInternational Journal of Computer Vision, 2018
- Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012
- ImageNet: A large-scale hierarchical image database2009