D$^3$epth: Self-Supervised Depth Estimation with Dynamic Mask in Dynamic Scenes
Preprint
Abstract
Depth estimation is a crucial technology in robotics. Recently, self-supervised depth estimation methods have demonstrated great potential as they can efficiently leverage large amounts of unlabelled real-world data. However, most existing methods are designed under the assumption of static scenes, which hinders their adaptability in dynamic environments. To address this issue, we present D$^3$epth, a novel method for self-supervised depth estimation in dynamic scenes. It tackles the challenge of dynamic objects from two key perspectives. First, within the self-supervised framework, we design a reprojection constraint to identify regions likely to contain dynamic objects, allowing the construction of a dynamic mask that mitigates their impact at the loss level. Second, for multi-frame depth estimation, we introduce a cost volume auto-masking strategy that leverages adjacent frames to identify regions associated with dynamic objects and generate corresponding masks. This provides guidance for subsequent processes. Furthermore, we propose a spectral entropy uncertainty module that incorporates spectral entropy to guide uncertainty estimation during depth fusion, effectively addressing issues arising from cost volume computation in dynamic environments. Extensive experiments on KITTI and Cityscapes datasets demonstrate that the proposed method consistently outperforms existing self-supervised monocular depth estimation baselines. Code is available at \url{https://github.com/Csyunling/D3epth}.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Mono-ViFI: A Unified Learning Framework for Self-supervised Single and Multi-frame Monocular Depth Estimation2024
- PMIndoor: Pose Rectified Network and Multiple Loss Functions for Self-Supervised Monocular Indoor Depth EstimationSensors, 2023
- SC-DepthV3: Robust Self-Supervised Monocular Depth Estimation for Dynamic ScenesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- Exploring the Mutual Influence Between Self-Supervised Single-Frame and Multi-Frame Depth EstimationIEEE Robotics and Automation Letters, 2023
- DS-Depth: Dynamic and Static Depth Estimation via a Fusion Cost VolumeIEEE Transactions on Circuits and Systems for Video Technology, 2023
- IterDepth: Iterative Residual Refinement for Outdoor Self-Supervised Multi-Frame Monocular Depth EstimationIEEE Transactions on Circuits and Systems for Video Technology, 2023
- Don’t Forget The Past: Recurrent Depth Estimation from Monocular VideoIEEE Robotics and Automation Letters, 2020
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004