Bilateral Propagation Network for Depth Completion
Authors:
- 16 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9763-9772
Abstract
Depth completion aims to derive a dense depth map from sparse depth measurements with a synchronized color image. Current state-of-the-art (SOTA) methods are predominantly propagation-based, which work as an iterative refinement on the initial estimated dense depth. However, the initial depth estimations mostly result from direct applications of convolutional layers on the sparse depth map. In this paper, we present a Bilateral Propagation Network (BP-Net), that propagates depth at the earliest stage to avoid directly convolving on sparse data. Specifically, our approach propagates the target depth from nearby depth measurements via a non-linear model, whose coefficients are generated through a multi-layer perceptron conditioned on both radiometric difference and spatial distance. By integrating bilateral propagation with multi-modal fusion and depth refinement in a multi-scale framework, our BP-Net demonstrates outstanding performance on both indoor and outdoor scenes. It achieves SOTA on the NYUv2 dataset and ranks 1st on the KITTI depth completion benchmark at the time of submission. Experimental results not only show the effectiveness of bilateral propagation but also emphasize the significance of early-stage propagation in contrast to the refinement stage. Our code and trained models will be available on the project page.Keywords
All Related Versions
This publication has 47 references indexed in Scilit:
- GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNs2022
- RigNet: Repetitive Image Guided Network for Depth Completion2022
- Non-local Spatial Propagation Network for Depth Completion2020
- Estimating Depth from RGB and Sparse Sensing2018
- In Defense of Classical Image Processing: Fast Depth Completion on the CPU2018
- Sparsity Invariant CNNs2017
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012
- Indoor Segmentation and Support Inference from RGBD Images2012
- Fusion of range and color images for denoising and resolution enhancement with a non-local filterComputer Vision and Image Understanding, 2010