Revisiting Deformable Convolution for Depth Completion
Authors:
- Xinglong SunXSXinglong Sun
- Jean PonceJPJean Ponce
- Yu-Xiong WangYWYu-Xiong Wang
Preprint
Abstract
Depth completion, which aims to generate high-quality dense depth maps from sparse depth maps, has attracted increasing attention in recent years. Previous work usually employs RGB images as guidance, and introduces iterative spatial propagation to refine estimated coarse depth maps. However, most of the propagation refinement methods require several iterations and suffer from a fixed receptive field, which may contain irrelevant and useless information with very sparse input. In this paper, we address these two challenges simultaneously by revisiting the idea of deformable convolution. We propose an effective architecture that leverages deformable kernel convolution as a single-pass refinement module, and empirically demonstrate its superiority. To better understand the function of deformable convolution and exploit it for depth completion, we further systematically investigate a variety of representative strategies. Our study reveals that, different from prior work, deformable convolution needs to be applied on an estimated depth map with a relatively high density for better performance. We evaluate our model on the large-scale KITTI dataset and achieve state-of-the-art level performance in both accuracy and inference speed. Our code is available at https://github.com/AlexSunNik/ReDC.Keywords
All Related Versions
This publication has 44 references indexed in Scilit:
- RigNet: Repetitive Image Guided Network for Depth Completion2022
- Non-local Spatial Propagation Network for Depth Completion2020
- Multi‐scale features fusion from sparse LiDAR data and single image for depth completionElectronics Letters, 2018
- Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network2018
- Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation2018
- Sparsity Invariant CNNs2017
- Depth Map Super-Resolution by Deep Multi-Scale Guidance2016
- Deep Joint Image Filtering2016
- Semantically Guided Depth Upsampling2016
- Guided Depth Enhancement via Anisotropic Diffusion2013