LiDAR Depth Completion Using Color-Embedded Information via Knowledge Distillation
- 3 December 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (9) , 14482-14496
Abstract
Depth completion is the task of reconstructing dense depth images from sparse LiDAR data. LiDAR depth completion, for which LiDAR data is the only input, is an ill-posed and challenging problem owing to the underlying properties of LiDAR data: extremely few points, presence of discontinuities, and absence of texture information. Accordingly, most approaches are heavily dependent on guided color images, which leads to unsatisfactory results when the color images are degraded. To alleviate the dependency on color images but leverage this information during training, we present a deep convolutional neural network (CNN) consisting of depth and edge CNNs via transferring of knowledge. In order to compensate for the limitations of LiDAR data, we design the edge CNN to learn a gradient depth image from a powerful teacher network through the Knowledge-Distillation method. Since the teacher network is trained with color images, color-embedded information can be obtained in the test phase even if color images are not used as an input. We further propose a Self-Distillation method for transferring the color-embedded features from the edge CNN to the depth CNN. Enforcing the depth features to contain edge information hardly observed in LiDAR data enables the depth CNN to generate more edge-attentive and structure-preserving results. Our novel methods show remarkable results in outdoor and indoor environments for KITTI and NYU-Depth-V2 datasets. Experiments performed with low-channel LiDAR data in KITTI and few depth points in the NYU-Depth-V2 dataset show that our method is robust to data sparsity and applicable in various scenarios.Keywords
Funding Information
- Institute of Information and Communications Technology Planning and Evaluation
- Korean Government, Ministry of Science and ICT (MSIT), the development of human re-identification and masked face recognition based on CCTV camera (2021-0-00172)
This publication has 51 references indexed in Scilit:
- ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic SegmentationIEEE Transactions on Intelligent Transportation Systems, 2017
- Handling pure camera rotation in semi-dense monocular SLAMThe Visual Computer, 2017
- Learning Dynamic Guidance for Depth Image Enhancement2017
- Nighttime Vehicle Detection Based on Bio-Inspired Image Enhancement and Weighted Score-Level Feature FusionIEEE Transactions on Intelligent Transportation Systems, 2016
- The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Efficient Deep Learning for Stereo Matching2016
- Superpixel-based depth map inpainting for RGB-D view synthesis2015
- Kinect v2 for mobile robot navigation: Evaluation and modeling2015
- A Multimodal ADAS System for Unmarked Urban Scenarios Based on Road Context UnderstandingIEEE Transactions on Intelligent Transportation Systems, 2014