LCCNet: LiDAR and Camera Self-Calibration using Cost Volume Network
Authors:
Top Cited Papers
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2888-2895
Abstract
Multi-sensor fusion is for enhancing environment perception and 3D reconstruction in self-driving and robot navigation. Calibration between sensors is the precondition of effective multi-sensor fusion. Laborious manual works and complex environment settings exist in old-fashioned calibration techniques for Light Detection and Ranging (Li-DAR) and camera. We propose an online LiDAR-Camera Self-calibration Network (LCCNet), different from the previous CNN-based methods. LCCNet can be trained end-to-end and predict the extrinsic parameters in real-time. In the LCCNet, we exploit the cost volume layer to express the correlation between the RGB image features and the depth image projected from point clouds. Besides using the smooth L1-Loss of the predicted extrinsic calibration parameters as a supervised signal, an additional self-supervised signal, point cloud distance loss, is applied during training. Instead of directly regressing the extrinsic parameters, we predict the decalibrated deviation from initial calibration to the ground truth. The calibration error decreases further with iterative refinement and the temporal filtering approach in the inference stage. The execution time of the calibration process is 24ms for each iteration on a single GPU. LCCNet achieves a mean absolute calibration error of 0.297cm in translation and 0.017° in rotation with miscalibration magnitudes of up to ±1.5m and ±20° on the KITTI-odometry dataset, which is better than the state-of-the-art CNN-based calibration methods. The code will be publicly available at https://github.com/LvXudong-HIT/LCCNetKeywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Automatic extrinsic calibration for lidar-stereo vehicle sensor setups2017
- Accurate Calibration of LiDAR-Camera Systems Using Ordinary Boxes2017
- Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed ChessboardRemote Sensing, 2017
- RegNet: Multimodal sensor registration using deep neural networks2017
- Automatic Extrinsic Calibration of Vision and Lidar by Maximizing Mutual InformationJournal of Field Robotics, 2014
- Multi‐Modal Sensor Calibration Using a Gradient Orientation MeasureJournal of Field Robotics, 2014
- Calibration between Color Camera and 3D LIDAR Instruments with a Polygonal Planar BoardSensors, 2014
- Targetless Calibration of a Lidar - Perspective Camera Pair2013
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012
- Automatic camera and range sensor calibration using a single shot2012