RegNet: Multimodal sensor registration using deep neural networks
Authors:
Top Cited Papers
- 1 June 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1803-1810
Abstract
In this paper, we present RegNet, the first deep convolutional neural network (CNN) to infer a 6 degrees of freedom (DOF) extrinsic calibration between multimodal sensors, exemplified using a scanning LiDAR and a monocular camera. Compared to existing approaches, RegNet casts all three conventional calibration steps (feature extraction, feature matching and global regression) into a single real-time capable CNN. Our method does not require any human interaction and bridges the gap between classical offline and target-less online calibration approaches as it provides both a stable initial estimation as well as a continuous online correction of the extrinsic parameters. During training we randomly decalibrate our system in order to train RegNet to infer the correspondence between projected depth measurements and RGB image and finally regress the extrinsic calibration. Additionally, with an iterative execution of multiple CNNs, that are trained on different magnitudes of decalibration, our approach compares favorably to state-of-the-art methods in terms of a mean calibration error of 0.28° for the rotational and 6 cm for the translation components even for large decalibrations up to 1.5 m and 20°.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation2016
- PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization2015
- SUN RGB-D: A RGB-D scene understanding benchmark suite2015
- Automatic Extrinsic Calibration of Vision and Lidar by Maximizing Mutual InformationJournal of Field Robotics, 2014
- Targetless Calibration of a Lidar - Perspective Camera Pair2013
- Vision meets robotics: The KITTI datasetThe International Journal of Robotics Research, 2013
- Automatic Online Calibration of Cameras and Lasers2013
- Automatic camera and range sensor calibration using a single shot2012
- 3D LIDAR–camera intrinsic and extrinsic calibration: Identifiability and analytical least-squares-based initializationThe International Journal of Robotics Research, 2012
- Fully automatic calibration of LIDAR and video streams from a vehicle2009