FAST-LIEO: Fast and Real-Time LiDAR-Inertial-Event-Visual Odometry
- 26 December 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (2) , 1680-1687
Abstract
Unlike a standard camera that relies on exposure to obtain output frame by frame, an event camera only outputs an event when the change of brightness intensity in a pixel exceeds a threshold, and the outputs of different pixels are independent to each other. Benefited from its bio-inspired design, event camera has the advantages of low latency and high dynamic range. The researches on multi-sensor fusion with event camera are few so far. In this paper, we propose FAST-LIEO, a framework for fast and real-time LiDAR-inertial-event odometry. The framework tightly fuses LiDAR and event camera measurements without any feature extraction or matching. Besides, our system supports both LIEO and LIEVO (extended with RGB camera fusion). We design a novel EIO subsystem for LiDAR-event fusion. The EIO subsystem maintains a semi-dense event map and estimates the state by aligning the event representation to map. The semi-dense event map is built from LiDAR points by utilizing the edge information and temporal information provided by event representations. Besides testing our method on public benchmark dataset, we also collected real-world data by utilizing our sensor suite and conducted experiments on our self-captured dataset. The experiment results show the high robustness and accuracy of our method in challenging conditions with high real-time ability. To the best of our knowledge, our FAST-LIEO is the first system that can tightly fuse LiDAR, IMU, event camera and standard camera measurements in simultaneously localization and mapping.Keywords
Funding Information
- National Science Foundation of China (62203205, U1913603)
This publication has 25 references indexed in Scilit:
- R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package2022
- DEVO: Depth-Event Camera Visual Odometry in Challenging Conditions2022
- LVI-SAM: Tightly-coupled Lidar-Visual-Inertial Odometry via Smoothing and Mapping2021
- LIC-Fusion 2.0: LiDAR-Inertial-Camera Odometry with Sliding-Window Plane-Feature Tracking2020
- LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping2020
- LIC-Fusion: LiDAR-Inertial-Camera Odometry2019
- Laser–visual–inertial odometry and mapping with high robustness and low driftJournal of Field Robotics, 2018
- Event-Based Visual Inertial Odometry2017
- Integrating generic sensor fusion algorithms with sound state representations through encapsulation of manifoldsInformation Fusion, 2013
- The iterated Kalman filter update as a Gauss-Newton methodIEEE Transactions on Automatic Control, 1993