Camera and LiDAR Fusion for On-road Vehicle Tracking with Reinforcement Learning
- 1 June 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1723-1730
Abstract
We formulate camera and LiDAR fusion tracking as a sequential decision-making process. With our deep reinforcement learning framework, we try to optimize the tracking trajectory to be as accurate, smooth, and long as possible. In contrast to traditional fusion algorithms involving complex feature and strategy design and hyperparameters tuned for different scenarios, our fusion agent can learn the confidence of each input by tracking the results from raw observation in a data-driven fashion. Given the input states of different sensors, our approach chooses one input with a higher expected cumulative reward as the observation of a Kalman filter to iteratively predict the target position. The expected cumulative reward is estimated with a convolutional neural network, trained with a modified DQN algorithm, which takes inputs from both LiDAR and a camera. Through case studies and quantitative result evaluation on our dataset from the 4th Ring Road in Beijing, our algorithm is validated to achieve more accurate and robust tracking performance.Keywords
This publication has 25 references indexed in Scilit:
- Learning to Track: Online Multi-object Tracking by Decision Making2015
- Conditional Monte Carlo Dense Occupancy Tracker2015
- Effectiveness of low speed autonomous emergency braking in real-world rear-end crashesAccident Analysis & Prevention, 2015
- Context-aware tracking of moving objects for distance keeping2015
- 360° Detection and tracking algorithm of both pedestrian and vehicle using fisheye images2015
- Human-level control through deep reinforcement learningNature, 2015
- Coordinated Adaptive Cruise Control System With Lane-Change AssistanceIEEE Transactions on Intelligent Transportation Systems, 2015
- High-Speed Tracking with Kernelized Correlation FiltersIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- A multi-sensor fusion system for moving object detection and tracking in urban driving environments2014
- Omni-directional detection and tracking of on-road vehicles using multiple horizontal laser scanners2012