Fast vehicle detection based on colored point cloud with bird’s eye view representation
Open Access
- 8 May 2023
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 13 (1) , 1-15
Abstract
RGB cameras and LiDAR are crucial sensors for autonomous vehicles that provide complementary information for accurate detection. Recent early-level fusion-based approaches, flourishing LiDAR data with camera features, may not accomplish promising performance ascribable to the immense difference between two modalities. This paper presents a simple and effective vehicle detection method based on an early-fusion strategy, unified 2D BEV grids, and feature fusion. The proposed method first eliminates many null point clouds through cor-calibration. It augments point cloud data by color information to generate 7D colored point cloud, and unifies augmented data into 2D BEV grids. The colored BEV maps can then be fed to any 2D convolution network. A peculiar Feature Fusion (2F) detection module is utilized to extract multiple scale features from BEV images. Experiments on the KITTI public benchmark and Nuscenes dataset show that fusing RGB image with point cloud rather than raw point cloud can lead to better detection accuracy. Besides, the inference time of the proposed method reaches 0.05 s/frame thanks to its simple and compact architecture.Keywords
Funding Information
- Shanghai Nature Science Foundation of Shanghai Science and Technology 514 Commission (20ZR14379007)
- National Nature Science Foundation of China (61374197)
This publication has 40 references indexed in Scilit:
- Deep Continuous Fusion for Multi-sensor 3D Object Detection2018
- RT3D: Real-Time 3-D Vehicle Detection in LiDAR Point Cloud for Autonomous DrivingIEEE Robotics and Automation Letters, 2018
- LMNet: Real-time Multiclass Object Detection on CPU Using 3D LiDAR2018
- Fusing Bird’s Eye View LIDAR Point Cloud and Front View Camera Image for 3D Object Detection2018
- 3D fully convolutional network for vehicle detection in point cloud2017
- Feature Pyramid Networks for Object Detection2017
- PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation2017
- Multi-view 3D Object Detection Network for Autonomous Driving2017
- Deep Residual Learning for Image Recognition2016
- Vision meets robotics: The KITTI datasetThe International Journal of Robotics Research, 2013