Nighttime Pedestrian Detection Based on a Fusion of Visual Information and Millimeter-Wave Radar
Open Access
- 3 July 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 68439-68451
Abstract
A target detection algorithm based on the fusion of vision sensors and millimeter wave radar data can effectively improve the safety of self-driving vehicles. However, a single sensor cannot obtain comprehensive category status information on the target in the nighttime environment. To improve pedestrian detection in nighttime traffic scenarios, this paper proposes a nighttime pedestrian detection method based on the fusion of infrared vision information and millimeter wave (MMW) radar data. The lateral localization and category features of the target are obtained using the improved YOLOv5 deep learning algorithm, the distance and velocity information of the target is obtained by preprocessing the MMW radar acquisition data, the pedestrian target is tracked by using the extended Kalman filter for MMW radar detection, the projection of the valid radar target point on the IR image is completed according to the spatiotemporal fusion, and then the correlation gate method is used to correlate the data, and the visual information is inherited to the radar points to get the target multimodal information, and the success associated valid target sequences are weighted to obtain the accurate target position. Finally, a decision-level fusion algorithm framework is proposed to complete the output of pedestrian multimodal information in nighttime traffic scenes. Theoretical analysis and experimental results show that the accuracy and robustness of this method for nighttime pedestrian recognition are better than those of a single sensor.Keywords
This publication has 31 references indexed in Scilit:
- Ship detection in spaceborne infrared images based on Convolutional Neural Networks and synthetic targetsInfrared Physics & Technology, 2019
- Real-Time Ground Vehicle Detection in Aerial Infrared Imagery Based on Convolutional Neural NetworkElectronics, 2018
- Focal Loss for Dense Object Detection2017
- YOLO9000: Better, Faster, Stronger2017
- Automotive radars: A review of signal processing techniquesIEEE Signal Processing Magazine, 2017
- SSD: Single Shot MultiBox Detector2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- Fast R-CNN2015
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation2014
- Asymptotic behavior of the extended Kalman filter as a parameter estimator for linear systemsIEEE Transactions on Automatic Control, 1979