Enhancing Autonomous Driving Perception: A Practical Approach to Event-Based Object Detection in CARLA and ROS
Open Access
Abstract
Robust object detection in autonomous driving is challenged by inherent limitations of conventional frame-based cameras, such as motion blur and limited dynamic range. In contrast, event-based cameras, which operate asynchronously and capture rapid changes with high temporal resolution and expansive dynamic range, offer a promising augmentation. While the previous research on event-based object detection has predominantly focused on algorithmic enhancements via advanced preprocessing and network optimizations to improve detection accuracy, the practical engineering and integration challenges of deploying these sensors in real-world systems remain underexplored. To address this gap, our study investigates the integration of event-based cameras as a complementary sensor modality in autonomous driving. We adapted a conventional frame-based detection model (YOLOv8) for event-based inputs by training it on the GEN1 dataset, achieving a mean average precision (mAP) of 70.1%, a significant improvement over previous benchmarks. Additionally, we developed a real-time object detection pipeline optimized for event-based data, integrating it into the CARLA simulation environment and ROS for system prototyping. The model was further refined using transfer learning to better adapt to simulation conditions, and the complete pipeline was validated across diverse simulated scenarios to address practical challenges. These results underscore the feasibility of incorporating event cameras into existing perception systems, paving the way for their broader deployment in autonomous vehicle applications.Keywords
This publication has 23 references indexed in Scilit:
- Safety validation for connected autonomous vehicles using large-scale testing tracks in high-fidelity simulation environmentAccident Analysis & Prevention, 2025
- Event-Based Spiking Neural Networks for Object Detection: A Review of Datasets, Architectures, Learning Rules, and ImplementationIEEE Access, 2024
- Low-latency automotive vision with event camerasNature, 2024
- Vision-based environmental perception for autonomous drivingProceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 2023
- Asynchronous Spatio-Temporal Memory Network for Continuous Event-Based Object DetectionIEEE Transactions on Image Processing, 2022
- DSEC: A Stereo Event Camera Dataset for Driving ScenariosIEEE Robotics and Automation Letters, 2021
- Self-driving cars: A surveyExpert Systems with Applications, 2021
- Event-Based Vision: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
- Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and PerceptionIEEE Signal Processing Magazine, 2020
- Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior AnalysisIEEE Transactions on Intelligent Transportation Systems, 2013