Cyber Mobility Mirror: A Deep Learning-Based Real-World Object Perception Platform Using Roadside LiDAR
- 5 May 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 24 (9) , 9476-9489
Abstract
Object perception plays a fundamental role in Cooperative Driving Automation (CDA) which is regarded as a revolutionary promoter for next-generation transportation systems. However, the vehicle-based perception may suffer from the limited sensing range and occlusion as well as low penetration rates in connectivity. In this paper, we propose Cyber Mobility Mirror ( CMM ), a next-generation real-world object perception system for 3D object detection, tracking, localization, and reconstruction, to explore the potential of roadside sensors for enabling CDA in the real world. The CMM system consists of six main components: i) the data pre-processor to retrieve and preprocess the raw data; ii) the roadside 3D object detector to generate 3D detection results; iii) the multi-object tracker to identify detected objects; iv) the global locator to generate geo-localization information; v) the mobile-edge-cloud-based communicator to transmit perception information to equipped vehicles, and vi) the onboard advisor to reconstruct and display the real-time traffic conditions. An automatic perception evaluation approach is proposed to support the assessment of data-driven models without human-labeling requirements and a CMM field-operational system is deployed at a real-world intersection to assess the performance of the CMM. Results from field tests demonstrate that our CMM prototype system can achieve 96.99% precision and 83.62% recall for detection and 73.55% ID-recall for tracking. High-fidelity real-time traffic conditions (at the object level) can be geo-localized with a root-mean-square error (RMSE) of $0.69m$ and $0.33m$ for lateral and longitudinal direction, respectively, and displayed on the GUI of the equipped vehicle with a frequency of $3-4 Hz$ .Keywords
Funding Information
- Toyota Motor North America InfoTech Labs
This publication has 35 references indexed in Scilit:
- Towards Multi-Object Detection and Tracking in Urban Scenario under Uncertainties2018
- Focal Loss for Dense Object Detection2017
- Simple online and realtime tracking with a deep association metric2017
- Simple online and realtime tracking2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- A Survey of Vision-Based Traffic Monitoring of Road IntersectionsIEEE Transactions on Intelligent Transportation Systems, 2016
- Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendationsTransportation Research Part A: Policy and Practice, 2015
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012
- PATH Investigations in Vehicle-Roadside Cooperation and Safety: A Foundation for Safety and Vehicle-Infrastructure Integration Research2006
- Statistic and knowledge-based moving object detection in traffic scenes2002