How to Evaluate Proving Grounds for Self-Driving? A Quantitative Approach
- 15 May 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 22 (9) , 5737-5748
Abstract
Proving ground has been a critical component in testing and validation for Connected and Automated Vehicles (CAV). Although quite a few world-class testing facilities have been under construction over the years, the evaluation of proving grounds themselves as testing approaches has rarely been studied. In this paper, we present the first attempt to systematically evaluate CAV proving grounds and contribute to a generative sample-based approach to assessing the representation of traffic scenarios in proving grounds. Leveraging typical use cases extracted from naturalistic driving events, we establish a strong link between proving ground testing results of CAVs and their anticipated public street performance. We present benchmark results of our approach on three world-class CAV testing facilities: Mcity, Almono (Uber ATG), and Kcity. We successfully show the overall evaluation of these proving grounds in terms of their capability to accommodate real-world traffic scenarios. We believe that when the effectiveness of a testing ground itself is validated, the testing results would grant more confidence for CAV public deployment.Keywords
All Related Versions
Funding Information
- National Science Foundation (IIS-1849304)
- Carnegie Mellon University’s Mobility21 National University Transportation Center
This publication has 27 references indexed in Scilit:
- Accelerated Evaluation of Automated Vehicles in Lane Change Scenarios2015
- An Integral Approach to Autonomous and Cooperative Vehicles Development and Testing2015
- Task-Specific Performance Evaluation of UGVs: Case Studies at the IVFCIEEE Transactions on Intelligent Transportation Systems, 2014
- Searching and mining trillions of time series subsequences under dynamic time warping2012
- V2X simulation runtime infrastructure VSimRTI: An assessment tool to design smart traffic management systemsComputer Networks, 2011
- Querying and mining of time series dataProceedings of the VLDB Endowment, 2008
- Tracking human motion using auxiliary particle filters and iterated likelihood weightingImage and Vision Computing, 2006
- Monte Carlo localization for mobile robots2003
- Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space ModelsJournal of Computational and Graphical Statistics, 1996
- Novel approach to nonlinear/non-Gaussian Bayesian state estimationIEE Proceedings F Radar and Signal Processing, 1993