Doppelgänger Test Generation for Revealing Bugs in Autonomous Driving Software
- 1 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2591-2603
Abstract
Vehicles controlled by autonomous driving software (ADS) are expected to bring many social and economic benefits, but at the current stage not being broadly used due to concerns with regard to their safety. Virtual tests, where autonomous vehicles are tested in software simulation, are common practices because they are more efficient and safer compared to field operational tests. Specifically, search-based approaches are used to find particularly critical situations. These approaches provide an opportunity to automatically generate tests; however, system-atically producing bug-revealing tests for ADS remains a major challenge. To address this challenge, we introduce DoppelTest, a test generation approach for ADSes that utilizes a genetic algorithm to discover bug-revealing violations by generating scenarios with multiple autonomous vehicles that account for traffic control (e.g., traffic signals and stop signs). Our extensive evaluation shows that DoppelTest can efficiently discover 123 bug-revealing violations for a production-grade ADS (Baidu Apollo) which we then classify into 8 unique bug categories.Keywords
Funding Information
- NSF (1823262,1929771,1932464,2145493)
This publication has 40 references indexed in Scilit:
- Automatically testing self-driving cars with search-based procedural content generation2019
- DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems2018
- Testing vision-based control systems using learnable evolutionary algorithms2018
- Autoware on Board: Enabling Autonomous Vehicles with Embedded Systems2018
- Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?Transportation Research Part A: Policy and Practice, 2016
- Testing advanced driver assistance systems using multi-objective search and neural networks2016
- Intelligence Testing for Autonomous Vehicles: A New ApproachIEEE Transactions on Intelligent Vehicles, 2016
- An Open Approach to Autonomous VehiclesIEEE Micro, 2015
- Towards Autonomous Cars2014
- Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approachIEEE Transactions on Evolutionary Computation, 1999