SimLane: A Risk-Orientated Benchmark for Lane Detection
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9-12
Abstract
Lane detection, which detects lanes on the road from a camera frame, plays an essential role in the successful and safe application of autonomous driving. A few previous works explore their robustness to adversarial road conditions (e.g., dirty patch [1]); however, to the best of our knowledge, no prior work systematically studies their robustness to common environmental variations, like weather, lighting, etc. Those variations will inevitably happen to autonomous vehicles worldwide, owning to the long-tail nature of driving. Therefore, it is critical to understand whether state-of-the-art lane detection models are sensitive to such environmental variations. This paper fills this research gap by collecting a dataset of variations from simulations and benchmarking state-of-the-art models against this new dataset.We first systematically investigate and categorize environmental variations that might incur negative impacts against lane detection based on existing datasets and online driving clips. We then develop a pipeline to collect driving clips under these variations from simulations. The dataset we collected, called SimLane, consists of 149 driving clips of eight different variations that could also serve as a baseline for future research on lane detection safety. Our empirical results verify that state-of-art models suffer from sensitivity to environmental variations from a minor scale to a vast scale. In particular, road graffiti with a medium dense level will cause the detection accuracy of SCNN model ([2], trained on TuSimple Dataset [3]) to drop from 0.897 to 0.219.Keywords
This publication has 7 references indexed in Scilit:
- CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point Blending2020
- LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving2020
- Lanelet2: A high-definition map framework for the future of automated driving2018
- Spatial as Deep: Spatial CNN for Traffic Scene UnderstandingProceedings of the AAAI Conference on Artificial Intelligence, 2018
- Trojaning Attack on Neural Networks2018
- Towards Evaluating the Robustness of Neural Networks2017
- Real time detection of lane markers in urban streets2008