Defensive Perception: Estimation and Monitoring of Neural Network Performance under Deployment
Authors:
Preprint
Abstract
In this paper, we propose a method for addressing the issue of unnoticed catastrophic deployment and domain shift in neural networks for semantic segmentation in autonomous driving. Our approach is based on the idea that deep learning-based perception for autonomous driving is uncertain and best represented as a probability distribution. As autonomous vehicles' safety is paramount, it is crucial for perception systems to recognize when the vehicle is leaving its operational design domain, anticipate hazardous uncertainty, and reduce the performance of the perception system. To address this, we propose to encapsulate the neural network under deployment within an uncertainty estimation envelope that is based on the epistemic uncertainty estimation through the Monte Carlo Dropout approach. This approach does not require modification of the deployed neural network and guarantees expected model performance. Our defensive perception envelope has the capability to estimate a neural network's performance, enabling monitoring and notification of entering domains of reduced neural network performance under deployment. Furthermore, our envelope is extended by novel methods to improve the application in deployment settings, including reducing compute expenses and confining estimation noise. Finally, we demonstrate the applicability of our method for multiple different potential deployment shifts relevant to autonomous driving, such as transitions into the night, rainy, or snowy domain. Overall, our approach shows great potential for application in deployment settings and enables operational design domain recognition via uncertainty, which allows for defensive perception, safe state triggers, warning notifications, and feedback for testing or development and adaptation of the perception stack.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- A survey of uncertainty in deep neural networksArtificial Intelligence Review, 2023
- BackboneAnalysis: Structured Insights into Compute Platforms from CNN Inference Latency2022
- Explainable neural networks that simulate reasoningNature Computational Science, 2021
- Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methodsMachine Learning, 2021
- Array programming with NumPyNature, 2020
- Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities2020
- Domain is of the Essence: Data Deployment for City-Scale Multi-Camera Vehicle Re-Identification2019
- Driver error or designer error: Using the Perceptual Cycle Model to explore the circumstances surrounding the fatal Tesla crash on 7th May 2016Safety Science, 2018
- A Survey of Scene Understanding by Event Reasoning in Autonomous DrivingInternational Journal of Automation and Computing, 2018
- The Proof and Measurement of Association between Two ThingsThe American Journal of Psychology, 1904