Deep Neural Network Testing: A Comprehensive Survey

Authors:
Preprint
Publisher website
Google Scholar
Abstract
Deep neural networks (DNNs) have been widely used in many different fields (including for medical diagnosis and autonomous driving). A challenge, however, is that internal defects may cause incorrect DNN behavior, potentially resulting in financial losses and even posing safety threats: DNN reliability and safety require comprehensive, in-depth testing and evaluation. Since DNN testing research began in 2017, various methods and tools have been proposed. This paper systematically surveys advances in DNN testing over the past 9 years, covering topics such as key technologies for testing objectives, noise-sample construction, test-case generation, addressing the test-oracle problem in DNN testing, test datasets, and evaluation metrics. We also examine open-source testing tools, and analyze typical applications and domains for DNN testing. Finally, we provide an in-depth discussion of the challenges faced in DNN testing, and discuss potential future work directions.