On Security Weaknesses and Vulnerabilities in Deep Learning Systems
- 17 October 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 22 (3) , 2243-2257
Abstract
The security guarantee of AI-enabled software systems (particularly using deep learning techniques as a functional core) is pivotal against the adversarial attacks exploiting software vulnerabilities. However, little attention has been paid to a systematic investigation of vulnerabilities in such systems. A common situation learned from the open source software community is that deep learning engineers frequently integrate off-the-shelf or open-source learning frameworks into their ecosystems. In this work, we specifically look into deep learning (DL) framework and perform the first systematic study of vulnerabilities in DL systems through a comprehensive analysis of identified vulnerabilities from Common Vulnerabilities and Exposures (CVE) and open-source DL tools, including TensorFlow, Caffe, OpenCV, Keras, and PyTorch. We propose a two-stream data analysis framework to explore vulnerability patterns from various databases. We investigate the unique DL frameworks and libraries development ecosystems that appear to be decentralized and fragmented. By revisiting the Common Weakness Enumeration (CWE) List, which provides the traditional software vulnerability related practices, we observed that it is more challenging to detect and fix the vulnerabilities throughout the DL systems lifecycle. Moreover, we conducted a large-scale empirical study of DL vulnerabilities to better understand the patterns of vulnerability and the challenges in fixing them. We have released the full replication package at https://github.com/codelzz/Vulnerabilities4DLSystem . We anticipate that our study can advance the development of secure DL systems.Keywords
All Related Versions
This publication has 45 references indexed in Scilit:
- Software vulnerabilities in TensorFlow-based deep learning applicationsComputers & Security, 2022
- A software engineering perspective on engineering machine learning systems: State of the art and challengesJournal of Systems and Software, 2021
- Unveiling the Mystery of API Evolution in Deep Learning Frameworks: A Case Study of Tensorflow 22021
- Are Machine Learning Cloud APIs Used Correctly?2021
- Architecting AI Deployment: A Systematic Review of State-of-the-Art and State-of-Practice Literature2021
- An Empirical Study on Bugs Inside TensorFlow2020
- A Study of Oracle Approximations in Testing Deep Learning Libraries2019
- Understanding Development Process of Machine Learning Systems: Challenges and Solutions2019
- CRADLE: Cross-Backend Validation to Detect and Localize Bugs in Deep Learning Libraries2019
- Testing and validating machine learning classifiers by metamorphic testingJournal of Systems and Software, 2011