Model and Method for Providing Resilience to Resource-Constrained AI-System
Open Access
Abstract
Artificial intelligence technologies are becoming increasingly prevalent in resource-constrained, safety-critical embedded systems. Numerous methods exist to enhance the resilience of AI systems against disruptive influences. However, when resources are limited, ensuring cost-effective resilience becomes crucial. A promising approach for reducing the resource consumption of AI systems during test-time involves applying the concepts and methods of dynamic neural networks. Nevertheless, the resilience of dynamic neural networks against various disturbances remains underexplored. This paper proposes a model architecture and training method that integrate dynamic neural networks with a focus on resilience. Compared to conventional training methods, the proposed approach yields a 24% increase in the resilience of convolutional networks and a 19.7% increase in the resilience of visual transformers under fault injections. Additionally, it results in a 16.9% increase in the resilience of convolutional network ResNet-110 and a 21.6% increase in the resilience of visual transformer DeiT-S under adversarial attacks, while saving more than 30% of computational resources. Meta-training the neural network model improves resilience to task changes by an average of 22%, while achieving the same level of resource savings.Keywords
This publication has 36 references indexed in Scilit:
- Adversarial robustness improvement for deep neural networksMachine Vision and Applications, 2024
- Multi-Exit Vision Transformer with Custom Fine-Tuning for Fine-Grained Image Recognition2023
- Dynamic Neural Network is All You Need: Understanding the Robustness of Dynamic Mechanisms in Neural Networks2023
- I3D: Transformer Architectures with Input-Dependent Dynamic Depth for Speech Recognition2023
- Parameter-efficient fine-tuning of large-scale pre-trained language modelsNature Machine Intelligence, 2023
- Deep neural networks compression: A comparative survey and choice recommendationsNeurocomputing, 2022
- AdaViT: Adaptive Vision Transformers for Efficient Image Recognition2022
- Adversarial attack vulnerability of medical image analysis systems: Unexplored factorsMedical Image Analysis, 2021
- Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPSIEEE Communications Surveys & Tutorials, 2020
- Convolutional Networks with Adaptive Inference Graphs2018