Concurrent Classifier Error Detection (CCED) in Large Scale Machine Learning Systems
- 31 January 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Reliability
- Vol. 73 (4) , 1782-1791
Abstract
The complexity of machine learning (ML) systems increases each year. As these systems are widely utilized, ensuring their reliable operation is becoming a design requirement. Traditional error detection mechanisms introduce circuit or time redundancy that significantly impacts system performance. An alternative is the use of concurrent error detection (CED) schemes that operate in parallel with the system and exploit their properties to detect errors. CED is attractive for large ML systems because it can potentially reduce the cost of error detection. In this article, we introduce concurrent classifier error detection (CCED), a scheme to implement CED in ML systems using a concurrent ML classifier to detect errors. CCED identifies a set of check signals in the main ML system and feed them to the concurrent ML classifier that is trained to detect errors. The proposed CCED scheme has been implemented and evaluated on two widely used large-scale ML models: Contrastive language-image pretraining (CLIP) used for image classification and bidirectional encoder representations from transformers (BERT) used for natural language applications. The results show that more than 95% of the errors are detected when using a simple Random Forest classifier that is orders of magnitude simpler than CLIP or BERT.Keywords
All Related Versions
Funding Information
- FUN4DATE (PID2022-136684OB-C22)
- 6G-INTEGRATION-3 (TSI-063000-2021-127)
- Spanish Agencia Estatal de Investigación
- Natural Science Funds of China (62171313)
This publication has 31 references indexed in Scilit:
- Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and PerspectivesIEEE Design & Test, 2023
- MOZART+: Masking Outputs With Zeros for Improved Architectural Robustness and Testing of DNN AcceleratorsIEEE Transactions on Device and Materials Reliability, 2022
- Improving Fault Tolerance for Reliable DNN Using Boundary-Aware ActivationIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2021
- Applying Lightweight Soft Error Mitigation Techniques to Embedded Mixed Precision Deep Neural NetworksIEEE Transactions on Circuits and Systems I: Regular Papers, 2021
- Selective Neuron Re-Computation (SNRC) for Error-Tolerant Neural NetworksIEEE Transactions on Computers, 2021
- AI and Reliability Trends in Safety-Critical Autonomous Systems on Ground and Air2020
- On the Evaluation Measures for Machine Learning Algorithms for Safety-Critical Systems2019
- Generative Adversarial Networks in AI-Enabled Safety-Critical Systems: Friend or Foe?Computer, 2019
- Bag of Tricks for Image Classification with Convolutional Neural Networks2019
- Deep Residual Learning for Image Recognition2016