Selective Neuron Re-Computation (SNRC) for Error-Tolerant Neural Networks
- 12 February 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 71 (3) , 684-695
Abstract
Artificial Neural networks (ANNs) are widely used to solve classification problems for many machine learning applications. When errors occur in the computational units of an ANN implementation due to for example radiation effects, the result of an arithmetic operation can be changed, and therefore, the predicted classification class may be erroneously affected. This is not acceptable when ANNs are used in many safety-critical applications, because the incorrect classification may result in a system failure. Existing error-tolerant techniques usually rely on physically replicating parts of the ANN implementation or incurring in a significant computation overhead. Therefore, efficient protection schemes are needed for ANNs that are run on a processor and used in resource-limited platforms. A technique referred to as Selective Neuron Re-Computation (SNRC), is proposed in this paper. As per the ANN structure and algorithmic properties, SNRC can identify the cases in which the errors have no impact on the outcome; therefore, errors only need to be handled by re-computation when the classification result is detected as unreliable. Compared with existing temporal redundancy-based protection schemes, SNRC saves more than 60 percent of the re-computation (more than 90 percent in many cases) overhead to achieve complete error protection as assessed over a wide range of datasets. Different activation functions are also evaluated.Keywords
Funding Information
- National Science Foundation (CCF-1953961, 1812467)
- ACHILLES (PID2019-104207RB-I00)
- Go2Edge (RED2018-102585-T)
- Ministerio de Ciencia e Innovación
- Madrid Community research (TAPIR-CM P2018/TCS-4496)
This publication has 39 references indexed in Scilit:
- Failure analysis of parameter-induced simulation crashes in climate modelsEGUsphere, 2013
- A Quantitative Comparison of Different MLP Activation Functions in Classification2006
- The discovery of experts' decision rules from qualitative bankruptcy data using genetic algorithmsExpert Systems with Applications, 2003
- A class of random multiple bits in a byte error correcting and single byte error detecting (S/sub t/bEC-S/sub b/ED) codesIEEE Transactions on Computers, 2003
- Improving the tolerance of multilayer perceptrons by minimizing the statistical sensitivity to weight deviationsNeurocomputing, 2000
- On Fault Tolerant Training of Feedforward Neural NetworksNeural Networks, 1997
- Complete and partial fault tolerance of feedforward neural netsIEEE Transactions on Neural Networks, 1995
- Determining and improving the fault tolerance of multilayer perceptrons in a pattern-recognition applicationIEEE Transactions on Neural Networks, 1993
- Analysis of hidden units in a layered network trained to classify sonar targetsNeural Networks, 1988
- The perceptron: A probabilistic model for information storage and organization in the brain.Psychological Review, 1958