Layer ensemble averaging for fault tolerance in memristive neural networks
Open Access
- 1 February 2025
- journal article
- research article
- Published by Springer Nature in Nature Communications
- Vol. 16 (1) , 1-14
Abstract
Artificial neural networks have advanced due to scaling dimensions, but conventional computing struggles with inefficiencies due to memory bottlenecks. In-memory computing architectures using memristor devices offer promise but face challenges due to hardware non-idealities. This work proposes layer ensemble averaging—a hardware-oriented fault tolerance scheme for improving inference performance of non-ideal memristive neural networks programmed with pre-trained solutions. Simulations on an image classification task and hardware experiments on a continual learning problem with a custom 20,000-device prototyping platform show significant performance gains, outperforming prior methods at similar redundancy levels and overheads. For the image classification task with 20% stuck-at faults, accuracy improves from 40% to 89.6% (within 5% of baseline), and for the continual learning problem, accuracy improves from 55% to 71% (within 1% of baseline). The proposed scheme is broadly applicable to accelerators based on a variety of different non-volatile device technologies.Keywords
Funding Information
- United States Department of Commerce | National Institute of Standards and Technology (70NANB22H018)
- Western Digital (ECNS21932N)
- National Science Foundation (2239951)
- United States Department of Defense | United States Air Force | AFMC | Air Force Office of Scientific Research (FA9550-23-1-0173)
- George Washington University (Cross-Disciplinary Research Fund)
This publication has 51 references indexed in Scilit:
- An ultrasmall organic synapse for neuromorphic computingNature Communications, 2023
- Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networksNature Communications, 2023
- A large-scale integrated vector–matrix multiplication processor based on monolayer molybdenum disulfide memoriesNature Electronics, 2023
- A compute-in-memory chip based on resistive random-access memoryNature, 2022
- Mixed‐Precision Continual Learning Based on Computational Resistance Random Access MemoryAdvanced Intelligent Systems, 2022
- Committee machines—a universal method to deal with non-idealities in memristor-based neural networksNature Communications, 2020
- Fault tolerance in memristive crossbar-based neuromorphic computing systemsIntegration, 2020
- A fully integrated reprogrammable memristor–CMOS system for efficient multiply–accumulate operationsNature Electronics, 2019
- Memristor‐Based Analog Computation and Neural Network Classification with a Dot Product EngineAdvanced Materials, 2018
- Analogue signal and image processing with large memristor crossbarsNature Electronics, 2017