Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI
Authors:
- 7 April 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 135-141
Abstract
Recent research demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vector multiplication-the intensive and key computation in deep neural networks (DNNs). However, hardware failure, such as stuck-at-fault defects, is one of the main concerns that impedes the ReRAM devices to be a feasible solution for real implementations. The existing solutions to address this issue usually require an optimization to be conducted for each individual device, which is impractical for mass-produced products (e.g., IoT devices). In this paper, we rethink the value of weight pruning in ReRAM-based DNN design from the perspective of model fault tolerance. And a differential mapping scheme is proposed to improve the fault tolerance under a high stuck-on fault rate. Our method can tolerate almost an order of magnitude higher failure rate than the traditional two-column method in representative DNN tasks. More importantly, our method does not require extra hardware cost compared to the traditional two-column mapping scheme. The improvement is universal and does not require the optimization process for each individual device.Keywords
All Related Versions
Funding Information
- National Science Foundation
This publication has 26 references indexed in Scilit:
- C ir CNN2017
- Memristor crossbar-based ultra-efficient next-generation baseband processors2017
- Rescuing Memristor-based Neuromorphic Design with High Defects2017
- Fault-Tolerant Training with On-Line Fault Detection for RRAM-Based Neural Computing Systems2017
- Accelerator-friendly neural-network training: Learning variations and defects in RRAM crossbar2017
- ISAACACM SIGARCH Computer Architecture News, 2016
- PRIMEACM SIGARCH Computer Architecture News, 2016
- Defect consideratons for robust sparse coding using memristor arrays2015
- Modeling, Detection, and Diagnosis of Faults in Multilevel Memristor MemoriesIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2015
- Memristor-The missing circuit elementIEEE Transactions on Circuit Theory, 1971