Noise Injection Adaption
Authors:
Open Access
- 2 June 2019
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Abstract
In this work, we investigate various non-ideal effects (Stuck-At-Fault (SAF), IR-drop, thermal noise, shot noise, and random telegraph noise)of ReRAM crossbar when employing it as a dot-product engine for deep neural network (DNN) acceleration. In order to examine the impacts of those non-ideal effects, we first develop a comprehensive framework called PytorX based on main-stream DNN pytorch framework. PytorX could perform end-to-end training, mapping, and evaluation for crossbar-based neural network accelerator, considering all above discussed non-ideal effects of ReRAM crossbar together. Experiments based on PytorX show that directly mapping the trained large scale DNN into crossbar without considering these non-ideal effects could lead to a complete system malfunction (i.e., equal to random guess) when the neural network goes deeper and wider. In particular, to address SAF side effects, we propose a digital SAF error correction algorithm to compensate for crossbar output errors, which only needs one-time profiling to achieve almost no system accuracy degradation. Then, to overcome IR drop effects, we propose a Noise Injection Adaption (NIA) methodology by incorporating statistics of current shift caused by IR drop in each crossbar as stochastic noise to DNN training algorithm, which could efficiently regularize DNN model to make it intrinsically adaptive to non-ideal ReRAM crossbar. It is a one-time training method without the request of retraining for every specific crossbar. Optimizing system operating frequency could easily take care of rest non-ideal effects. Various experiments on different DNNs using image recognition application are conducted to show the efficacy of our proposed methodology.Keywords
Funding Information
- National Science Foundation (1740126)
- Semiconductor Research Corporation (nCore)
This publication has 10 references indexed in Scilit:
- Stuck-at Fault Tolerance in RRAM Computing SystemsIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2017
- Accelerator-friendly neural-network training: Learning variations and defects in RRAM crossbar2017
- PRIME: A Novel Processing-in-Memory Architecture for Neural Network Computation in ReRAM-Based Main Memory2016
- A Complete Statistical Investigation of RTN in HfO2-Based RRAM in High Resistive StateIEEE Transactions on Electron Devices, 2015
- RRAM Defect Modeling and Failure Analysis Based on March Test and a Novel Squeeze-Search SchemeIEEE Transactions on Computers, 2014
- Understanding the trade-offs in multi-level cell ReRAM memory design2013
- High precision tuning of state for memristive devices by adaptable variation-tolerant algorithmNanotechnology, 2012
- Resistance-dependent amplitude of random telegraph-signal noise in resistive switching memoriesApplied Physics Letters, 2010
- Noise in nanotechnologyMicroelectronics Reliability, 2000
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998