Multi-Objective Optimization of ReRAM Crossbars for Robust DNN Inferencing under Stochastic Noise
Authors:
- 1 November 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-9
Abstract
Resistive random-access memory (ReRAM) is a promising technology for designing hardware accelerators for deep neural network (DNN) inferencing. However, stochastic noise in ReRAM crossbars can degrade the DNN inferencing accuracy. We propose the design and optimization of a high-performance, area-and energy-efficient ReRAM-based hardware accelerator to achieve robust DNN inferencing in the presence of stochastic noise. We make two key technical contributions. First, we propose a stochastic-noise-aware training method, referred to as ReSNA, to improve the accuracy of DNN inferencing on ReRAM crossbars with stochastic noise. Second, we propose an information-theoretic algorithm, referred to as CF-MESMO, to identify the Pareto set of solutions to trade-off multiple objectives, including inferencing accuracy, area overhead, execution time, and energy consumption. The main challenge in this context is that executing the ReSNA method to evaluate each candidate ReRAM design is prohibitive. To address this challenge, we utilize the continuous-fidelity evaluation of ReRAM designs associated with prohibitive high computation cost by varying the number of training epochs to trade-off accuracy and cost. CF-MESMO iteratively selects the candidate ReRAM design and fidelity pair that maximizes the information gained per unit computation cost about the optimal Pareto front. Our experiments on benchmark DNNs show that the proposed algorithms efficiently uncover high-quality Pareto fronts. On average, ReSNA achieves 2.57% inferencing accuracy improvement for ResNet20 on the CIFAR-10 dataset with respect to the baseline configuration. Moreover, CF-MESMO algorithm achieves 90.91% reduction in computation cost compared to the popular multi-objective optimization algorithm NSGA-II to reach the best solution from NSGA-II.Keywords
All Related Versions
Funding Information
- NSF (CNS-1955196,CNS-1955353,IIS-1845922,OAC-1910213)
This publication has 38 references indexed in Scilit:
- PRIMEACM SIGARCH Computer Architecture News, 2016
- Taking the Human Out of the Loop: A Review of Bayesian OptimizationProceedings of the IEEE, 2015
- Impact of Cell Failure on Reliable Cross-Point Resistive Memory DesignACM Transactions on Design Automation of Electronic Systems, 2015
- Memristor-Based Multilayer Neural Networks With Online Gradient Descent TrainingIEEE Transactions on Neural Networks and Learning Systems, 2015
- Hardware realization of BSB recall function using memristor crossbar arrays2012
- Multi-level switching of triple-layered TaOx RRAM with excellent reliability for storage class memory2012
- A Simulated Annealing-Based Multiobjective Optimization Algorithm: AMOSAIEEE Transactions on Evolutionary Computation, 2008
- A fast and elitist multiobjective genetic algorithm: NSGA-IIIEEE Transactions on Evolutionary Computation, 2002
- Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approachIEEE Transactions on Evolutionary Computation, 1999
- Thermal Agitation of Electricity in ConductorsPhysical Review B, 1928