Memristive Quantized Neural Networks: A Novel Approach to Accelerate Deep Learning On-Chip
- 3 May 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 51 (4) , 1875-1887
Abstract
Existing deep neural networks (DNNs) are computationally expensive and memory intensive, which hinder their further deployment in novel nanoscale devices and applications with lower memory resources or strict latency requirements. In this paper, a novel approach to accelerate on-chip learning systems using memristive quantized neural networks (M-QNNs) is presented. A real problem of multilevel memristive synaptic weights due to device-to-device (D2D) and cycle-to-cycle (C2C) variations is considered. Different levels of Gaussian noise are added to the memristive model during each adjustment. Another method of using memristors with binary states to build M-QNNs is presented, which suffers from fewer D2D and C2C variations compared with using multilevel memristors. Furthermore, methods of solving the sneak path issues in the memristive crossbar arrays are proposed. The M-QNN approach is evaluated on two image classification datasets, that is, ten-digit number and handwritten images of mixed National Institute of Standards and Technology (MNIST). In addition, input images with different levels of zero-mean Gaussian noise are tested to verify the robustness of the proposed method. Another highlight of the proposed method is that it can significantly reduce computational time and memory during the process of image recognition.Keywords
Funding Information
- National Natural Science Foundation of China (61806129, 61672357, U1713214, 61673188, 61761130081)
- Science and technology project of Guangdong Province (2018A050501014)
- National Key Research and Development Program of China (2016YFB0800402)
- Foundation for Innovative Research Groups of Hubei Province (2017CFA005)
- China Post-Doctoral Science Foundation (2018M640820, 2018M633132)
This publication has 48 references indexed in Scilit:
- A Novel Design for a Memristor-Based or GateIEEE Transactions on Circuits and Systems II: Express Briefs, 2015
- A Novel Design for Memristor-Based Logic Switch and Crossbar CircuitsIEEE Transactions on Circuits and Systems I: Regular Papers, 2015
- Memristor-Based Multilayer Neural Networks With Online Gradient Descent TrainingIEEE Transactions on Neural Networks and Learning Systems, 2015
- A Novel Nondestructive Read/Write Circuit for Memristor-Based Memory ArraysIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2014
- Neuromorphic crossbar circuit with nanoscale filamentary-switching binary memristors for speech recognitionDiscover Nano, 2014
- A Circuit-Based Learning Architecture for Multilayer Neural Networks With Memristor Bridge SynapsesIEEE Transactions on Circuits and Systems I: Regular Papers, 2014
- Neuromorphic Hardware System for Visual Pattern Recognition With Memristor Array and CMOS NeuronIEEE Transactions on Industrial Electronics, 2014
- Memristor-based memory: The sneak paths problem and solutionsMicroelectronics Journal, 2013
- ‘Memristive’ switches enable ‘stateful’ logic operations via material implicationNature, 2010
- The missing memristor foundNature, 2008