Crossbar-Aware Neural Network Pruning
Open Access
- 8 October 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 6, 58324-58337
Abstract
Crossbar architecture has been widely adopted in neural network accelerators due to the efficient implementations on vector-matrix multiplication operations. However, in the case of convolutional neural networks (CNNs), the efficiency is compromised dramatically because of the large amounts of data reuse. Although some mapping methods have been designed to achieve a balance between the execution throughput and resource overhead, the resource consumption cost is still huge while maintaining the throughput. Network pruning is a promising and widely studied method to shrink the model size, whereas prior work for CNNs compression rarely considered the crossbar architecture and the corresponding mapping method and cannot be directly utilized by crossbar-based neural network accelerators. This paper proposes a crossbar-aware pruning framework based on a formulated $L_{0}$ -norm constrained optimization problem. Specifically, we design an $L_{0}$ -norm constrained gradient descent with relaxant probabilistic projection to solve this problem. Two types of sparsity are successfully achieved: 1) intuitive crossbar-grain sparsity and 2) column-grain sparsity with output recombination, based on which we further propose an input feature maps reorder method to improve the model accuracy. We evaluate our crossbar-aware pruning framework on the median-scale CIFAR10 data set and the large-scale ImageNet data set with VGG and ResNet models. Our method is able to reduce the crossbar overhead by 44%–72% with insignificant accuracy degradation. This paper significantly reduce the resource overhead and the related energy cost and provides a new co-design solution for mapping CNNs onto various crossbar devices with much better efficiency.Keywords
All Related Versions
Funding Information
- National Science Foundation (1719160, 1725447, 1730309)
- National Natural Science Foundation of China (61603209, 61876215)
This publication has 29 references indexed in Scilit:
- ReRAM Crossbar based Recurrent Neural Network for human activity detection2016
- Convolutional networks for fast, energy-efficient neuromorphic computingProceedings of the National Academy of Sciences, 2016
- ISAACACM SIGARCH Computer Architecture News, 2016
- PRIMEACM SIGARCH Computer Architecture News, 2016
- Deep Residual Learning for Image Recognition2016
- TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic ChipIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2015
- Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight ElementIEEE Transactions on Electron Devices, 2015
- Complex Learning in Bio-plausible Memristive NetworksScientific Reports, 2015
- A million spiking-neuron integrated circuit with a scalable communication network and interfaceScience, 2014
- Hardware realization of BSB recall function using memristor crossbar arrays2012