Addressing Sparsity in Deep Neural Networks
- 8 August 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- Vol. 38 (10) , 1858-1871
Abstract
Neural networks (NNs) have been demonstrated to be useful in a broad range of applications, such as image recognition, automatic translation, and advertisement recommendation. State-of-the-art NNs are known to be both computationally and memory intensive, due to the ever-increasing deep structure, i.e., multiple layers with massive neurons and connections (i.e., synapses). Sparse NNs have emerged as an effective solution to reduce the amount of computation and memory required. Though existing NN accelerators are able to efficiently process dense and regular networks, they cannot benefit from the reduction of synaptic weights. In this paper, we propose a novel accelerator, Cambricon-X, to exploit the sparsity and irregularity of NN models for increased efficiency. The proposed accelerator features a processing element (PE)-based architecture consisting of multiple PEs. An indexing module efficiently selects and transfers needed neurons to connected PEs with reduced bandwidth requirement, while each PE stores irregular and compressed synapses for local computation in an asynchronous fashion. With 16 PEs, our accelerator is able to achieve at most 544 GOP/s in a small form factor (6.38 mm 2 and 954 mW at 65 nm). Experimental results over a number of representative sparse networks show that our accelerator achieves, on average, $7.23\times$ speedup and $6.43\times$ energy saving against the state-of-the-art NN accelerator. We further investigate possibilities of leveraging activation sparsity and multi-issue controller, which improve the efficiency of Cambricon-X. To ease the burden of programmers, we also propose a high efficient library-based programming environment for our accelerator.
Keywords
Funding Information
- National Basic Research Program of China (2017YFA0700902, 2017YFA0700900, 2017YFB1003101)
- National Natural Science Foundation of China (61472396, 61432016, 61473275, 61522211, 61532016, 61521092, 61502446, 61672491, 61602441, 61602446, 61732002, 61702478)
- National Basic Research Program of China (2015CB358800)
- National Science and Technology Major Project (2018ZX01031102)
- Chinese Academy of Sciences (XDBS01050200)
This publication has 34 references indexed in Scilit:
- ShiDianNao2015
- A Small-Footprint Accelerator for Large-Scale Neural NetworksACM Transactions on Computer Systems, 2015
- DianNaoACM SIGPLAN Notices, 2014
- Learning deep structured semantic models for web search using clickthrough data2013
- Mlp neural network and on-line backpropagation learning implementation in a low-cost fpga2008
- Neural Network Stream Processing Core (NnSP) for Embedded Systems2006
- FPGA Implementation of a Pipelined On-Line BackpropagationJournal of Signal Processing Systems, 2005
- An overview of the sparse basic linear algebra subprogramsACM Transactions on Mathematical Software, 2002
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Emergence of simple-cell receptive field properties by learning a sparse code for natural imagesNature, 1996