A 41.3/26.7 pJ per Neuron Weight RBM Processor Supporting On-Chip Learning/Inference for IoT Applications
Authors:
Open Access
- 3 July 2017
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 52 (10) , 2601-2612
Abstract
An energy-efficient restricted Boltzmann machine (RBM) processor (RBM-P) supporting on-chip learning and inference is proposed for machine learning and Internet of Things (IoT) applications in this paper. To train a neural network (NN) model, the RBM structure is applied to supervised and unsupervised learning, and a multi-layer NN can be constructed and initialized by stacking multiple RBMs. Featuring NN model reduction for external memory bandwidth saving, low power neuron binarizer (LPNB) with dynamic clock gating and area-efficient NN-like activation function calculators for power reduction, user-defined connection map (UDCM) for both computation time and bandwidth saving, and early stopping (ES) mechanism for learning process, the proposed system integrates 32 RBM cores with maximal 4k neurons per layer and 128 candidates per sample for machine learning applications. Implemented in 65nm CMOS technology, the proposed RBM-P chip costs 2.2 M gates and 128 kB SRAM with 8.8 mm 2 area. Operated at 1.2 V and 210 MHz, this chip achieves 7.53G neuron weights (NWs) and 11.63G NWs per second with 41.3 and 26.7 pJ per NW for learning and inference, respectively.Keywords
Funding Information
- MOST of Taiwan (MOST 103-2221-E-009-198)
- NSF of U.S. (DMS 1330132, DMS 1407557)
This publication has 10 references indexed in Scilit:
- A Hardware-Efficient Sigmoid Function With Adjustable Precision for a Neural Network SystemIEEE Transactions on Circuits and Systems II: Express Briefs, 2015
- A 2 GOPS quad-mean shift processor with early termination for machine learning applications2014
- A 240 G-ops/s Mobile Coprocessor for Deep Neural Networks2014
- A 48.6-to-105.2 µW Machine Learning Assisted Cardiac Sensor SoC for Mobile Healthcare ApplicationsIEEE Journal of Solid-State Circuits, 2014
- Training restricted Boltzmann machines: An introductionPattern Recognition, 2014
- An adaptive FPGA implementation of multi-core K-nearest neighbour ensemble classifier using dynamic partial reconfiguration2012
- High-Performance Reconfigurable Hardware Architecture for Restricted Boltzmann MachinesIEEE Transactions on Neural Networks, 2010
- A highly scalable Restricted Boltzmann Machine FPGA implementation2009
- A Real-Time Learning Processor Based on K-means Algorithm with Automatic Seeds Generation2007
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006