A Flexible Memory Controller Supporting Deep Belief Networks with Fixed-Point Arithmetic
Authors:
- Jingfei JiangJJJingfei Jiang
- Rongdong HuRHRongdong Hu
- Mikel LujanMLMikel Lujan
- 1 May 2013
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 144-152
Abstract
Deep Belief Networks (DBNs) are state-of-art Machine Learning techniques and one of the most important unsupervised learning algorithms. Training DBNs is computationally intensive which naturally leads to investigate FPGA acceleration. Fixed-point arithmetic can have an important influence on the execution time and prediction accuracy of a DBN. Previous studies have focused only on customized DBN accelerators with a fixed data-width. Our results experiments demonstrate that supporting various data-widths in different DBN configurations and application environments does make sense for achieving acceptable performance. From this we conclude that a DBN accelerator should support various data-widths rather than the fixed one as done in previous work. The processing performance of DBN accelerators in FPGA is almost always constrained not by the capacity of the processing units, but by their on-chip RAM capacity and speed. We propose an efficient memory controller for DBN accelerators, which shows that supporting various data-widths is not as difficult as it may sound. The cost is only little in hardware terms and does not affect the critical path. We have designed a tool to help users reconfiguring the memory controller with arbitrary data-width flexibly.Keywords
This publication has 15 references indexed in Scilit:
- Resource Efficient Arithmetic Effects on RBM Neural Network Solution Quality Using MNIST2011
- Development and Implementation of Parameterized FPGA-Based General Purpose Neural Networks for Online ApplicationsIEEE Transactions on Industrial Informatics, 2010
- Extracting and composing robust features with denoising autoencoders2008
- The Impact of Arithmetic Representation on Implementing MLP-BP on FPGAs: A StudyIEEE Transactions on Neural Networks, 2007
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- On the capabilities of neural networks using limited precision weightsNeural Networks, 2002
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Piecewise linear approximation applied to nonlinear function of a neural networkIEE Proceedings - Circuits, Devices and Systems, 1997
- Finite precision error analysis of neural network hardware implementationsIEEE Transactions on Computers, 1993