ExPAN(N)D: Exploring Posits for Efficient Artificial Neural Network Design in FPGA-Based Systems
Open Access
- 20 July 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 9, 103691-103708
Abstract
The high computational complexity, memory footprints, and energy requirements of machine learning models, such as Artificial Neural Networks (ANNs), hinder their deployment on resource-constrained embedded systems. Most state-of-the-art works have considered this problem by proposing various low bit-width data representation schemes and optimized arithmetic operators’ implementations. To further elevate the implementation gains offered by these individual techniques, there is a need to cross-examine and combine these techniques’ unique features. This paper presents ExPAN(N)D, a framework to analyze and ingather the efficacy of the Posit number representation scheme and the efficiency of fixed-point arithmetic implementations for ANNs. The Posit scheme offers a better dynamic range and higher precision for various applications than IEEE 754 single-precision floating-point format. However, due to the dynamic nature of the various fields of the Posit scheme, the corresponding arithmetic circuits have higher critical path delay and resource requirements than the single-precision-based arithmetic units. Towards this end, we propose a novel Posit to fixed-point converter for enabling high-performance and energy-efficient hardware implementations for ANNs with minimal drop in the output accuracy. We also propose a modified Posit-based representation to store the trained parameters of a network. With the proposed Posit to fixed-point converter-based designs, we provide multiple design points with varying accuracy-performance trade-offs for an ANN. For instance, compared to the lowest power dissipating Posit-only accelerator design, one of our proposed designs results in 80% and 48% reduction in power dissipation and LUT utilization respectively, with marginal increase in classification error for Imagenet dataset classification using VGG-16.Keywords
All Related Versions
Funding Information
- German Research Foundation (DFG) funded Project “Runtime Reconfigurable Approximate Architecture” (380524764)
This publication has 30 references indexed in Scilit:
- Ristretto: A Framework for Empirical Study of Resource-Efficient Inference in Convolutional Neural NetworksIEEE Transactions on Neural Networks and Learning Systems, 2018
- DeMAS: An efficient design methodology for building approximate adders for FPGA-based systems2018
- Universal number posit arithmetic generator on FPGA2018
- Design of power-efficient approximate multipliers for approximate artificial neural networks2016
- XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks2016
- EyerissACM SIGARCH Computer Architecture News, 2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- New types of deep neural network learning for speech recognition and related applications: an overview2013
- Xilinx Zynq-7000 EPP: An extensible processing platform family2011