CORDIC Based Implementation of the Softmax Activation Function
Authors:
- 1 July 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-4
Abstract
Interest has been renewed in the domain of neuromorphic circuits, with the emergence of Deep Neural Networks as the leading model for various classification tasks. In this work, we present a CORDIC based implementation for one of the most important layers, the Softmax layer. The layer is particularly complex for CMOS implementations due to the requirement for multiple evaluations of the exponential function in calculating the output of this layer. We apply the CORDIC method for computing the exponent and the binary division algorithm for calculating the output. It is shown via experiments that our approach for realizing the Softmax function simultaneously out-performs existing implementations using McLaurin series approximations and Padé polynomial approximations from literature in three key metrics: timing, area, and accuracy. We also investigate an inverse form of the Softmax function that avoids the division operation. The implementation can be used to realize the Softmax activation in both ASIC and FPGA implementations of large scale neuromorphic circuitsKeywords
This publication has 3 references indexed in Scilit:
- High accuracy FPGA activation function implementation for neural networksNeurocomputing, 2017
- A Mini Review of Neuromorphic Architectures and ImplementationsIEEE Transactions on Electron Devices, 2016
- 50 Years of CORDIC: Algorithms, Architectures, and ApplicationsIEEE Transactions on Circuits and Systems I: Regular Papers, 2009