Fixed-point performance analysis of recurrent neural networks
Authors:
- 1 March 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 976-980
Abstract
Recurrent neural networks have shown excellent performance in many applications; however they require increased complexity in hardware or software based implementations. The hardware complexity can be much lowered by minimizing the word-length of weights and signals. This work analyzes the fixed-point performance of recurrent neural networks using a retrain based quantization method. The quantization sensitivity of each layer in RNNs is studied, and the overall fixed-point optimization results minimizing the capacity of weights while not sacrificing the performance are presented. A language model and a phoneme recognition examples are used.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Long Short-Term MemoryNeural Computation, 1997
- Simulation-based word-length optimization method for fixed-point digital signal processing systemsIEEE Transactions on Signal Processing, 1995
- Finite precision error analysis of neural network hardware implementationsIEEE Transactions on Computers, 1993
- Multilayer feedforward neural networks with single powers-of-two weightsIEEE Transactions on Signal Processing, 1993
- A method for self-determination of adaptive learning rates in back propagationNeural Networks, 1991
- Finding Structure in TimeCognitive Science, 1990
- Backpropagation through time: what it does and how to do itProceedings of the IEEE, 1990
- Increased rates of convergence through learning rate adaptationNeural Networks, 1988