A Fast Learning Method for Multilayer Perceptrons in Automatic Speech Recognition Systems
Open Access
- 23 February 2015
- journal article
- research article
- Published by Wiley in Journal of Robotics
- Vol. 2015, 1-7
Abstract
We propose a fast learning method for multilayer perceptrons (MLPs) on large vocabulary continuous speech recognition (LVCSR) tasks. A preadjusting strategy based on separation of training data and dynamic learning-rate with a cosine function is used to increase the accuracy of a stochastic initial MLP. Weight matrices of the preadjusted MLP are restructured by a method based on singular value decomposition (SVD), reducing the dimensionality of the MLP. A back propagation (BP) algorithm that fits the unfolded weight matrices is used to train the restructured MLP, reducing the time complexity of the learning process. Experimental results indicate that on LVCSR tasks, in comparison with the conventional learning method, this fast learning method can achieve a speedup of around 2.0 times with improvement on both the cross entropy loss and the frame accuracy. Moreover, it can achieve a speedup of approximately 3.5 times with only a little loss of the cross entropy loss and the frame accuracy. Since this method consumes less time and space than the conventional method, it is more suitable for robots which have limitations on hardware.Keywords
This publication has 11 references indexed in Scilit:
- The Deep Tensor Neural Network With Applications to Large Vocabulary Speech RecognitionIEEE Transactions on Audio, Speech, and Language Processing, 2012
- Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research GroupsIEEE Signal Processing Magazine, 2012
- Training Deep and Recurrent Networks with Hessian-Free Optimization2012
- Introduction to the Special Section on Deep Learning for Speech and Language ProcessingIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech RecognitionIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Deep and Wide: Multiple Layers in Automatic Speech RecognitionIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Acoustic Modeling Using Deep Belief NetworksIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Visual recognition of pointing gestures for human–robot interactionImage and Vision Computing, 2007
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- The singular value decomposition: Its computation and some applicationsIEEE Transactions on Automatic Control, 1980