Adaptation of context-dependent deep neural networks for automatic speech recognition
Authors:
- 1 December 2012
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 366-369
Abstract
In this paper, we evaluate the effectiveness of adaptation methods for context-dependent deep-neural-network hidden Markov models (CD-DNN-HMMs) for automatic speech recognition. We investigate the affine transformation and several of its variants for adapting the top hidden layer. We compare the affine transformations against direct adaptation of the softmax layer weights. The feature-space discriminative linear regression (fDLR) method with the affine transformations on the input layer is also evaluated. On a large vocabulary speech recognition task, a stochastic gradient ascent implementation of the fDLR and the top hidden layer adaptation is shown to reduce word error rates (WERs) by 17% and 14%, respectively, compared to the baseline DNN performances. With a batch update implementation, the softmax layer adaptation technique reduces WERs by 10%. We observe that using bias shift performs as well as doing scaling plus bias shift.Keywords
This publication has 12 references indexed in Scilit:
- Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research GroupsIEEE Signal Processing Magazine, 2012
- Hermitian based hidden activation functions for adaptation of hybrid HMM/ANN models2012
- Feature engineering in Context-Dependent Deep Neural Networks for conversational speech transcription2011
- Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech RecognitionIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Acoustic Modeling Using Deep Belief NetworksIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Equivalence of Generative and Log-Linear ModelsIEEE Transactions on Audio, Speech, and Language Processing, 2010
- Comparison of discriminative input and output transformations for speaker adaptation in the hybrid NN/HMM systems2010
- Adaptation of Hybrid ANN/HMM Models Using Linear Hidden Transformations and Conservative Training2006
- Training Products of Experts by Minimizing Contrastive DivergenceNeural Computation, 2002
- Maximum likelihood linear transformations for HMM-based speech recognitionComputer Speech & Language, 1998