Matrix updates for perceptron training of continuous density hidden Markov models
Authors:
Open Access
- 14 June 2009
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 153-160
Abstract
In this paper, we investigate a simple, mistake-driven learning algorithm for discriminative training of continuous density hidden Markov models (CD-HMMs). Most CD-HMMs for automatic speech recognition use multivariate Gaussian emission densities (or mixtures thereof) parameterized in terms of their means and covariance matrices. For discriminative training of CD-HMMs, we reparameterize these Gaussian distributions in terms of positive semidefinite matrices that jointly encode their mean and covariance statistics. We show how to explore the resulting parameter space in CDHMMs with perceptron-style updates that minimize the distance between Viterbi decodings and target transcriptions. We experiment with several forms of updates, systematically comparing the effects of different matrix factorizations, initializations, and averaging schemes on phone accuracies and convergence rates. We present experimental results for context-independent CD-HMMs trained in this way on the TIMIT speech corpus. Our results show that certain types of perceptron training yield consistently significant and rapid reductions in phone error rates.Keywords
Funding Information
- National Science Foundation (812576)
This publication has 9 references indexed in Scilit:
- Large Margin Training of Continuous Density Hidden Markov Models2009
- Approximate Test Risk Bound Minimization Through Soft Margin EstimationIEEE Transactions on Audio, Speech, and Language Processing, 2007
- Large margin hidden Markov models for speech recognitionIEEE Transactions on Audio, Speech, and Language Processing, 2006
- Maximum mutual information estimation of hidden Markov model parameters for speech recognition2005
- Discriminative training methods for hidden Markov models2002
- Large Margin Classification Using the Perceptron AlgorithmMachine Learning, 1999
- Discriminative learning for minimum error classification (pattern recognition)IEEE Transactions on Signal Processing, 1992
- Speaker-independent phone recognition using hidden Markov modelsIEEE Transactions on Acoustics, Speech, and Signal Processing, 1989
- A decision theorectic formulation of a training problem in speech recognition and a comparison of training by unconditional versus conditional maximum likelihoodIEEE Transactions on Acoustics, Speech, and Signal Processing, 1983