Adaptive Permutation Invariant Training with Auxiliary Information for Monaural Multi-Talker Speech Recognition
Authors:
- 1 April 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5974-5978
Abstract
In this paper, we extend our previous work on direct recognition of single-channel multi-talker mixed speech using permutation invariant training (PIT). We propose to adapt the PIT models with auxiliary features such as pitch and i-vector, and to exploit the gender information with multi-task learning which jointly optimizes for the speech recognition and speaker-pair prediction. We also compare CNN-BLSTMs against BLSTM-RNNs used in our previous PIT-ASR model. The experimental results on the artificially mixed two-talker AMI data indicate that our proposed model improvements can reduce word error rate (WER) by ~ 10.0% relative to our previous work for both speakers in the mixed speech. Our results also confirm that PIT can be easily combined with advanced techniques to improve the performance on multi-talker speech recognition.Keywords
This publication has 17 references indexed in Scilit:
- Deep Convolutional Neural Networks for Large-scale Speech TasksNeural Networks, 2015
- Learning hidden unit contributions for unsupervised speaker adaptation of neural network acoustic models2014
- Convolutional Neural Networks for Speech RecognitionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- A pitch extraction algorithm tuned for automatic speech recognition2014
- Speaker adaptation of neural network acoustic models using i-vectors2013
- KL-divergence regularized deep neural network adaptation for improved large vocabulary speech recognition2013
- Adaptation of context-dependent deep neural networks for automatic speech recognition2012
- Monaural speech separation and recognition challengeComputer Speech & Language, 2010
- A sawtooth waveform inspired pitch estimator for speech and musicThe Journal of the Acoustical Society of America, 2008
- Maximum likelihood linear transformations for HMM-based speech recognitionComputer Speech & Language, 1998