Knowledge Transfer in Permutation Invariant Training for Single-Channel Multi-Talker Speech Recognition
Authors:
- 1 April 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 571-5718
Abstract
This paper proposes a framework that combines teacher-student training and permutation invariant training (PIT) for single-channel multi-talker speech recognition. In contrast to most of conventional teacher-student training methods that aim at compressing the model, the proposed method distills knowledge from the single-talker model to improve the multi-talker model in the PIT framework. The inputs to the teacher and student networks are the single-talker clean speech and the multi-talker mixed speech, respectively. The knowledge is transferred to the student through the soft labels generated by the teacher. Furthermore, the ensemble of multiple teachers is exploited with a progressive training scheme to further improve the system. In this framework it is easy to take advantage of data augmentation and perform domain adaptation for multi-talker speech recognition using only untranscribed data. The proposed techniques were evaluated on artificially mixed two-talker AMI speech data. The experimental results show that the teacher-student training can cut the word error rate (WER) by relative 20% against the baseline PIT model. We also evaluated our unsupervised domain adaptation method on an artificially mixed WSJO corpus and achieved relative 30% WER reduction against the AMI PIT model.Keywords
This publication has 14 references indexed in Scilit:
- The microsoft 2016 conversational speech recognition system2017
- Deep attractor network for single-microphone speaker separation2017
- Deep Convolutional Neural Networks with Layer-Wise Context Expansion and Attention2016
- Single-Channel Multi-Speaker Separation Using Deep Clustering2016
- Distilling Knowledge from Ensembles of Neural Networks for Speech Recognition2016
- Very Deep Convolutional Neural Networks for Noise Robust Speech RecognitionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2016
- Deep clustering: Discriminative embeddings for segmentation and separation2016
- Deep Neural Networks for Single-Channel Multi-Talker Speech RecognitionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research GroupsIEEE Signal Processing Magazine, 2012
- Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech RecognitionIEEE Transactions on Audio, Speech, and Language Processing, 2011