Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation
Preprint
Abstract
We propose a novel deep learning model, which supports permutation invariant training (PIT), for speaker independent multi-talker speech separation, commonly known as the cocktail-party problem. Different from most of the prior arts that treat speech separation as a multi-class regression problem and the deep clustering technique that considers it a segmentation (or clustering) problem, our model optimizes for the separation regression error, ignoring the order of mixing sources. This strategy cleverly solves the long-lasting label permutation problem that has prevented progress on deep learning based techniques for speech separation. Experiments on the equal-energy mixing setup of a Danish corpus confirms the effectiveness of PIT. We believe improvements built upon PIT can eventually solve the cocktail-party problem and enable real-world adoption of, e.g., automatic meeting transcription and multi-party human-computer interaction, where overlapping speech is common.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Deep clustering: Discriminative embeddings for segmentation and separation2016
- On Training Targets for Supervised Speech SeparationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- Deep Embedding Network for Clustering2014
- An Experimental Study on Speech Enhancement Based on Deep Neural NetworksIEEE Signal Processing Letters, 2013
- Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research GroupsIEEE Signal Processing Magazine, 2012
- Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech RecognitionIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Monaural speech separation and recognition challengeComputer Speech & Language, 2010
- Convolutive Speech Bases and Their Application to Supervised Speech SeparationIEEE Transactions on Audio, Speech, and Language Processing, 2006
- Factorial Hidden Markov ModelsMachine Learning, 1997
- Laws of organization in perceptual forms.1938