Utterance-level Permutation Invariant Training with Latency-controlled BLSTM for Single-channel Multi-talker Speech Separation
Preprint
Abstract
Utterance-level permutation invariant training (uPIT) has achieved promising progress on single-channel multi-talker speech separation task. Long short-term memory (LSTM) and bidirectional LSTM (BLSTM) are widely used as the separation networks of uPIT, i.e. uPIT-LSTM and uPIT-BLSTM. uPIT-LSTM has lower latency but worse performance, while uPIT-BLSTM has better performance but higher latency. In this paper, we propose using latency-controlled BLSTM (LC-BLSTM) during inference to fulfill low-latency and good-performance speech separation. To find a better training strategy for BLSTM-based separation network, chunk-level PIT (cPIT) and uPIT are compared. The experimental results show that uPIT outperforms cPIT when LC-BLSTM is used during inference. It is also found that the inter-chunk speaker tracing (ST) can further improve the separation performance of uPIT-LC-BLSTM. Evaluated on the WSJ0 two-talker mixed-speech separation task, the absolute gap of signal-to-distortion ratio (SDR) between uPIT-BLSTM and uPIT-LC-BLSTM is reduced to within 0.7 dB.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Single-channel multi-talker speech recognition with permutation invariant trainingSpeech Communication, 2018
- TaSNet: Time-Domain Audio Separation Network for Real-Time, Single-Channel Speech Separation2018
- Adaptive Permutation Invariant Training with Auxiliary Information for Monaural Multi-Talker Speech Recognition2018
- Knowledge Transfer in Permutation Invariant Training for Single-Channel Multi-Talker Speech Recognition2018
- Progressive Joint Modeling in Unsupervised Single-Channel Overlapped Speech RecognitionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- Multitalker Speech Separation With Utterance-Level Permutation Invariant Training of Deep Recurrent Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- Low Latency Acoustic Modeling Using Temporal Convolution and LSTMsIEEE Signal Processing Letters, 2017
- Permutation invariant training of deep models for speaker-independent multi-talker speech separationMechanical Engineering Journal, 2017
- Training Deep Bidirectional LSTM Acoustic Model for LVCSR by a Context-Sensitive-Chunk BPTT ApproachIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2016
- Framewise phoneme classification with bidirectional LSTM and other neural network architecturesNeural Networks, 2005