Multi-microphone Complex Spectral Mapping for Utterance-wise and Continuous Speech Separation
Top Cited Papers
- 26 May 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 29 (23299290) , 2001-2014
Abstract
We propose multi-microphone complex spectral mapping, a simple way of applying deep learning for time-varying non-linear beamforming, for speaker separation in reverberant conditions. We aim at both speaker separation and dereverberation. Our study first investigates offline utterance-wise speaker separation and then extends to block-online continuous speech separation (CSS). Assuming a fixed array geometry between training and testing, we train deep neural networks (DNN) to predict the real and imaginary (RI) components of target speech at a reference microphone from the RI components of multiple microphones. We then integrate multi-microphone complex spectral mapping with minimum variance distortionless response (MVDR) beamforming and post-filtering to further improve separation, and combine it with frame-level speaker counting for block-online CSS. Although our system is trained on simulated room impulse responses (RIR) based on a fixed number of microphones arranged in a given geometry, it generalizes well to a real array with the same geometry. State-of-the-art separation performance is obtained on the simulated two-talker SMS-WSJ corpus and the real-recorded LibriCSS dataset.Keywords
All Related Versions
Funding Information
- National Institute on Deafness and Other Communication Disorders (R01 DC012048)
- NSF (ECCS-1808932)
- Ohio Supercomputer Center
This publication has 38 references indexed in Scilit:
- The third ‘CHiME’ speech separation and recognition challenge: Analysis and outcomesComputer Speech & Language, 2017
- Multitalker Speech Separation With Utterance-Level Permutation Invariant Training of Deep Recurrent Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- Densely Connected Convolutional Networks2017
- A Consolidated Perspective on Multimicrophone Speech Enhancement and Source SeparationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- An Algorithm for Predicting the Intelligibility of Speech Masked by Modulated Noise MaskersIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2016
- Deep beamforming networks for multi-channel speech recognition2016
- Deep clustering: Discriminative embeddings for segmentation and separation2016
- Complex Ratio Masking for Monaural Speech SeparationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- BLSTM supported GEV beamformer front-end for the 3RD CHiME challenge2015
- The NTT CHiME-3 system: Advances in speech enhancement and recognition for mobile multi-microphone devices2015