Enhancing End-to-End Multi-Channel Speech Separation Via Spatial Feature Learning
Authors:
- 1 May 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7319-7323
Abstract
Hand-crafted spatial features (e.g., inter-channel phase difference, IPD) play a fundamental role in recent deep learning based multi-channel speech separation (MCSS) methods. However, these manually designed spatial features are hard to incorporate into the end-to-end optimized MCSS framework. In this work, we propose an integrated architecture for learning spatial features directly from the multi-channel speech waveforms within an end-to-end speech separation framework. In this architecture, time-domain filters spanning signal channels are trained to perform adaptive spatial filtering. These filters are implemented by a 2d convolution (conv2d) layer and their parameters are optimized using a speech separation objective function in a purely data-driven fashion. Furthermore, inspired by the IPD formulation, we design a conv2d kernel to compute the inter-channel convolution differences (ICDs), which are expected to provide the spatial cues that help to distinguish the directional sources. Evaluation results on simulated multi-channel reverberant WSJ0 2-mix dataset demonstrate that our proposed ICD based MCSS model improves the overall signal-to-distortion ratio by 10.4% over the IPD based MCSS model.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Tight Integration of Spatial and Spectral Features for BSS with Deep Clustering Embeddings2017
- Raw Multichannel Processing Using Deep Neural Networks2017
- Permutation invariant training of deep models for speaker-independent multi-talker speech separationMechanical Engineering Journal, 2017
- Neural Network Adaptive Beamforming for Robust Multichannel Speech Recognition2016
- Deep clustering: Discriminative embeddings for segmentation and separation2016
- Speech acoustic modeling from raw multichannel waveforms2015
- Tonotopic organization of human auditory cortexNeuroImage, 2010
- Performance measurement in blind audio source separationIEEE Transactions on Audio, Speech, and Language Processing, 2006
- Image method for efficiently simulating small-room acousticsThe Journal of the Acoustical Society of America, 1979
- Contribution to a Study of the “Cocktail Party Problem”The Journal of the Acoustical Society of America, 1960