Multi-Dimensional and Multi-Scale Modeling for Speech Separation Optimized by Discriminative Learning
Authors:
- 4 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Abstract
Transformer has shown advanced performance in speech separation, benefiting from its ability to capture global features. However, capturing local features and channel information of audio sequences in speech separation is equally important. In this paper, we present a novel approach named Intra-SE-Conformer and Inter-Transformer (ISCIT) for speech separation. Specifically, we design a new network SE-Conformer that can model audio sequences in multiple dimensions and scales, and apply it to the dual-path speech separation framework. Furthermore, we propose Multi-Block Feature Aggregation to improve the separation effect by selectively utilizing information from the intermediate blocks of the separation network. Meanwhile, we propose a speaker similarity discriminative loss to optimize the speech separation model to address the problem of poor performance when speakers have similar voices. Experimental results on the bench-mark datasets WSJ0-2mix and WHAM! show that ISCIT can achieve state-of-the-art results.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- Dual-Path RNN: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation2020
- End-to-End Post-Filter for Speech Separation With Deep Attention Fusion FeaturesIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020
- WHAM!: Extending Speech Separation to Noisy Environments2019
- Discriminative Learning for Monaural Speech Separation Using Deep Embedding Features2019
- Conv-TasNet: Surpassing Ideal Time–Frequency Magnitude Masking for Speech SeparationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019
- SDR – Half-baked or Well Done?2019
- VoxCeleb2: Deep Speaker Recognition2018
- Squeeze-and-Excitation Networks2018
- Multi-Scale multi-band densenets for audio source separation2017
- Deep clustering: Discriminative embeddings for segmentation and separation2016