Wavesplit: End-to-End Speech Separation by Speaker Clustering
Authors:
- Neil ZeghidourNZNeil Zeghidour
- David GrangierDGDavid Grangier
Preprint
Abstract
We introduce Wavesplit, an end-to-end speech separation system. From a single recording of mixed speech, the model infers and clusters representations of each speaker and then estimates each source signal conditioned on the inferred representations. The model is trained on the raw waveform to jointly perform the two tasks. Our model infers a set of speaker representations through clustering, which addresses the fundamental permutation problem of speech separation. Moreover, the sequence-wide speaker representations provide a more robust separation of long, challenging sequences, compared to previous approaches. We show that Wavesplit outperforms the previous state-of-the-art on clean mixtures of 2 or 3 speakers (WSJ0-2mix, WSJ0-3mix), as well as in noisy (WHAM!) and reverberated (WHAMR!) conditions. As an additional contribution, we further improve our model by introducing online data augmentation for separation.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- TaSNet: Time-Domain Audio Separation Network for Real-Time, Single-Channel Speech Separation2018
- Alternative Objective Functions for Deep Clustering2018
- Single Channel Speech Separation with Constrained Utterance Level Permutation Invariant Training Using Grid LSTM2018
- CBLDNN-Based Speaker-Independent Speech Separation Via Generative Adversarial Training2018
- Novel Deep Architectures in Speech Processing2017
- Improving music source separation based on deep neural networks through data augmentation and network blending2017
- Deep attractor network for single-microphone speaker separation2017
- Permutation invariant training of deep models for speaker-independent multi-talker speech separationMechanical Engineering Journal, 2017
- Deep clustering: Discriminative embeddings for segmentation and separation2016
- Guaranteeing approach to solving quantile optimization problemsAnnals of Operations Research, 1991