Personalized speech enhancement: new models and Comprehensive evaluation
Authors:
- 23 May 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 356-360
Abstract
Personalized speech enhancement (PSE) models utilize additional cues, such as speaker embeddings like d-vectors, to remove background noise and interfering speech in real-time and thus improve the speech quality of online video conferencing systems for various acoustic scenarios. In this work, we propose two neural networks for PSE that achieve superior performance to the previously proposed VoiceFilter. In addition, we create test sets that capture a variety of scenarios that users can encounter during video conferencing. Furthermore, we propose a new metric to measure the target speaker over-suppression (TSOS) problem, which was not sufficiently investigated before despite its critical importance in deployment. Besides, we propose multi-task training with a speech recognition back-end. Our results show that the proposed models can yield better speech recognition accuracy, speech intelligibility, and perceptual quality than the baseline models, and the multi-task training can alleviate the TSOS issue in addition to improving the speech recognition accuracy.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Intrusive and Non-Intrusive Perceptual Speech Quality Assessment Using a Convolutional Neural Network2019
- VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking2019
- Single-channel Speech Extraction Using Speaker Inventory and Attention Network2019
- Single Channel Target Speaker Extraction and Recognition with Speaker Beam2018
- Deep attractor network for single-microphone speaker separation2017
- Single-Channel Multi-Speaker Separation Using Deep Clustering2016
- Deep clustering: Discriminative embeddings for segmentation and separation2016
- Complex Ratio Masking for Monaural Speech SeparationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- LibriVox: Free Public Domain AudiobooksReference Reviews, 2014
- An Algorithm for Intelligibility Prediction of Time–Frequency Weighted Noisy SpeechIEEE Transactions on Audio, Speech, and Language Processing, 2011