Masked Spectrogram Prediction for Unsupervised Domain Adaptation in Speech Enhancement
Open Access
- 14 December 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Open Journal of Signal Processing
- Vol. 5, 274-283
Abstract
Supervised learning-based speech enhancement methods often work remarkably well in acoustic situations represented in the training corpus but generalize poorly to out-of-domain situations, i.e. situations not seen during training. This stands in the way of further improvement of these methods in realistic scenarios, as collecting paired noisy-clean recordings in the target application domain is typically not feasible. Recording noisy-only in-domain data is, though, much more practical. In this paper, we tackle the problem of unsupervised domain adaptation in speech enhancement. Specifically, we propose a way to use in-domain noisy-only data in the training of a neural network to improve upon a model trained solely on out-of-domain paired data. For this, we make use of masked spectrogram prediction, a technique from self-supervised learning that aims to interpolate masked regions of a spectrogram. We hypothesize that masked spectrogram prediction encourages learning of features that represent well both speech and noise components of the noisy signals. These features can then be used to train a more robust speech enhancement system. We evaluate the proposed method on the VoiceBank-DEMAND and LibriFSD50k databases, with WSJ0-CHiME3 serving as the out-of-domain database. We show that the proposed method outperforms both the out-of-domain system and the baseline approaches, i.e. RemixIT and noisy-target training, and also combines well with the previously proposed RemixIT method.Keywords
Funding Information
- Innovationsfonden
This publication has 45 references indexed in Scilit:
- SEGAN: Speech Enhancement Generative Adversarial Network2017
- Investigating RNN-based speech enhancement methods for noise-robust Text-to-Speech2016
- An Algorithm for Predicting the Intelligibility of Speech Masked by Modulated Noise MaskersIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2016
- The third ‘CHiME’ speech separation and recognition challenge: Dataset, task and baselines2015
- Librispeech: An ASR corpus based on public domain audio books2015
- The voice bank corpus: Design, collection and data analysis of a large regional accent speech database2013
- The diverse environments multi-channel acoustic noise database: A database of multichannel environmental noise recordingsThe Journal of the Acoustical Society of America, 2013
- Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs2002
- HMM-based strategies for enhancement of speech signals embedded in nonstationary noiseIEEE Transactions on Speech and Audio Processing, 1998
- The design for the wall street journal-based CSR corpus1992