Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification
Preprint
Abstract
Improving speech system performance in noisy environments remains a challenging task, and speech enhancement (SE) is one of the effective techniques to solve the problem. Motivated by the promising results of generative adversarial networks (GANs) in a variety of image processing tasks, we explore the potential of conditional GANs (cGANs) for SE, and in particular, we make use of the image processing framework proposed by Isola et al. [1] to learn a mapping from the spectrogram of noisy speech to an enhanced counterpart. The SE cGAN consists of two networks, trained in an adversarial manner: a generator that tries to enhance the input noisy spectrogram, and a discriminator that tries to distinguish between enhanced spectrograms provided by the generator and clean ones from the database using the noisy spectrogram as a condition. We evaluate the performance of the cGAN method in terms of perceptual evaluation of speech quality (PESQ), short-time objective intelligibility (STOI), and equal error rate (EER) of speaker verification (an example application). Experimental results show that the cGAN method overall outperforms the classical short-time spectral amplitude minimum mean square error (STSA-MMSE) SE algorithm, and is comparable to a deep neural network-based SE approach (DNN-SE).Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Image De-Raining Using a Conditional Generative Adversarial NetworkIEEE Transactions on Circuits and Systems for Video Technology, 2019
- Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network2017
- Speech Intelligibility Potential of General and Specialized Deep Neural Network Based Speech Enhancement SystemsIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2016
- Text Dependent Speaker Verification Using Un-Supervised HMM-UBM and Temporal GMM-UBM2016
- Large-scale training to increase speech intelligibility for hearing-impaired listeners in novel noisesThe Journal of the Acoustical Society of America, 2016
- A Regression Approach to Speech Enhancement Based on Deep Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- On Training Targets for Supervised Speech SeparationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- Speech enhancement based on deep denoising autoencoder2013
- An Algorithm for Intelligibility Prediction of Time–Frequency Weighted Noisy SpeechIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Speech enhancement using a minimum-mean square error short-time spectral amplitude estimatorIEEE Transactions on Acoustics, Speech, and Signal Processing, 1984