Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorization
Authors:
Preprint
Abstract
This paper presents a statistical method of single-channel speech enhancement that uses a variational autoencoder (VAE) as a prior distribution on clean speech. A standard approach to speech enhancement is to train a deep neural network (DNN) to take noisy speech as input and output clean speech. Although this supervised approach requires a very large amount of pair data for training, it is not robust against unknown environments. Another approach is to use non-negative matrix factorization (NMF) based on basis spectra trained on clean speech in advance and those adapted to noise on the fly. This semi-supervised approach, however, causes considerable signal distortion in enhanced speech due to the unrealistic assumption that speech spectrograms are linear combinations of the basis spectra. Replacing the poor linear generative model of clean speech in NMF with a VAE---a powerful nonlinear deep generative model---trained on clean speech, we formulate a unified probabilistic generative model of noisy speech. Given noisy speech as observed data, we can sample clean speech from its posterior distribution. The proposed method outperformed the conventional DNN-based method in unseen noisy environments.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Multichannel Audio Source Separation With Deep Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2016
- Unseen Noise Estimation Using Separable Deep Auto Encoder for Speech EnhancementIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- Speech Enhancement Under Low SNR Conditions Via Noise Estimation Using Sparse and Low-Rank NMF with Kullback–Leibler DivergenceIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- NMF-based Target Source Separation Using Deep Neural NetworkIEEE Signal Processing Letters, 2014
- Bayesian Inference for Nonnegative Matrix Factor Deconvolution Models2010
- Multichannel Nonnegative Matrix Factorization in Convolutive Mixtures for Audio Source SeparationIEEE Transactions on Audio, Speech, and Language Processing, 2009
- Nonnegative Matrix Factorization with the Itakura-Saito Divergence: With Application to Music AnalysisNeural Computation, 2009
- Supervised and Semi-supervised Separation of Sounds from Single-Channel Mixtures2007
- Performance measurement in blind audio source separationIEEE Transactions on Audio, Speech, and Language Processing, 2006
- Speech enhancement using a minimum-mean square error short-time spectral amplitude estimatorIEEE Transactions on Acoustics, Speech, and Signal Processing, 1984