A Time-Frequency Generative Adversarial based method for Audio Packet Loss Concealment
Preprint
Abstract
Packet loss is a major cause of voice quality degradation in VoIP transmissions with serious impact on intelligibility and user experience. This paper describes a system based on a generative adversarial approach, which aims to repair the lost fragments during the transmission of audio streams. Inspired by the powerful image-to-image translation capability of Generative Adversarial Networks (GANs), we propose bin2bin, an improved pix2pix framework to achieve the translation task from magnitude spectrograms of audio frames with lost packets, to noncorrupted speech spectrograms. In order to better maintain the structural information after spectrogram translation, this paper introduces the combination of two STFT-based loss functions, mixed with the traditional GAN objective. Furthermore, we employ a modified PatchGAN structure as discriminator and we lower the concealment time by a proper initialization of the phase reconstruction algorithm. Experimental results show that the proposed method has obvious advantages when compared with the current state-of-the-art methods, as it can better handle both high packet loss rates and large gaps.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- A temporal-spectral generative adversarial network based end-to-end packet loss concealment for wideband speech transmissionThe Journal of the Acoustical Society of America, 2021
- WebRTC - Realtime Communication for the Open Web PlatformQueue, 2021
- Audio Inpainting: Revisited and ReweightedIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020
- A Context Encoder For Audio InpaintingIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019
- Fast Spectrogram Inversion Using Multi-Head Convolutional Neural NetworksIEEE Signal Processing Letters, 2018
- Packet Loss Concealment Based on Deep Neural Networks for Digital Speech TransmissionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- Audio InpaintingIEEE Transactions on Audio, Speech, and Language Processing, 2011
- QoS evaluation of sender-based loss-recovery techniques for VoIPIEEE Network, 2006
- Signal estimation from modified short-time Fourier transformIEEE Transactions on Acoustics, Speech, and Signal Processing, 1984