SoundSpring: Loss-Resilient Audio Transceiver With Dual-Functional Masked Language Modeling
Authors:
- 20 January 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Communications
- Vol. 43 (4) , 1308-1322
Abstract
In this paper, we propose “SoundSpring”, a cutting-edge error-resilient audio transceiver that marries the robustness benefits of joint source-channel coding (JSCC) while also being compatible with current digital communication systems. Unlike recent deep JSCC transceivers, which learn to directly map audio signals to analog channel-input symbols via neural networks, our SoundSpring adopts the layered architecture that delineates audio compression from digital coded transmission, but it sufficiently exploits the impressive in-context predictive capabilities of large language (foundation) models. Integrated with the casual-order mask learning strategy, our single model operates on the latent feature domain and serve dual-functionalities: as efficient audio compressors at the transmitter and as effective mechanisms for packet loss concealment at the receiver. By jointly optimizing towards both audio compression efficiency and transmission error resiliency, we show that mask-learned language models are indeed powerful contextual predictors, and our dual-functional compression and concealment framework offers fresh perspectives on the application of foundation language models in audio communication. Through extensive experimental evaluations, we establish that SoundSpring apparently outperforms contemporary audio transmission systems in terms of signal fidelity metrics and perceptual quality scores. These new findings not only advocate for the practical deployment of SoundSpring in learning-based audio communication systems but also inspire the development of future audio semantic transceivers.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2024YFF0509700)
- National Natural Science Foundation of China (62321001, 62293481, 62371063, 92267301, 62201089)
- Natural Science Foundation of Beijing Municipality (L232047, 4222012)
- Program for Youth Innovative Research Team of BUPT (2023YQTD02)
This publication has 33 references indexed in Scilit:
- Wireless Deep Speech Semantic Transmission2023
- Low-Bitrate Redundancy Coding of Speech Using A Rate-Distortion-Optimized Variational Autoencoder2023
- MAGVIT: Masked Generative Video Transformer2023
- Robust Semantic Transmission of Images with Generative Adversarial Networks2022
- Semantic Communication Systems for Speech TransmissionIEEE Journal on Selected Areas in Communications, 2021
- Efficient and Scalable Neural Residual Waveform Coding with Collaborative Quantization2020
- SDR – Half-baked or Well Done?2019
- TaSNet: Time-Domain Audio Separation Network for Real-Time, Single-Channel Speech Separation2018
- Wavenet Based Low Rate Speech Coding2018
- Librispeech: An ASR corpus based on public domain audio books2015