Large Speech Model Enabled Semantic Communication
Preprint
Abstract
Existing speech semantic communication systems mainly based on Joint Source-Channel Coding (JSCC) architectures have demonstrated impressive performance, but their effectiveness remains limited by model structures specifically designed for particular tasks and datasets. Recent advances indicate that generative large models pre-trained on massive datasets, can achieve outstanding performance arexhibit exceptional performance across diverse downstream tasks with minimal fine-tuning. To exploit the rich semantic knowledge embedded in large models and enable adaptive transmission over lossy channels, we propose a Large Speech Model enabled Semantic Communication (LargeSC) system. Simultaneously achieving adaptive compression and robust transmission over lossy channels remains challenging, requiring trade-offs among compression efficiency, speech quality, and latency. In this work, we employ the Mimi as a speech codec, converting speech into discrete tokens compatible with existing network architectures. We propose an adaptive controller module that enables adaptive transmission and in-band Unequal Error Protection (UEP), dynamically adjusting to both speech content and packet loss probability under bandwidth constraints. Additionally, we employ Low-Rank Adaptation (LoRA) to finetune the Moshi foundation model for generative recovery of lost speech tokens. Simulation results show that the proposed system supports bandwidths ranging from 550 bps to 2.06 kbps, outperforms conventional baselines in speech quality under high packet loss rates and achieves an end-to-end latency of approximately 460 ms, thereby demonstrating its potential for real-time deployment.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Low-Bitrate High-Quality Digital Semantic Communication Based on RVQGANIEEE Internet of Things Journal, 2025
- SoundSpring: Loss-Resilient Audio Transceiver With Dual-Functional Masked Language ModelingIEEE Journal on Selected Areas in Communications, 2025
- Big AI Models for 6G Wireless Networks: Opportunities, Challenges, and Research DirectionsIEEE Wireless Communications, 2024
- Large AI Model-Based Semantic CommunicationsIEEE Wireless Communications, 2024
- A Perceptually Motivated Approach for Low-Complexity Speech Semantic CommunicationIEEE Internet of Things Journal, 2024
- Computing Networks Enabled Semantic CommunicationsIEEE Network, 2024
- Vector Quantized Semantic Communication SystemIEEE Wireless Communications Letters, 2023
- Semantic-Preserved Communication System for Highly Efficient Speech TransmissionIEEE Journal on Selected Areas in Communications, 2022
- SoundStream: An End-to-End Neural Audio CodecIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- Semantic Communication Systems for Speech TransmissionIEEE Journal on Selected Areas in Communications, 2021