Reliable Multi-Modal Semantic Communication via AI-Oriented Residual Encoding

Authors:
Open Access
Publisher website
Google Scholar
Abstract
This letter introduces a communication-efficient multi-modal semantic communication (MMSC) framework over wireless networks, termed Residual Hierarchical-Adaptive Modality Control (ReHi-AMC). Under an outage-based transmission without CSI feedback, ReHi-AMC preserves generative reconstruction quality. The framework assigns cross-modal transmission priorities and progressively encodes spatial sketches through a hierarchical multi-resolution representation. To further enhance payload efficiency, residual information between hierarchical levels is compressed via sparse residual encoding, reducing redundancy while maintaining essential structural cues. Numerical evaluations on a Stable Diffusion–based generative architecture demonstrate up to 46.47% improvement in CLIP score, with particularly pronounced gains in low-SNR regimes, confirming robustness against channel outages compared to single-scale baselines without residual encoding.
Funding Information
  • Institute for Information and Communications Technology Promotion (RS-2024-00435652)
  • Smart Networks and Services Joint Undertaking (SNS JU) under European Union’s Horizon Europe Research and Innovation Program (101192194)

This publication has 11 references indexed in Scilit: