Generative Semantic Communication for Joint Image Transmission and Segmentation
Authors:
- 8 June 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1110-1115
Abstract
Semantic communication has emerged as a promising technology for enhancing communication efficiency. However, most existing research emphasizes single-task reconstruction, neglecting model adaptability and generalization across multi-task systems. In this paper, we propose a novel generative semantic communication system that supports both image reconstruction and segmentation tasks. Our approach builds upon semantic knowledge bases (KBs) at both the transmitter and receiver, with each semantic KB comprising a source KB and a task KB. The source KB at the transmitter leverages a hierarchical Swin-Transformer, a generative AI scheme, to extract multi-level features from the input image. Concurrently, the counterpart source KB at the receiver utilizes hierarchical residual blocks to generate task-specific knowledge. Furthermore, the task KBs adopt a semantic similarity model to map different task requirements into pre-defined task instructions, thereby facilitating the feature selection of the source KBs. Additionally, we develop a unified residual block-based joint source and channel (JSCC) encoder and two task-specific JSCC decoders to achieve the two image tasks. In particular, a generative diffusion model is adopted to construct the JSCC decoder for the image reconstruction task. Experimental results show that our multi-task generative semantic communication system outperforms previous single-task communication systems in terms of peak signal-to-noise ratio and segmentation accuracy.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Generative Semantic Communication for Text-to-Speech Synthesis2024
- A Scalable Multi-Device Semantic Communication System for Multi - Task Execution2023
- Scalable Diffusion Models with Transformers2023
- Knowledge-enhanced semantic communication system with OFDM transmissionsScience China Information Sciences, 2023
- WITT: A Wireless Image Transmission Transformer for Semantic Communications2023
- Task-Oriented Multi-User Semantic CommunicationsIEEE Journal on Selected Areas in Communications, 2022
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows2021
- NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study2017
- Deep Residual Learning for Image Recognition2016
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009