Enhancing Deep Semantic Communication Systems (Deep-SC) using the Pre-trained Models; BERT and BART
Authors:
- 19 May 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 515-519
Abstract
In recent years, emphasis has been placed on semantic communication as one of the proposals to reach 6 generation wireless communication due to its ability to preserve basic resources for a communications network. The reason for the development in this field is the remarkable development in deep learning, which has enabled us to enable these systems. One of these proposed systems is the deep learning based semantic communication system named Deep-SC which is a system based on transformer for text transmission, which shows a great potential in sent text over physical channel in low signal-to-noise (SNR) regime. In this paper, a new system was proposed using pre-trained models in the semantic encoder and decoder. That was done by utilizing BART and BERT models instead of the transformer model. The work contributes to the ongoing effort to bridge the gap between machine and human communication by demonstrating the transformative potential of leveraging pre-trained models like BERT and BART. BLEU Score and Sentence Similarity are used as performance metrics to assess the system performance. The results showed that the proposed system has significant enhanced the system performance against benchmark systems. It has shown more robustness to channel variation and achieved better performance, especially in low SNR conditions.Keywords
This publication has 18 references indexed in Scilit:
- WITT: A Wireless Image Transmission Transformer for Semantic Communications2023
- Terahertz Communications: Challenges in the Next DecadeIEEE Transactions on Terahertz Science and Technology, 2021
- Semantic Communications for Speech Signals2021
- Deep Learning Enabled Semantic Communication SystemsIEEE Transactions on Signal Processing, 2021
- Internet of Autonomous Vehicles: Architecture, Features, and Socio-Technological ChallengesIEEE Wireless Communications, 2019
- Turbo Decoding2005
- Fundamentals of Wireless Communication2005
- Improved backing-off for M-gram language modeling2002
- Low-density parity-check codesIEEE Transactions on Information Theory, 1962
- Polynomial Codes Over Certain Finite FieldsJournal of the Society for Industrial and Applied Mathematics, 1960