Addressing the Curse of Scenario and Task Generalization in AI-6G: A Multi-Modal Paradigm
- 21 April 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 24 (9) , 7377-7391
Abstract
Existing works on machine learning (ML)-empowered wireless communication primarily focus on monolithic scenarios and single tasks. However, with the blooming growth of communication task classes coupled with various task requirements in future 6G systems, this working pattern is obviously unsustainable. Therefore, identifying a groundbreaking paradigm that enables a universal model to solve multiple tasks in the physical layer within diverse scenarios is crucial for future system evolution. This paper aims to fundamentally address the curse of ML model generalization across diverse scenarios and tasks by unleashing multi-modal feature integration capabilities in future systems. Given the universality of electromagnetic propagation theory, the communication process is determined by the scattering environment, which can be more comprehensively characterized by cross-modal perception, thus providing sufficient information for all communication tasks across varied environments. This fact motivates us to propose a transformative two-stage multi-modal pre-training and downstream task adaptation paradigm. In the pre-training stage, we introduce a multi-modal two-tower model and a corresponding contrastive learning method to integrate the explicit description of the scattering environment and implicit channel state information (CSI) into a universal representation, which encapsulates rich high-level knowledge and can be leveraged for all downstream tasks in different scenarios. Additionally, we present two specially designed model structures to enhance the interaction of communication modalities. In the second stage, based on the frozen pre-trained model, we propose a direct method and a pluggable method for flexible and low-cost task adaptation. Experimental results demonstrate that our proposed approach significantly outperforms benchmarks in both task performance and tuning parameter size for exemplary sub-tasks in unseen scenarios.Keywords
All Related Versions
Funding Information
- Shanghai Municipal Education Commission (2024AIYB002)
- National Natural Science Foundation of China Program (62422111, 62371291, 62271316, 62222111)
- National Key Research and Development Program of China (2024YFE0200600)
- Science and Technology Commission Foundation of Shanghai (24DP1500702)
This publication has 35 references indexed in Scilit:
- Position-Aided Beam Prediction in the Real World: How Useful GPS Locations Actually are?2023
- DDA-Net: A Discrepancy-Based Domain Adaptation Network for CSI Feedback Transferability2023
- Self-Supervised WiFi-Based Activity Recognition2022
- Towards Real-World 6G Drone Communication: Position and Camera Aided Beam Prediction2022
- A CSI Based Localization and Identification Recognition Algorithm Using Multi-task Learning and Deep Residual Shrinkage Network2022
- MAP-CSI: Single-site Map-Assisted Localization Using Massive MIMO CSI2021
- DNN-based Beam and Blockage Prediction in 3GPP InH Scenario2020
- Focal Loss for Dense Object Detection2017
- Channel parameter estimation in mobile radio environments using the SAGE algorithmIEEE Journal on Selected Areas in Communications, 1999
- ESPRIT-estimation of signal parameters via rotational invariance techniquesIEEE Transactions on Acoustics, Speech, and Signal Processing, 1989