Optimizing Communication Efficiency through Training Potential in Multi-Modal Federated Learning
Open Access
- 19 August 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Internet Technology
- Vol. 25 (3) , 1-31
Abstract
Multi-modal Federated Learning (FL) is a type of FL that considers utilizing multiple modalities of data to improve overall performance. While multi-modal data brings richer information, it also introduces more significant communication overhead. Reducing this overhead hinges on two key strategies: increasing the convergence speed of the training or reducing the communication overhead in each communication round. However, few studies have considered these two strategies simultaneously and formed a unified optimization framework. Thus, we propose a joint client and modality selection framework to reduce communication overhead. Modality selection executed on each client assigns weights to modalities based on their contribution to training potential, aiming at accelerating the convergence. Client selection executed on the server assigns weights to clients by considering different metrics, especially total training potential after the modality selection. We validate our proposed method on the five widely used open-source datasets, achieving satisfactory accuracy while reducing the total communication overhead to 2.43%–14.24% compared to without selection on different datasets, significantly outperforming existing state-of-the-art (SOTA) methods. Code is available at https://github.com/1643204431/OCETPMMFL .Keywords
Funding Information
- National Natural Science Foundation of China (62202055 and 62202016)
- Guangdong Basic and Applied Basic Research Foundation (2025A1515012843)
- Start-up Fund from Beijing Normal University (312200502510)
- Internal Fund from Beijing Normal-Hong Kong Baptist University (UICR0400003-24)
- Project of Young Innovative Talents of Guangdong Education Department (2022KQNCX102)
- Interdisciplinary Intelligence SuperComputer Center of Beijing Normal University
This publication has 18 references indexed in Scilit:
- Structural MRI of brain similarity networksNature Reviews Neuroscience, 2024
- Fedmm: Federated Multi-Modal Learning with Modality Heterogeneity in Computational Pathology2024
- HCMS: Hierarchical and Conditional Modality Selection for Efficient Video RecognitionACM Transactions on Multimedia Computing, Communications, and Applications, 2023
- FedMix: Mixed Supervised Federated Learning for Medical Image SegmentationIEEE Transactions on Medical Imaging, 2022
- Robust Federated Learning with Noisy and Heterogeneous Clients2022
- A unified framework for multi-modal federated learningNeurocomputing, 2022
- Optimizing Federated Learning on Non-IID Data with Reinforcement Learning2020
- Learning Not to Learn: Training Deep Neural Networks With Biased Data2019
- Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge2019
- An ensemble based data fusion approach for early diagnosis of Alzheimer’s diseaseInformation Fusion, 2008