Advocating for the Silent: Enhancing Federated Generalization for Nonparticipating Clients
- 27 December 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (8) , 14174-14188
Abstract
Federated learning (FL) has surged in prominence due to its capability of collaborative model training without direct data sharing. However, the vast disparity in local data distributions among clients, often termed the nonindependent identically distributed (Non-IID) challenge, poses a significant hurdle to FL’s generalization efficacy. The scenario becomes even more complex when not all clients participate in the training process, a common occurrence due to unstable network connections or limited computational capacities. This can greatly complicate the assessment of the trained models’ generalization abilities. While a plethora of recent studies has centered on the generalization gap pertaining to unseen data from participating clients with diverse distributions, the distinction between the training distributions of participating clients and the testing distributions of nonparticipating ones has been largely overlooked. In response, our paper unveils an information-theoretic generalization framework for FL. Specifically, it quantifies generalization errors by evaluating the information entropy of local distributions and discerning discrepancies across these distributions. Inspired by our deduced generalization bounds, we introduce a weighted aggregation approach and a duo of client selection strategies. These innovations are designed to strengthen FL’s ability to generalize and thus ensure that trained models perform better on nonparticipating clients by incorporating a more diverse range of client data distributions. Our extensive empirical evaluations reaffirm the potency of our proposed methods, aligning seamlessly with our theoretical construct.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62350710797)
- Major Key Project of PCL (PCL2023A09)
This publication has 37 references indexed in Scilit:
- Leveraging Federated Learning for Unsecured Loan Risk Assessment on Decentralized Finance Lending Platforms2023
- Federated Domain Generalization with Generalization Adjustment2023
- A survey on federated learning for security and privacy in healthcare applicationsComputer Communications, 2023
- Improving Generalization in Federated Learning by Seeking Flat Minima2022
- A state-of-the-art survey on solving non-IID data in Federated LearningFuture Generation Computer Systems, 2022
- Federated learning on non-IID data: A surveyNeurocomputing, 2021
- Towards Non-I.I.D. image classification: A dataset and baselinesPattern Recognition, 2021
- Byzantine-Robust Decentralized Stochastic Optimization2020
- Federated Learning in Mobile Edge Networks: A Comprehensive SurveyIEEE Communications Surveys & Tutorials, 2020
- Deep Residual Learning for Image Recognition2016