Information-Theoretic Generalization Analysis for Topology-Aware Heterogeneous Federated Edge Learning Over Noisy Channels
- 26 December 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 31 (10709908) , 691-695
Abstract
With the rapid growth of edge intelligence, the deployment of federated learning (FL) over wireless networks has garnered increasing attention, which is called Federated Edge Learning (FEEL). In FEEL, both mobile devices transmitting model parameters over noisy channels and collecting data in diverse environments pose challenges to the generalization of trained models. Moreover, devices can engage in decentralized FL via Device-to-Device communication while the communication topology of connected devices also impacts the generalization of models. Most recent theoretical studies overlook the incorporation of all these effects into FEEL when developing generalization analyses and ignore designing algorithms to enhance the generalization of models based on their analysis. In contrast, our work presents an information-theoretic generalization analysis for topology-aware FEEL with data heterogeneity and noisy channels. Additionally, we propose a novel regularization method called Federated Global Mutual Information Reduction (FedGMIR) to enhance the performance of models based on our analysis. Numerical results validate our theoretical findings and provide evidence for the effectiveness of the proposed method.Keywords
Funding Information
- Key Program of Fundamental Research from Shenzhen Science and Technology Innovation Commission (JCYJ20200109113403826)
- Major Key Project of PCL (2022ZD0115301)
- Open Research Project of Zhejiang Lab (2022RC0AB04)
This publication has 9 references indexed in Scilit:
- Gradient and Channel Aware Dynamic Scheduling for Over-the-Air Computation in Federated Edge Learning SystemsIEEE Journal on Selected Areas in Communications, 2023
- FedHybrid: A Hybrid Federated Optimization Method for Heterogeneous ClientsIEEE Transactions on Signal Processing, 2023
- Improved Information-Theoretic Generalization Bounds for Distributed, Federated, and Iterative LearningEntropy, 2022
- Federated Learning on Non-IID Data Silos: An Experimental Study2022
- Federated Learning Over Wireless Device-to-Device Networks: Algorithms and Convergence AnalysisIEEE Journal on Selected Areas in Communications, 2021
- Stability and Generalization of Decentralized Stochastic Gradient DescentProceedings of the AAAI Conference on Artificial Intelligence, 2021
- Information-Theoretic Bounds on the Generalization Error and Privacy Leakage in Federated Learning2020
- Federated Learning Over Wireless Fading ChannelsIEEE Transactions on Wireless Communications, 2020
- Generalization Error Bounds for Noisy, Iterative Algorithms2018