Federated PAC-Bayesian Learning on Non-IID data
Authors:
- Zihao ZhaoZZZihao Zhao
- Xiao-Ping ZhangXZXiao-Ping Zhang
Preprint
Abstract
Existing research has either adapted the Probably Approximately Correct (PAC) Bayesian framework for federated learning (FL) or used information-theoretic PAC-Bayesian bounds while introducing their theorems, but few considering the non-IID challenges in FL. Our work presents the first non-vacuous federated PAC-Bayesian bound tailored for non-IID local data. This bound assumes unique prior knowledge for each client and variable aggregation weights. We also introduce an objective function and an innovative Gibbs-based algorithm for the optimization of the derived bound. The results are validated on real-world datasets.Keywords
All Related Versions
This publication has 4 references indexed in Scilit:
- More Communication Does Not Result in Smaller Generalization Error in Federated Learning2023
- Towards efficient communications in federated learning: A contemporary surveyJournal of the Franklin Institute, 2023
- Randomized learning and generalization of fair and private classifiers: From PAC-Bayes to stability and differential privacyNeurocomputing, 2020
- PAC-Bayesian Generalisation Error Bounds for Gaussian Process ClassificationJournal of Machine Learning Research : Jmlr, 2003