Federated PAC-Bayesian Learning on Non-IID Data
Authors:
- 14 April 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5945-5949
Abstract
Existing research has either adapted the Probably Approximately Correct (PAC) Bayesian framework for federated learning (FL) or used information-theoretic PAC-Bayesian bounds while introducing their theorems, but few consider the non-IID challenges in FL. Our work presents the first non-vacuous federated PAC-Bayesian bound tailored for non-IID local data. This bound assumes unique prior knowledge for each client and variable aggregation weights. We also introduce an objective function and an innovative Gibbs-based algorithm for the optimization of the derived bound. The results are validated on real-world datasets.Keywords
All Related Versions
Funding Information
- Tsinghua University
- Tsinghua Shenzhen International Graduate School
This publication has 11 references indexed in Scilit:
- More Communication Does Not Result in Smaller Generalization Error in Federated Learning2023
- Towards efficient communications in federated learning: A contemporary surveyJournal of the Franklin Institute, 2023
- Improved Information Theoretic Generalization Bounds for Distributed and Federated Learning2022
- Federated Algorithm with Bayesian Approach: Omni-Fedge2021
- MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis2021
- Randomized learning and generalization of fair and private classifiers: From PAC-Bayes to stability and differential privacyNeurocomputing, 2020
- PAC-Bayesian Generalisation Error Bounds for Gaussian Process ClassificationJournal of Machine Learning Research : Jmlr, 2003
- PAC-Bayesian model averaging1999
- Some PAC-Bayesian theorems1998
- On a Variational Formula for the Principal Eigenvalue for Operators with Maximum PrincipleProceedings of the National Academy of Sciences, 1975