Kindle Federated Generalization With Domain Specialized and Invariant Knowledge
- 10 March 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 20 (15566013) , 3461-3474
Abstract
Federated learning, hailed as a privacy-preserving collaboration paradigm, has garnered significant attention in research circles. Typically, it involves multiple clients collaborating to integrate multi-party knowledge, facilitating the learning of a shared global model with decentralized local data. Despite the popularity of federated learning, the surge in approaches addressing various realistic challenges has highlighted a critical issue. The aggregated model may struggle to capture diverse domain knowledge across participants, leading to limited performance in cross-client domain scenarios. Furthermore, the incorporation of knowledge from participating parties can hinder generalization on out-of-client distributions. To comprehensively address this challenge, we dissect federated generalization into two dimensions: the participating domain and the unseen domain. In this paper, we propose a novel solution incorporating domain-specialized and invariant experts. These experts are designed to faithfully represent individual domain characteristics and different domain universality. Additionally, we introduce a pioneering test-time expert aggregation strategy that utilizes prediction consistency metrics to aggregate different experts, specifically tailored for handling agnostic testing distributions. Empirical results validate that our proposed methodology significantly enhances federated performance on both cross-client and out-of-client generalization under different scenarios and with various related methods. A comprehensive ablation study demonstrates the effectiveness of the proposed modules.Keywords
Funding Information
- National Key Research and Development Project of China (2024YFC3308400)
- National Natural Science Foundation of China (623B2080)
- Key Research and Development Program of Hubei Province (2022BAD175)
- Innovative Research Group Project of Hubei Province (2024AFA017)
This publication has 74 references indexed in Scilit:
- Federated learning enables big data for rare cancer boundary detectionNature Communications, 2022
- Federated learning on non-IID data: A surveyNeurocomputing, 2021
- The EU General Data Protection Regulation (GDPR)2017
- Geodesic flow kernel for unsupervised domain adaptation2012
- Adapting Visual Category Models to New Domains2010
- A Tutorial on the Cross-Entropy MethodAnnals of Operations Research, 2005
- Improved learning algorithms for mixture of experts in multiclass classificationNeural Networks, 1999
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- A database for handwritten text recognition researchIEEE Transactions on Pattern Analysis and Machine Intelligence, 1994
- A Mathematical Theory of CommunicationBell System Technical Journal, 1948