Progressive Distribution Matching for Federated Semi-Supervised Learning
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (5) , 5191-5199
Abstract
Federated Learning (FL) enables collaborative learning from distributed data while preserving the privacy of participating clients. While supervised federated learning with labeled data has made notable strides and achieved success, federated semi-supervised learning (FSSL) lags in its progress. Existing works for FSSL heavily rely on fully-labeled clients, while ignoring the distribution of pseudo-labels generated from skewed unlabeled data. In this work, we offer empirical and theoretical insights into the challenges encountered when applying conventional semi-supervised algorithms in the federated regime. Specifically, we highlight how the inherent data heterogeneity in FSSL can exacerbate issues within the pseudo-labeling process. Motivated by these observations, we propose federated learning with progressive distribution matching (FedPDM) to regularize the distribution of pseudo-labels, aiming to progressively reshape it to align with the ground-truth distribution. The matching problem could be formulated as an optimal transport (OT) problem and efficiently solved by Sinkhorn-Knopp iteration. Through extensive experiments, we demonstrate the superiority of FedPDM on a variety of models and datasets compared with prior arts for FSSL.Keywords
This publication has 5 references indexed in Scilit:
- SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training2022
- Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China, Italy, JapanMedical Image Analysis, 2021
- A continual learning survey: Defying forgetting in classification tasksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
- Differential Earth Mover's Distance with Its Applications to Visual TrackingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2008
- The Sinkhorn–Knopp Algorithm: Convergence and ApplicationsSIAM Journal on Matrix Analysis and Applications, 2008