Dynamic Client Distillation for Semi-Supervised Federated Learning in a Realistic Scenario
- 1 November 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 44 (11) , 4198-4212
Abstract
Recent advancements in semi-supervised federated learning (SSFL) have significantly enhanced public health services by enabling medical institutions to share model updates via a central server. However, most SSFL approaches are based on conservative assumptions, such as labels-at-server and labels-at-client, which fail to fully capture the complex and diverse data distributions inherent in medical institutions. To address this limitation, we introduce a novel application of SSFL tailored to a realistic client data scenario, encompassing clients with fully-labeled, partially-labeled, and fully-unlabeled data. This approach effectively navigates varying levels of data annotation by maximizing the utility of unlabeled samples within the client federation. To tackle the challenges posed by such a complex scenario, we propose a new SSFL framework, FedCD. FedCD incorporates three client-distilled models, each corresponding to a distinct client data distribution, alongside server-client federation. First, each client-distilled model condenses the diverse parameters of the client federation into robust knowledge through distillation. The contribution of each client model is then dynamically adjusted based on its proximity to the client-distilled model, ensuring that the framework adapts to the heterogeneous characteristics of individual clients. By aggregating client-distilled models, FedCD implements model drift correction, effectively mitigating parameter drift across heterogeneous models. This dynamic federated approach not only harnesses unlabeled data efficiently but also accommodates diverse annotation levels while adapting to varying data distributions. Extensive experiments on two medical image segmentation tasks and one classification task demonstrate the superiority of our method, highlighting its ability to address realistic challenges in medical data scenarios.Keywords
This publication has 37 references indexed in Scilit:
- FedIIC: Towards Robust Federated Learning for Class-Imbalanced Medical Image Classification2023
- Scale Federated Learning for Label Set Mismatch in Medical Image Classification2023
- Interpolation consistency training for semi-supervised learningNeural Networks, 2022
- Federated Semi-supervised Medical Image Classification via Inter-client Relation Matching2021
- Federated Whole Prostate Segmentation in MRI with Personalized Neural Architectures2021
- Boundary-Aware Transformers for Skin Lesion Segmentation2021
- Efficient Semi-supervised Gross Target Volume of Nasopharyngeal Carcinoma Segmentation via Uncertainty Rectified Pyramid Consistency2021
- Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China, Italy, JapanMedical Image Analysis, 2021
- Uncertainty-Aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation2019
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015