SemiSFL: A Novel Semi-Supervised Split Federated Learning System for Efficient Training With Non-IID Data

Authors:
Publisher website
Google Scholar
Abstract
Recently, Split Federated Learning (SFL) advances typical Federated Learning (FL) and offers a feasible solution by splitting a full model into a bottom model (for clients) and a top model (for the server), which alleviates the computation and/or communication burden on clients. However, existing SFL works often assume sufficient labeled data on clients, which is usually impractical. Besides, non-IID data poses another challenge to ensure efficient model training. To our best knowledge, the above two data-related issues have not been simultaneously addressed in SFL. Herein, we review the distinct properties of SFL and develop a novel Semi-supervised SFL system, called SemiSFL. In SemiSFL, a student bottom model and a teacher bottom model are assigned to each client, while the server maintains the top parts and full versions of both the global and teacher models. Cross-entity (i.e., clients and the server) semi-supervised training and server-side supervised training are alternately conducted. During semi-supervised training, we explore regularizing the student features (produced by student bottom models) from different clients with the teacher features (produced by teacher bottom models) before conducting semi-supervised training, so as to improve the generalization ability of student bottom models. Moreover, our theoretical and experimental investigations reveal that the inconsistent training processes on labeled and unlabeled data have an influence on the effectiveness of SemiSFL. To mitigate training inconsistency, we develop an algorithm for dynamically regulating the frequency of performing server-side supervised training. Extensive experiments on benchmark models and datasets show that SemiSFL provides a 3.8× speed-up in training time, reduces the communication cost by about 70.3% while reaching the target accuracy, and achieves up to 5.8% improvement in accuracy under different non-IID settings compared to the state-of-the-art baselines.
Funding Information
  • National Natural Science Foundation of China (62472401)
  • National Natural Science Foundation of China (624B2136)
  • National Natural Science Foundation of China (62132019)
  • National Natural Science Foundation of China (62502487)
  • National Natural Science Foundation of China (WK2150250044)

This publication has 37 references indexed in Scilit: