Towards Resource-Efficient Edge AI: From Federated Learning to Semi-Supervised Model Personalization
- 18 September 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 23 (5) , 6104-6115
Abstract
A central question in edge intelligence is “how can an edge device learn its local model with limited data and constrained computing capacity?” In this study, we explore the approach where a global model initialization is first obtained by running federated learning (FL) across multiple edge devices, based on which a semi-supervised algorithm is devised for a single edge device to carry out quick adaptation with its local data. Specifically, to account for device heterogeneity and resource constraints, a global model is first trained via FL, where each device conducts multiple local updates only for its customized subnet. A subset of devices can be selected to upload updates for aggregation during each training round. Further, device scheduling is optimized to minimize the training loss of FL, subject to resource constraints, based on the carefully crafted reward function defined as the one-round progress of FL each device can provide. We examine the convergence behavior of FL for the general non-convex case. For semi-supervised model personalization, we use the FL-based model initialization as a teacher network to impute soft labels on unlabeled data, thereby addressing the insufficiency of labeled data. Experiments are conducted to evaluate the performance of the proposed algorithms.Keywords
This publication has 23 references indexed in Scilit:
- Helios: Heterogeneity-Aware Federated Learning with Dynamically Balanced Collaboration2021
- A Collaborative Learning Framework via Federated Meta-Learning2020
- Distributionally Robust Edge Learning with Dirichlet Process Prior2020
- Importance Estimation for Neural Network Pruning2019
- Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition2016
- ImageNet: A large-scale hierarchical image database2009
- Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories2005
- Structural learning with forgettingNeural Networks, 1996
- Pruning algorithms-a surveyIEEE Transactions on Neural Networks, 1993
- A simple procedure for pruning back-propagation trained neural networksIEEE Transactions on Neural Networks, 1990