GitFL: Uncertainty-Aware Real-Time Asynchronous Federated Learning Using Version Control
Authors:
- 5 December 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 145-157
Abstract
As a promising distributed machine learning paradigm that enables collaborative training without compromising data privacy, Federated Learning (FL) has been increasingly used in large-scale A IoT (Artificial Intelligence of Things) system design. However, due to the lack of efficient management of straggling devices, existing FL methods greatly suffer from the problems of long response time (e.g., training and communication latency) and low inference accuracy. Things become even worse when taking various uncertain factors (e.g., network delays, performance variances caused by process variation) existing in AIoT scenarios into account. To address this issue, this paper proposes a novel asynchronous FL framework named GitFL, whose implementation is inspired by the famous version control system Git. Unlike traditional FL, the cloud server of GitFL maintains a master model (i.e., the global model) together with a set of branch models indicating the trained local models committed by selected devices, where the master model is updated based on both all the pushed branch models and their version information, and only the branch models after the pull operation are dispatched to devices. By using our proposed Reinforcement Learning (RL)-based device selection mechanism, a pulled branch model with an older version will be more likely to be dispatched to a faster and less frequently selected device for the next round of local training. In this way, GitFL enables both effective controls of model staleness and adaptive load balance of versioned models among straggling devices, thus avoiding performance deterioration while ensuring real-time performance. Comprehensive experimental results on well-known models and datasets show that, compared with state-of-the-art asynchronous and synchronous FL methods, GitFL can achieve up to 2.64X training acceleration and 7.88 % inference accuracy improvements in various uncertain scenarios.Keywords
Funding Information
- National Research Foundation Singapore (AISG2-RP-2020-019)
- DSO National Laboratories under the AI Singapore Programme (AISG2-RP-2020-019)
- Natural Science Foundation of China (62272170)
This publication has 32 references indexed in Scilit:
- Advances and Open Problems in Federated LearningFoundations and Trends in Machine Learning, 2021
- SAFA: A Semi-Asynchronous Protocol for Fast Federated Learning With Low OverheadIEEE Transactions on Computers, 2021
- Quantitative Timing Analysis for Cyber-Physical Systems Using Uncertainty-Aware Scenario-Based SpecificationsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020
- Adaptive Federated Learning and Digital Twin for Industrial Internet of ThingsIEEE Transactions on Industrial Informatics, 2020
- Optimizing Federated Learning on Non-IID Data with Reinforcement Learning2020
- Blockchain Empowered Asynchronous Federated Learning for Secure Data Sharing in Internet of VehiclesIEEE Transactions on Vehicular Technology, 2020
- Curiosity-Driven Exploration by Self-Supervised Prediction2017
- Underdesigned and Opportunistic Computing in Presence of Hardware VariabilityIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2012
- Reinforcement Learning and Markov Decision Processes2012
- Q-learningMachine Learning, 1992