Online Federated Learning via Non-Stationary Detection and Adaptation Amidst Concept Drift
- 1 August 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Networking
- Vol. 32 (1) , 643-653
Abstract
Federated Learning (FL) is an emerging domain in the broader context of artificial intelligence research. Methodologies pertaining to FL assume distributed model training, consisting of a collection of clients and a server, with the main goal of achieving optimal global model with restrictions on data sharing due to privacy concerns. It is worth highlighting that the diverse existing literature in FL mostly assume stationary data generation processes; such an assumption is unrealistic in real-world conditions where concept drift occurs due to, for instance, seasonal or period observations, faults in sensor measurements. In this paper, we introduce a multiscale algorithmic framework which combines theoretical guarantees of FedAvg and FedOMD algorithms in near stationary settings with a non-stationary detection and adaptation technique to ameliorate FL generalization performance in the presence of concept drifts. We present a multi-scale algorithmic framework leading to $\Tilde{\mathcal{O}} (\min \{ \sqrt{LT}, \Delta^{({1}/{3})}T^{({2}/{3})} + \sqrt{T} \})$ dynamic regret for $T$ rounds with an underlying general convex loss function, where $L$ is the number of times non-stationary drifts occurred and $\Delta$ is the cumulative magnitude of drift experienced within $T$ rounds.
Keywords
All Related Versions
Funding Information
- Cisco Inc., Defense Advanced Research Projects Agency (W911NF2020003)
- National Science Foundation (CCF-2149588)
This publication has 17 references indexed in Scilit:
- Online Federated Learning2021
- Distributed Online Convex Optimization With an Aggregative VariableIEEE Transactions on Control of Network Systems, 2021
- Concept drift detection and adaptation for federated and continual learningMultimedia Tools and Applications, 2021
- A Comprehensive Survey of Privacy-preserving Federated LearningACM Computing Surveys, 2021
- ElStream: An Ensemble Learning Approach for Concept Drift Detection in Dynamic Social Big Data Stream LearningIEEE Access, 2021
- From Federated to Fog Learning: Distributed Machine Learning over Heterogeneous Wireless NetworksIEEE Communications Magazine, 2020
- Online Learning Over Dynamic Graphs via Distributed Proximal Gradient AlgorithmIEEE Transactions on Automatic Control, 2020
- Online Distributed Optimization With Strongly Pseudoconvex-Sum Cost FunctionsIEEE Transactions on Automatic Control, 2019
- Distributed Online Optimization in Dynamic Environments Using Mirror DescentIEEE Transactions on Automatic Control, 2017
- On the Generalization Ability of On-Line Learning AlgorithmsIEEE Transactions on Information Theory, 2004