Online Federated Learning via Non-Stationary Detection and Adaptation amidst Concept Drift
Authors:
- Bhargav GangulyBGBhargav Ganguly
- Vaneet AggarwalVAVaneet Aggarwal
Preprint
Abstract
Federated Learning (FL) is an emerging domain in the broader context of artificial intelligence research. Methodologies pertaining to FL assume distributed model training, consisting of a collection of clients and a server, with the main goal of achieving optimal global model with restrictions on data sharing due to privacy concerns. It is worth highlighting that the diverse existing literature in FL mostly assume stationary data generation processes; such an assumption is unrealistic in real-world conditions where concept drift occurs due to, for instance, seasonal or period observations, faults in sensor measurements. In this paper, we introduce a multiscale algorithmic framework which combines theoretical guarantees of \textit{FedAvg} and \textit{FedOMD} algorithms in near stationary settings with a non-stationary detection and adaptation technique to ameliorate FL generalization performance in the presence of model/concept drifts. We present a multi-scale algorithmic framework leading to $\Tilde{\mathcal{O}} ( \min \{ \sqrt{LT} , \Delta^{\frac{1}{3}}T^{\frac{2}{3}} + \sqrt{T} \})$ \textit{dynamic regret} for $T$ rounds with an underlying general convex loss function, where $L$ is the number of times non-stationary drifts occured and $\Delta$ is the cumulative magnitude of drift experienced within $T$ rounds.
Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- A Lightweight Concept Drift Detection and Adaptation Framework for IoT Data StreamsIEEE Internet of Things Magazine, 2021
- ElStream: An Ensemble Learning Approach for Concept Drift Detection in Dynamic Social Big Data Stream LearningIEEE Access, 2021
- Federated Learning for Internet of Things: A Comprehensive SurveyIEEE Communications Surveys & Tutorials, 2021
- Distributed anomaly detection using concept drift detection based hybrid ensemble techniques in streamed network dataCluster Computing, 2021
- Federated Learning for Vehicular Internet of Things: Recent Advances and Open IssuesIEEE Open Journal of the Computer Society, 2020
- A Distributed Online Convex Optimization Algorithm with Improved Dynamic Regret2019
- Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoTIEEE Internet of Things Journal, 2019
- Global Convergence of ADMM in Nonconvex Nonsmooth OptimizationJournal of Scientific Computing, 2018
- Distributed Online Optimization in Dynamic Environments Using Mirror DescentIEEE Transactions on Automatic Control, 2017
- Improving predictive inference under covariate shift by weighting the log-likelihood functionJournal of Statistical Planning and Inference, 2000