Decentralized Event-Triggered Federated Learning with Heterogeneous Communication Thresholds
Open Access
- 6 December 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4680-4687
Abstract
A recent emphasis of distributed learning research has been on federated learning (FL), in which model training is conducted by the data-collecting devices. Existing research on FL has mostly focused on a star topology learning architecture with synchronized (time-triggered) model training rounds, where the local models of the devices are periodically aggregated by a centralized coordinating node. However, in many settings, such a coordinating node may not exist, motivating efforts to fully decentralize FL. In this work, we propose a novel methodology for distributed model aggregations via asynchronous, event-triggered consensus iterations over the network graph topology. We consider heterogeneous communication event thresholds at each device that weigh the change in local model parameters against the available local resources in deciding the benefit of aggregations at each iteration. Through theoretical analysis, we demonstrate that our methodology achieves asymptotic convergence to the globally optimal learning model under standard assumptions in distributed learning and graph consensus literature, and without restrictive connectivity requirements on the underlying topology. Subsequent numerical results demonstrate that our methodology obtains substantial improvements in communication requirements compared with FL baselines.Keywords
All Related Versions
Funding Information
- Office of Naval Research
This publication has 19 references indexed in Scilit:
- Federated Learning With Cooperating Devices: A Consensus Approach for Massive IoT NetworksIEEE Internet of Things Journal, 2020
- Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge2019
- Adaptive Federated Learning in Resource Constrained Edge Computing SystemsIEEE Journal on Selected Areas in Communications, 2019
- A Linear Algorithm for Optimization Over Directed Graphs With Geometric ConvergenceIEEE Control Systems Letters, 2018
- Fog and IoT: An Overview of Research OpportunitiesIEEE Internet of Things Journal, 2016
- Constrained Consensus and Optimization in Multi-Agent NetworksIEEE Transactions on Automatic Control, 2010
- Distributed Subgradient Methods for Multi-Agent OptimizationIEEE Transactions on Automatic Control, 2009
- Fast linear iterations for distributed averagingSystems & Control Letters, 2004
- Fastest Mixing Markov Chain on a GraphSIAM Review, 2004
- Distributed asynchronous deterministic and stochastic gradient optimization algorithmsIEEE Transactions on Automatic Control, 1986