Byzantine Machine Learning: A Primer
Open Access
- 9 April 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 56 (7) , 1-39
Abstract
The problem of Byzantine resilience in distributed machine learning, a.k.a. Byzantine machine learning , consists of designing distributed algorithms that can train an accurate model despite the presence of Byzantine nodes—that is, nodes with corrupt data or machines that can misbehave arbitrarily. By now, many solutions to this important problem have been proposed, most of which build upon the classical stochastic gradient descent scheme. Yet, the literature lacks a unified structure of this emerging field. Consequently, the general understanding on the principles of Byzantine machine learning remains poor. This article addresses this issue by presenting a primer on Byzantine machine learning. In particular, we introduce three pillars of Byzantine machine learning, namely the concepts of breakdown point , robustness , and gradient complexity , to curate the efficacy of a solution. The introduced systematization enables us to (i) bring forth the merits and limitations of the state-of-the-art solutions, and (ii) pave a clear path for future advancements in this field.Keywords
Funding Information
- Swiss National Science Foundation (200021_200477)
This publication has 117 references indexed in Scilit:
- Attack-Resilient State Estimation for Noisy Dynamical SystemsIEEE Transactions on Control of Network Systems, 2016
- Datatrust: Or, the political quest for numerical evidence and the epistemologies of Big DataBig Data & Society, 2016
- Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and ApplicationsIEEE Communications Surveys & Tutorials, 2014
- Resilient Asymptotic Consensus in Robust NetworksIEEE Journal on Selected Areas in Communications, 2013
- Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic ProgrammingSIAM Journal on Optimization, 2013
- A survey of methods for distributed machine learningProgress in Artificial Intelligence, 2012
- Iterative approximate byzantine consensus in arbitrary directed graphs2012
- Dual Averaging for Distributed Optimization: Convergence Analysis and Network ScalingIEEE Transactions on Automatic Control, 2011
- Impossibility of distributed consensus with one faulty processJournal of the ACM, 1985
- A Note on Estimation from a Cauchy SampleJournal of the American Statistical Association, 1964