BadSampler: Harnessing the Power of Catastrophic Forgetting to Poison Byzantine-robust Federated Learning
Open Access
- 24 August 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1944-1955
Abstract
Federated Learning (FL) is susceptible to poisoning attacks, wherein compromised clients manipulate the global model by modifying local datasets or sending manipulated model updates. Experienced defenders can readily detect and mitigate the poisoning effects of malicious behaviors using Byzantine-robust aggregation rules. However, the exploration of poisoning attacks in scenarios where such behaviors are absent remains largely unexplored for Byzantine-robust FL. This paper addresses the challenging problem of poisoning Byzantine-robust FL by introducing catastrophic forgetting. To fill this gap, we first formally define generalization error and establish its connection to catastrophic forgetting, paving the way for the development of a clean-label data poisoning attack named BadSampler. This attack leverages only clean-label data (i.e., without poisoned data) to poison Byzantine-robust FL and requires the adversary to selectively sample training data with high loss to feed model training and maximize the model's generalization error. We formulate the attack as an optimization problem and present two elegant adversarial sampling strategies, Top-k sampling, and meta-sampling, to approximately solve it. Additionally, our formal error upper bound and time complexity analysis demonstrate that our design can preserve attack utility with high efficiency. Extensive evaluations on two real-world datasets illustrate the effectiveness and performance of our proposed attacks.Keywords
All Related Versions
Funding Information
- City University of Hong Kong (9678146 and 9678126)
- Hong Kong Research Grants Council (11218521, 11218322, R6021-20F, R1012-21, RFS2122-1S04, C2004-21G, C1029-22G, and N_CityU139/21)
This publication has 28 references indexed in Scilit:
- Oblivion: Poisoning Federated Learning by Inducing Catastrophic Forgetting2023
- More than Enough is Too Much: Adaptive Defenses against Gradient Leakage in Production Federated Learning2023
- Federated Learning on Non-IID Data Silos: An Experimental Study2022
- Romoa: Robust Model Aggregation for the Resistance of Federated Learning to Model Poisoning Attacks2021
- CONTRA: Defending Against Poisoning Attacks in Federated Learning2021
- Data Poisoning Attacks Against Federated Learning Systems2020
- Enabling Execution Assurance of Federated Learning at Untrusted Participants2020
- Self-paced Ensemble for Highly Imbalanced Massive Data Classification2020
- Byzantine-resilient Distributed Large-scale Matrix Completion2019
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences of the United States of America, 2017