Improving $(α, f)$-Byzantine Resilience in Federated Learning via layerwise aggregation and cosine distance
Preprint
Abstract
The rapid development of artificial intelligence systems has amplified societal concerns regarding their usage, necessitating regulatory frameworks that encompass data privacy. Federated Learning (FL) is posed as potential solution to data privacy challenges in distributed machine learning by enabling collaborative model training {without data sharing}. However, FL systems remain vulnerable to Byzantine attacks, where malicious nodes contribute corrupted model updates. While Byzantine Resilient operators have emerged as a widely adopted robust aggregation algorithm to mitigate these attacks, its efficacy diminishes significantly in high-dimensional parameter spaces, sometimes leading to poor performing models. This paper introduces Layerwise Cosine Aggregation, a novel aggregation scheme designed to enhance robustness of these rules in such high-dimensional settings while preserving computational efficiency. A theoretical analysis is presented, demonstrating the superior robustness of the proposed Layerwise Cosine Aggregation compared to original robust aggregation operators. Empirical evaluation across diverse image classification datasets, under varying data distributions and Byzantine attack scenarios, consistently demonstrates the improved performance of Layerwise Cosine Aggregation, achieving up to a 16% increase in model accuracy.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Mitigating backdoor attacks in Federated Learning based intrusion detection systems through Neuron Synaptic Weight AdjustmentKnowledge-Based Systems, 2025
- A Tutorial on Federated Learning from Theory to Practice: Foundations, Software Frameworks, Exemplary Use Cases, and Selected TrendsIEEE/CAA Journal of Automatica Sinica, 2024
- Manipulating vulnerability: Poisoning attacks and countermeasures in federated cloud–edge–client learning for image classificationKnowledge-Based Systems, 2022
- Survey on federated learning threats: Concepts, taxonomy on attacks and defences, experimental study and challengesInformation Fusion, 2022
- Backdoor attacks-resilient aggregation based on Robust Filtering of Outliers in federated learning for image classificationKnowledge-Based Systems, 2022
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Multivariate Estimation with High Breakdown Point1985
- The Byzantine Generals ProblemACM Transactions on Programming Languages and Systems, 1982