How Robust Are Language Models against Backdoors in Federated Learning?
Authors:
Open Access
- 1 January 2025
- journal article
- Published by Tech Science Press in Computer Modeling in Engineering & Sciences
- Vol. 145 (2) , 2617-2630
Abstract
Federated Learning enables privacy-preserving training of Transformer-based language models, but remains vulnerable to backdoor attacks that compromise model reliability. This paper presents a comparative analysis of defense strategies against both classical and advanced backdoor attacks, evaluated across autoencoding and autoregressive models. Unlike prior studies, this work provides the first systematic comparison of perturbation-based, screening-based, and hybrid defenses in Transformer-based FL environments. Our results show that screening-based defenses consistently outperform perturbation-based ones, effectively neutralizing most attacks across architectures. However, this robustness comes with significant computational overhead, revealing a clear trade-off between security and efficiency. By explicitly identifying this trade-off, our study advances the understanding of defense strategies in federated learning and highlights the need for lightweight yet effective screening methods for trustworthy deployment in diverse application domains.Keywords
This publication has 8 references indexed in Scilit:
- SDBA: A Stealthy and Long-Lasting Durable Backdoor Attack in Federated LearningIEEE Transactions on Dependable and Secure Computing, 2025
- Efficient Defense Against Adversarial Attacks on Multimodal Emotion AI ModelsIEEE Transactions on Computational Social Systems, 2025
- Federated Reinforcement Learning-Based Dynamic Resource Allocation and Task Scheduling in Edge for IoT ApplicationsSensors, 2025
- A Comprehensive Survey on Federated Learning Applications in Computational Mental HealthcareComputer Modeling in Engineering & Sciences, 2025
- Backdoor attacks and defenses in federated learning: Survey, challenges and future research directionsEngineering Applications of Artificial Intelligence, 2023
- FedBERT: When Federated Learning Meets Pre-trainingACM Transactions on Intelligent Systems and Technology, 2022
- BadNets: Evaluating Backdooring Attacks on Deep Neural NetworksIEEE Access, 2019
- Deep Learning with Differential Privacy2016