A Fine-Tuning Data Recovery Attack on Generative Language Models via Backdooring
- 5 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 21 (15566013) , 3006-3021
Abstract
Generative language models (GLMs) are increasingly integrated into modern intelligent applications to power intelligent functionalities. Developers often fine-tune open-source GLMs on proprietary data and deploy them in real-world applications. In this paper, we reveal a novel model supply chain attack that exploits this workflow: by injecting backdoors into the source code of an open-source GLM, an adversary can induce the model to memorize fine-tuning data and later regenerate it via crafted prompts. We propose LURE, a new backdoor-based data recovery attack that exploits memorization capabilities of fine-tuned models. During fine-tuning, LURE stealthily injects unique and attacker-enumerable hash prompts, and incorporates a Position-Decay Weighted Aligned Cross-Entropy Loss into the original fine-tuning loss, strengthening the association between injected prompts and corresponding data samples for effective data recovery. To achieve stealthy and transparent attack injection, LURE employs a stealthy backdoor within the model’s source code, enabling automatic injection of hash prompts during fine-tuning and thus maintaining the user’s original fine-tuning workflow. LURE also proposes several optimizations to maintain minimal impact on the performance of the original task and external training state. Extensive evaluations demonstrate the remarkable efficacy of LURE, achieving a 45%-68% data recovery rate while maintaining the attack’s transparency, stealthiness, and showcasing its ability to evade existing defenses.Keywords
Funding Information
- National Natural Science Foundation of China (62402267)
- National Natural Science Foundation of China (62432004)
- National Natural Science Foundation of China (62372472)
- National Natural Science Foundation of China (62306109)
- Key Research and Development Program of Hunan Province of China (2023SK2020)
- Natural Science Foundation of Hainan Province (2024JJ4068)
- Guoqiang Institute, Tsinghua University
This publication has 34 references indexed in Scilit:
- RoPe-Door: Toward Robust and Persistent Backdoor Data Poisoning Attacks in Federated LearningIEEE Network, 2024
- Analyzing Leakage of Personally Identifiable Information in Language Models2023
- Hardly Perceptible Trojan Attack Against Neural Networks with Bit Flips2022
- See through Gradients: Image Batch Recovery via GradInversion2021
- Backdoor Attacks Against Deep Learning Systems in the Physical World2021
- Deep Leakage from Gradients2020
- Exploiting Unintended Feature Leakage in Collaborative Learning2019
- Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting2018
- Membership Inference Attacks Against Machine Learning Models2017
- The Enron Corpus: A New Dataset for Email Classification Research2004