Membership Inference Attacks Against In-Context Learning
Open Access
- 2 December 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 3481-3495
Abstract
Adapting Large Language Models (LLMs) to specific tasks introduces concerns about computational efficiency, prompting an exploration of efficient methods such as In-Context Learning (ICL). However, the vulnerability of ICL to privacy attacks under realistic assumptions remains largely unexplored. In this work, we present the first membership inference attack tailored for ICL, relying solely on generated texts without their associated probabilities. We propose four attack strategies tailored to various constrained scenarios and conduct extensive experiments on four popular large language models. Empirical results show that our attacks can accurately determine membership status in most cases, e.g., 95% accuracy advantage against LLaMA, indicating that the associated risks are much higher than those shown by existing probability-based attacks. Additionally, we propose a hybrid attack that synthesizes the strengths of the aforementioned strategies, achieving an accuracy advantage of over 95% in most cases. Furthermore, we investigate three potential defenses targeting data, instruction, and output. Results demonstrate combining defenses from orthogonal dimensions significantly reduces privacy leakage and offers enhanced privacy assurances.Keywords
All Related Versions
Funding Information
- European Health and Digital Executive Agency (101057917)
- BMBF (16KISAO29K)
This publication has 21 references indexed in Scilit:
- SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning2023
- Membership Inference Attacks against Language Models via Neighbourhood Comparison2023
- SummQA at MEDIQA-Chat 2023: In-Context Learning with GPT-4 for Medical Summarization2023
- Enhanced Membership Inference Attacks against Machine Learning Models2022
- Membership Inference Attacks by Exploiting Loss Trajectory2022
- Privacy Risks of Securing Machine Learning Models against Adversarial Examples2019
- Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning2019
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks2019
- Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting2018
- Learning question classifiers: the role of semantic informationNatural Language Engineering, 2005