Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning
Authors:
- 19 May 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3257-3275
Abstract
Machine unlearning has become a promising solution for fulfilling the "right to be forgotten", under which individuals can request the deletion of their data from machine learning models. However, existing studies of machine unlearning mainly focus on the efficacy and efficiency of unlearning methods, while neglecting the investigation of the privacy vulnerability during the unlearning process. With two versions of a model available to an adversary, that is, the original model and the unlearned model, machine unlearning opens up a new attack surface. In this paper, we conduct the first investigation to understand the extent to which machine unlearning can leak the confidential content of the unlearned data. Specifically, under the Machine Learning as a Service setting, we propose unlearning inversion attacks that can reveal the feature and label information of an unlearned sample by only accessing the original and unlearned model. The effectiveness of the proposed unlearning inversion attacks is evaluated through extensive experiments on benchmark datasets across various model architectures and on both exact and approximate representative unlearning approaches. The experimental results indicate that the proposed attack can reveal the sensitive information of the unlearned data. As such, we identify three possible defenses that help to mitigate the proposed attacks, while at the cost of reducing the utility of the unlearned model. The study in this paper uncovers an underexplored gap between machine unlearning and the privacy of unlearned data, highlighting the need for the careful design of mechanisms for implementing unlearning without leaking the information of the unlearned data.Keywords
All Related Versions
Funding Information
- Australian Research Council
This publication has 33 references indexed in Scilit:
- SoK: Model Inversion Attack Landscape: Taxonomy, Challenges, and Future Roadmap2023
- See through Gradients: Image Batch Recovery via GradInversion2021
- Label Leakage from Gradients in Distributed Machine Learning2021
- A Pragmatic Approach to Membership Inferences on Machine Learning Models2020
- QEBA: Query-Efficient Boundary-Based Blackbox Attack2020
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- Pyramid Scene Parsing Network2017
- Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset2017
- The Limitations of Deep Learning in Adversarial Settings2016
- Representations of quasi-Newton matrices and their use in limited memory methodsMathematical Programming, 1994