MUter: Machine Unlearning on Adversarially Trained Models
Authors:
- 1 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4869-4879
Abstract
Machine unlearning is an emerging task of removing the influence of selected training datapoints from a trained model upon data deletion requests, which echoes the widely enforced data regulations mandating the Right to be Forgotten. Many unlearning methods have been proposed recently, achieving significant efficiency gains over the naive baseline of retraining from scratch. However, existing methods focus exclusively on unlearning from standard training models and do not apply to adversarial training models (ATMs) despite their popularity as effective defenses against adversarial examples. During adversarial training, the training data are involved in not only an outer loop for minimizing the training loss, but also an inner loop for generating the adversarial perturbation. Such bi-level optimization greatly complicates the influence measure for the data to be deleted and renders the unlearning more challenging than standard model training with single-level optimization. This paper proposes a new approach called MUter for unlearning from ATMs. We derive a closed-form unlearning step underpinned by a total Hessian-related data influence measure, while existing methods can mis-capture the data influence associated with the indirect Hessian part. We further alleviate the computational cost by introducing a series of approximations and conversions to avoid the most computationally demanding parts of Hessian inversions. The efficiency and effectiveness of MUter have been validated through experiments on four datasets using both linear and neural network models.Keywords
Funding Information
- National Science Foundation
- National Key Research and Development Program of China
- National Science Foundation
- Health
This publication has 40 references indexed in Scilit:
- ERM-KTP: Knowledge-Level Machine Unlearning via Knowledge Transfer2023
- Learning with Recoverable Forgetting2022
- Towards Efficient Adversarial Training on Vision Transformers2022
- Deep Unlearning via Randomized Conditionally Independent HessiansConference on Computer Vision and Pattern Recognition (CVPR), 2022
- Mixed-Privacy Forgetting in Deep Networks2021
- Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations2020
- Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks2020
- Understanding adversarial training: Increasing local stability of supervised models through robust optimizationNeurocomputing, 2018
- SoK: Security and Privacy in Machine Learning2018
- The Theory of Max-Min and its Application to Weapons Allocation Problems1967