ERM-KTP: Knowledge-Level Machine Unlearning via Knowledge Transfer
Authors:
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 20147-20155
Abstract
Machine unlearning can fortify the privacy and security of machine learning applications. Unfortunately, the exact unlearning approaches are inefficient, and the approximate unlearning approaches are unsuitable for complicated CNNs. Moreover, the approximate approaches have serious security flaws because even unlearning completely different data points can produce the same contribution estimation as unlearning the target data points. To address the above problems, we try to define machine unlearning from the knowledge perspective, and we propose a knowledge-level machine unlearning method, namely ERM-KTP. Specifically, we propose an entanglement-reduced mask (ERM) structure to reduce the knowledge entanglement among classes during the training phase. When receiving the un-learning requests, we transfer the knowledge of the non-target data points from the original model to the unlearned model and meanwhile prohibit the knowledge of the target data points via our proposed knowledge transfer and prohibition (KTP) method. Finally, we will get the un-learned model as the result and delete the original model to accomplish the unlearning process. Especially, our proposed ERM-KTP is an interpretable unlearning method because the ERM structure and the crafted masks in KTP can explicitly explain the operation and the effect of un-learning data points. Extensive experiments demonstrate the effectiveness, efficiency, high fidelity, and scalability of the ERM-KTP unlearning method. Code is available at https://github.com/RUIYUN-ML/ERM-KTPKeywords
Funding Information
- National Natural Science Foundation of China (61960206014,62102300,62121001)
- 111 Project (B16037)
- Fundamental Research Funds for the Central Universities (XJS211510)
This publication has 9 references indexed in Scilit:
- Membership Inference Attacks on Machine Learning: A SurveyACM Computing Surveys, 2022
- Amnesiac Machine LearningProceedings of the AAAI Conference on Artificial Intelligence, 2021
- Machine Unlearning2021
- Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks2020
- Interpretation of Neural Networks Is FragileProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Aggregated Residual Transformations for Deep Neural Networks2017
- Membership Inference Attacks Against Machine Learning Models2017
- Deep Residual Learning for Image Recognition2016
- Learning Deep Features for Discriminative Localization2016