Client-Free Federated Unlearning via Training Reconstruction with Anchor Subspace Calibration
Authors:
- 14 April 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9281-9285
Abstract
Federated learning (FL) model usually needs to forget what it has learned from a certain client for various considerations, which gives birth to the federated unlearning (FU) technique. Due to the distributed nature of FL, removing a specific client’s contribution from the global model potentially requires the cooperation of all participants, making FU difficult to apply in real-world scenarios. This paper proposes a simple-yet-effective client-free FU algorithm that runs solely on the central server. The algorithm utilizes the cached historical updates of the initial training from clients to rebuild the training after excluding the target client. To circumvent the issue of adaptivity, which is the key challenge for training reconstruction, we leverage the low-dimensional structure of gradient space in deep networks. Specifically, we propose to project the historical gradients to a low-dimensional subspace, which is given by the top gradient eigenspace on a small public dataset. According to experiments on three canonical datasets, our method achieves efficient unlearning while also preserving a high-level model utility.Keywords
Funding Information
- State Grid Corporation of China
- National Natural Science Foundation of China
This publication has 10 references indexed in Scilit:
- Machine unlearning: linear filtration for logit-based classifiersMachine Learning, 2022
- Federated Knowledge Distillation2022
- Federated Learning for Healthcare: Systematic Review and Architecture ProposalACM Transactions on Intelligent Systems and Technology, 2022
- FedEraser: Enabling Efficient Client-Level Data Removal from Federated Learning Models2021
- Amnesiac Machine LearningProceedings of the AAAI Conference on Artificial Intelligence, 2021
- Federated Learning for Open Banking2020
- Federated Learning: Challenges, Methods, and Future DirectionsIEEE Signal Processing Magazine, 2020
- Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization2020
- Deep Residual Learning for Image Recognition2016
- Deep Learning Face Attributes in the Wild2015