Balancing Forget Quality and Model Utility: A Reverse KL-Divergence Knowledge Distillation Approach for Better Unlearning in LLMs
Open Access
- 1 January 2025
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 1306-1321
Abstract
Bichen Wang, Yuzhe Zi, Yixin Sun, Yanyan Zhao, Bing Qin. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.Keywords
This publication has 8 references indexed in Scilit:
- Rethinking machine unlearning for large language modelsNature Machine Intelligence, 2025
- A Survey on Federated Unlearning: Challenges, Methods, and Future DirectionsACM Computing Surveys, 2024
- Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models2024
- Evaluating Copyright Takedown Methods for Language Models2024
- Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference2024
- SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning2024
- Knowledge Unlearning for Mitigating Privacy Risks in Language Models2023
- QUARK: Controllable Text Generation with Reinforced Unlearning2022