Separate the Wheat from the Chaff: Model Deficiency Unlearning via Parameter-Efficient Module Operation
Open Access
- 24 March 2024
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (16) , 18252-18260
Abstract
Large language models (LLMs) have been widely used in various applications but are known to suffer from issues related to untruthfulness and toxicity. While parameter-efficient modules (PEMs) have demonstrated their effectiveness in equipping models with new skills, leveraging PEMs for deficiency unlearning remains underexplored. In this work, we propose a PEMs operation approach, namely Extraction-before-Subtraction (Ext-Sub), to enhance the truthfulness and detoxification of LLMs through the integration of ``expert'' PEM and ``anti-expert'' PEM. Remarkably, even anti-expert PEM possess valuable capabilities due to their proficiency in generating fabricated content, which necessitates language modeling and logical narrative competence. Rather than merely negating the parameters, our approach involves extracting and eliminating solely the deficiency capability within anti-expert PEM while preserving the general capabilities. To evaluate the effectiveness of our approach in terms of truthfulness and detoxification, we conduct extensive experiments on LLMs, encompassing additional abilities such as language modelling and mathematical reasoning. Our empirical results demonstrate that our approach effectively improves truthfulness and detoxification, while largely preserving the fundamental abilities of LLMs.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization2024
- Survey of Hallucination in Natural Language GenerationACM Computing Surveys, 2023
- Evaluating Verifiability in Generative Search Engines2023
- Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space2022
- Editing Factual Knowledge in Language Models2021
- DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts2021
- Prefix-Tuning: Optimizing Continuous Prompts for Generation2021
- Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification2019