An Efficient Rehearsal Scheme for Catastrophic Forgetting Mitigation during Multi-stage Fine-tuning
Open Access
- 1 January 2025
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 2557-2569
Abstract
Andrew Bai, Chih-Kuan Yeh, Cho-Jui Hsieh, Ankur Taly. Findings of the Association for Computational Linguistics: NAACL 2025. 2025.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Mitigating the Alignment Tax of RLHF2024
- Semantic segmentation using Vision Transformers: A surveyEngineering Applications of Artificial Intelligence, 2023
- Large-scale Lifelong Learning of In-context Instructions and How to Tackle It2023
- Rehearsal revealed: The limits and merits of revisiting samples in continual learning2021
- How Much Knowledge Can You Pack Into the Parameters of a Language Model?2020
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Text Summarization with Pretrained Encoders2019
- Know What You Don’t Know: Unanswerable Questions for SQuAD2018