Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning
Open Access
- 1 January 2024
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 9386-9406
Abstract
Yang Zhao, Li Du, Xiao Ding, Kai Xiong, Zhouhao Sun, Shi Jun, Ting Liu, Bing Qin. Findings of the Association for Computational Linguistics ACL 2024. 2024.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Training data influence analysis and estimation: a surveyMachine Learning, 2024
- Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge2024
- Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research2024
- AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models2024
- A Pretrainer’s Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity2024
- Self-Instruct: Aligning Language Models with Self-Generated Instructions2023
- Knowledge Unlearning for Mitigating Privacy Risks in Language Models2023
- Amnesiac Machine LearningProceedings of the AAAI Conference on Artificial Intelligence, 2021
- Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering2018
- Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization2018