Contrastive Error Attribution for Finetuned Language Models
Preprint
Abstract
Recent work has identified noisy and misannotated data as a core cause of hallucinations and unfaithful outputs in Natural Language Generation (NLG) tasks. Consequently, identifying and removing these examples is a key open challenge in creating reliable NLG systems. In this work, we introduce a framework to identify and remove low-quality training instances that lead to undesirable outputs, such as faithfulness errors in text summarization. We show that existing approaches for error tracing, such as gradient-based influence measures, do not perform reliably for detecting faithfulness errors in NLG datasets. We overcome the drawbacks of existing error tracing methods through a new, contrast-based estimate that compares undesired generations to human-corrected outputs. Our proposed method can achieve a mean average precision of 0.93 at detecting known data errors across synthetic tasks with known ground truth, substantially outperforming existing approaches. Using this approach and re-training models on cleaned data leads to a 70% reduction in entity hallucinations on the NYT dataset and a 55% reduction in semantic errors on the E2E dataset.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Entity-level Factual Consistency of Abstractive Text Summarization2021
- Improving Factual Consistency of Abstractive Summarization via Question Answering2021
- Quantifying Gender Bias in Different Corpora2020
- Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions2020
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension2020
- On Faithfulness and Factuality in Abstractive Summarization2020
- FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization2020
- Improved Natural Language Generation via Loss Truncation2020
- Evaluating the state-of-the-art of End-to-End Natural Language Generation: The E2E NLG challengeComputer Speech & Language, 2020
- Unbiased look at dataset bias2011