FRUIT: Faithfully Reflecting Updated Information in Text
Authors:
- Robert L. Logan IvRIRobert L. Logan Iv
- Alexandre PassosAPAlexandre Passos
- Sameer SinghSSSameer Singh
- Ming-Wei ChangMCMing-Wei Chang
Preprint
Abstract
Textual knowledge bases such as Wikipedia require considerable effort to keep up to date and consistent. While automated writing assistants could potentially ease this burden, the problem of suggesting edits grounded in external knowledge has been under-explored. In this paper, we introduce the novel generation task of *faithfully reflecting updated information in text* (FRUIT) where the goal is to update an existing article given new evidence. We release the FRUIT-WIKI dataset, a collection of over 170K distantly supervised data produced from pairs of Wikipedia snapshots, along with our data generation pipeline and a gold evaluation set of 914 instances whose edits are guaranteed to be supported by the evidence. We provide benchmark results for popular generation systems as well as EDIT5 -- a T5-based approach tailored to editing we introduce that establishes the state of the art. Our analysis shows that developing models that can update articles faithfully requires new capabilities for neural generation models, and opens doors to many new applications.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- A Comprehensive Survey of Grammatical Error CorrectionACM Transactions on Intelligent Systems and Technology, 2021
- A Thorough Evaluation of Task-Specific Pretraining for Summarization2021
- IGA: An Intent-Guided Authoring Assistant2021
- Entity-Based Knowledge Conflicts in Question Answering2021
- WikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections2021
- DESCGEN: A Distantly Supervised Datasetfor Generating Entity Descriptions2021
- ToTTo: A Controlled Table-To-Text Generation Dataset2020
- ENT-DESC: Entity Description Generation by Exploring Knowledge Graph2020
- Gmail Smart Compose2019
- With Few Eyes, All Hoaxes are DeepProceedings of the ACM on Human-Computer Interaction, 2018