Preprint
Abstract
Fact checking at scale is difficult -- while the number of active fact checking websites is growing, it remains too small for the needs of the contemporary media ecosystem. However, despite good intentions, contributions from volunteers are often error-prone, and thus in practice restricted to claim detection. We investigate how to increase the accuracy and efficiency of fact checking by providing information about the claim before performing the check, in the form of natural language briefs. We investigate passage-based briefs, containing a relevant passage from Wikipedia, entity-centric ones consisting of Wikipedia pages of mentioned entities, and Question-Answering Briefs, with questions decomposing the claim, and their answers. To produce QABriefs, we develop QABriefer, a model that generates a set of questions conditioned on the claim, searches the web for evidence, and generates answers. To train its components, we introduce QABriefDataset which we collected via crowdsourcing. We show that fact checking with briefs -- in particular QABriefs -- increases the accuracy of crowdworkers by 10% while slightly decreasing the time taken. For volunteer (unpaid) fact checkers, QABriefs slightly increase accuracy and reduce the time required by around 20%.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Fake news, fast and slow: Deliberation reduces belief in false (but not true) news headlines.Journal of Experimental Psychology: General, 2020
- Scalable Zero-shot Entity Linking with Dense Entity Retrieval2020
- Dense Passage Retrieval for Open-Domain Question Answering2020
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension2020
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019
- DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning2018
- Improving Large-Scale Fact-Checking using Decomposable Attention Models and Lexical Tagging2018
- Learning Together Slowly: Bayesian Learning about Political FactsThe Journal of Politics, 2017
- Fact-Checking Polarized Politics: Does The Fact-Check Industry Provide Consistent Guidance on Disputed Realities?The Forum, 2015
- Checking the Fact-Checkers in 2008: Predicting Political Ad Scrutiny and Assessing ConsistencyJournal of Political Marketing, 2014