"The Data Says Otherwise" — Towards Automated Fact-checking and Communication of Data Claims
Open Access
- 11 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-20
Abstract
Fact-checking data claims requires data evidence retrieval and analysis, which can become tedious and intractable when done manually. This work presents Aletheia, an automated fact-checking prototype designed to facilitate data claims verification and enhance data evidence communication. For verification, we utilize a pre-trained LLM to parse the semantics for evidence retrieval. To effectively communicate the data evidence, we design representations in two forms: data tables and visualizations, tailored to various data fact types. Additionally, we design interactions that showcase a real-world application of these techniques. We evaluate the performance of two core NLP tasks with a curated dataset comprising 400 data claims and compare the two representation forms regarding viewers’ assessment time, confidence, and preference via a user study with 20 participants. The evaluation offers insights into the feasibility and bottlenecks of using LLMs for data fact-checking tasks, potential advantages and disadvantages of using visualizations over data tables, and design recommendations for presenting data evidence.Keywords
All Related Versions
This publication has 80 references indexed in Scilit:
- The state of human-centered NLP technology for fact-checkingInformation Processing & Management, 2022
- Unsupervised Matching of Data and Text2022
- Why Authors Don't Visualize UncertaintyIEEE Transactions on Visualization and Computer Graphics, 2019
- Extracting Statistical Mentions from Textual Claims to Provide Trusted Content2019
- Augmenting Visualizations with Interactive Data Facts to Facilitate Interpretation and CommunicationIEEE Transactions on Visualization and Computer Graphics, 2018
- THE PROMISE OF COMPUTATIONAL JOURNALISMJournalism Practice, 2011
- Toward effective insight management in visual analytics systems2009
- The Value of Information Visualization2008
- Toward measuring visualization insightIEEE Computer Graphics and Applications, 2006
- Low-level components of analytic activity in information visualization2005