Detecting hallucinations in large language models using semantic entropy
Top Cited Papers
Open Access
- 19 June 2024
- journal article
- research article
- Published by Springer Nature in Nature
- Vol. 630 (8017) , 625-630
Abstract
Large language model (LLM) systems, such as ChatGPT1or Gemini2, can show impressive reasoning and question-answering capabilities but often ‘hallucinate’ false outputs and unsubstantiated answers3,4. Answering unreliably or without the necessary information prevents adoption in diverse fields, with problems including fabrication of legal precedents5or untrue facts in news articles6and even posing a risk to human life in medical domains such as radiology7. Encouraging truthfulness through supervision or reinforcement has been only partially successful8. Researchers need a general method for detecting hallucinations in LLMs that works even with new and unseen questions to which humans might not know the answer. Here we develop new methods grounded in statistics, proposing entropy-based uncertainty estimators for LLMs to detect a subset of hallucinations—confabulations—which are arbitrary and incorrect generations. Our method addresses the fact that one idea can be expressed in many ways by computing uncertainty at the level of meaning rather than specific sequences of words. Our method works across datasets and tasks without a priori knowledge of the task, requires no task-specific data and robustly generalizes to new tasks not seen before. By detecting when a prompt is likely to produce a confabulation, our method helps users understand when they must take extra care with LLMs and opens up new possibilities for using LLMs that are otherwise prevented by their unreliability.Keywords
This publication has 36 references indexed in Scilit:
- Talking about Large Language ModelsCommunications of the ACM, 2024
- Deep Deterministic Uncertainty: A New Simple Baseline2023
- Trustworthy journalism through AIData & Knowledge Engineering, 2023
- ChatGPT and Other Large Language Models Are Double-edged SwordsRadiology, 2023
- Survey of Hallucination in Natural Language GenerationACM Computing Surveys, 2023
- An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competitionBMC Bioinformatics, 2015
- Measuring machine translation quality as semantic equivalence: A metric based on entailment featuresMachine Translation, 2009
- Aleatory or epistemic? Does it matter?Structural Safety, 2009
- Confabulations: A Conceptual HistoryJournal of the History of the Neurosciences, 1998
- Information-Based Objective Functions for Active Data SelectionNeural Computation, 1992