Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models
Preprint
Abstract
The increased deployment of LMs for real-world tasks involving knowledge and facts makes it important to understand model epistemology: what LMs think they know, and how their attitudes toward that knowledge are affected by language use in their inputs. Here, we study an aspect of model epistemology: how epistemic markers of certainty, uncertainty, or evidentiality like "I'm sure it's", "I think it's", or "Wikipedia says it's" affect models, and whether they contribute to model failures. We develop a typology of epistemic markers and inject 50 markers into prompts for question answering. We find that LMs are highly sensitive to epistemic markers in prompts, with accuracies varying more than 80%. Surprisingly, we find that expressions of high certainty result in a 7% decrease in accuracy as compared to low certainty expressions; similarly, factive verbs hurt performance, while evidentials benefit performance. Our analysis of a popular pretraining dataset shows that these markers of uncertainty are associated with answers on question-answering websites, while markers of certainty are associated with questions. These associations may suggest that the behavior of LMs is based on mimicking observed language use, rather than truly reflecting epistemic uncertainty.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Calibrated Interpretation: Confidence Estimation in Semantic ParsingTransactions of the Association for Computational Linguistics, 2023
- Reducing Conversational Agents’ Overconfidence Through Linguistic CalibrationTransactions of the Association for Computational Linguistics, 2022
- He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on PragmaticsTransactions of the Association for Computational Linguistics, 2021
- How Can We Know When Language Models Know? On the Calibration of Language Models for Question AnsweringTransactions of the Association for Computational Linguistics, 2021
- How machine-learning recommendations influence clinician treatment selections: the example of antidepressant selectionTranslational Psychiatry, 2021
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019
- Definitely, maybe: A new experimental paradigm for investigating the pragmatics of evidential devices across languagesJournal of Pragmatics, 2019
- The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems2015
- Did It Happen? The Pragmatic Complexity of Veridicality AssessmentComputational Linguistics, 2012
- Stance and engagement: a model of interaction in academic discourseDiscourse Studies, 2005