Know Your Limits: A Survey of Abstention in Large Language Models
Authors:
Top Cited Papers
Open Access
- 27 June 2025
- journal article
- Published by MIT Press in Transactions of the Association for Computational Linguistics
- Vol. 13, 529-556
Abstract
Abstention, the refusal of large language models (LLMs) to provide an answer, is increasingly recognized for its potential to mitigate hallucinations and enhance safety in LLM systems. In this survey, we introduce a framework to examine abstention from three perspectives: the query, the model, and human values. We organize the literature on abstention methods, benchmarks, and evaluation metrics using this framework, and discuss merits and limitations of prior work. We further identify and motivate areas for future research, such as whether abstention can be achieved as a meta-capability that transcends specific tasks or domains, and opportunities to optimize abstention abilities in specific contexts. In doing so, we aim to broaden the scope and impact of abstention methodologies in AI systems.1Keywords
All Related Versions
This publication has 79 references indexed in Scilit:
- "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust2024
- Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings2024
- “As an AI language model, I cannot”: Investigating LLM Denials of User Requests2024
- Defending ChatGPT against jailbreak attack via self-remindersNature Machine Intelligence, 2023
- Jailbreaker in Jail: Moving Target Defense for Large Language Models2023
- Survey of Hallucination in Natural Language GenerationACM Computing Surveys, 2023
- Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectly2022
- BDDR: An Effective Defense Against Textual Backdoor AttacksComputers & Security, 2021
- Asking Clarifying Questions in Open-Domain Information-Seeking Conversations2019
- Rethinking the Inception Architecture for Computer Vision2016