Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method
Preprint
Abstract
Large Language Models (LLMs) have shown great potential in Natural Language Processing (NLP) tasks. However, recent literature reveals that LLMs generate nonfactual responses intermittently, which impedes the LLMs' reliability for further utilization. In this paper, we propose a novel self-detection method to detect which questions that a LLM does not know that are prone to generate nonfactual results. Specifically, we first diversify the textual expressions for a given question and collect the corresponding answers. Then we examine the divergencies between the generated answers to identify the questions that the model may generate falsehoods. All of the above steps can be accomplished by prompting the LLMs themselves without referring to any other external resources. We conduct comprehensive experiments and demonstrate the effectiveness of our method on recently released LLMs, e.g., Vicuna, ChatGPT, and GPT-4.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Survey of Hallucination in Natural Language GenerationACM Computing Surveys, 2023
- Reducing Conversational Agents’ Overconfidence Through Linguistic CalibrationTransactions of the Association for Computational Linguistics, 2022
- FaVIQ: FAct Verification from Information-seeking Questions2022
- Can Explanations Be Useful for Calibrating Black Box Models?2022
- Recipes for Building an Open-Domain Chatbot2021
- Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases2021
- Are NLP Models really able to Solve Simple Math Word Problems?2021
- Calibration of Pre-trained Transformers2020
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension2020
- Calibrating Structured Output Predictors for Natural Language ProcessingProceedings of the conference. Association for Computational Linguistics. Meeting, 2020