Uncertainty in Natural Language Processing: Sources, Quantification, and Applications
Preprint
Abstract
As a main field of artificial intelligence, natural language processing (NLP) has achieved remarkable success via deep neural networks. Plenty of NLP tasks have been addressed in a unified manner, with various tasks being associated with each other through sharing the same paradigm. However, neural networks are black boxes and rely on probability computation. Making mistakes is inevitable. Therefore, estimating the reliability and trustworthiness (in other words, uncertainty) of neural networks becomes a key research direction, which plays a crucial role in reducing models' risks and making better decisions. Therefore, in this survey, we provide a comprehensive review of uncertainty-relevant works in the NLP field. Considering the data and paradigms characteristics, we first categorize the sources of uncertainty in natural language into three types, including input, system, and output. Then, we systemically review uncertainty quantification approaches and the main applications. Finally, we discuss the challenges of uncertainty estimation in NLP and discuss potential future directions, taking into account recent trends in the field. Though there have been a few surveys about uncertainty estimation, our work is the first to review uncertainty from the NLP perspective.Keywords
All Related Versions
This publication has 78 references indexed in Scilit:
- A survey of uncertainty in deep neural networksArtificial Intelligence Review, 2023
- Paradigm Shift in Natural Language ProcessingMachine Intelligence Research, 2022
- LTP: A New Active Learning Strategy for CRF-Based Named Entity RecognitionNeural Processing Letters, 2022
- Algorithmic Learning in a Random World2022
- Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methodsMachine Learning, 2021
- A Survey on Deep Learning for Named Entity RecognitionIEEE Transactions on Knowledge and Data Engineering, 2020
- Uncertain Ontology-Aware Knowledge Graph EmbeddingsCommunications in Computer and Information Science, 2020
- GTransE: Generalizing Translation-Based Model on Uncertain Knowledge Graph Embedding2020
- Neural sentence embedding using only in-domain sentences for out-of-domain sentence detection in dialog systemsPattern Recognition Letters, 2017
- Evaluating Evaluation Methods for Generation in the Presence of Variation2005