Survey of Hallucination in Natural Language Generation

Authors:
  • ZJZiwei Ji
    Ziwei Ji
  • NLNayeon Lee
    Nayeon Lee
  • RFRita Frieske
    Rita Frieske
  • TYTiezheng Yu
    Tiezheng Yu
  • DSDan Su
    Dan Su
  • YXYan Xu
    Yan Xu
  • EIEtsuko Ishii
    Etsuko Ishii
  • YBYejin Bang
    Yejin Bang
  • AMAndrea Madotto
    Andrea Madotto
  • PFPascale Fung
    Pascale Fung

Abstract
Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent natural language generation, naturally leading to development in downstream tasks such as abstractive summarization, dialogue generation and data-to-text generation. However, it is also investigated that such generation includes hallucinated texts, which makes the performances of text generation fail to meet users' expectations in many real-world scenarios. In order to address this issue, studies in evaluation and mitigation methods of hallucinations have been presented in various tasks, but have not been reviewed in a combined manner. In this survey, we provide a broad overview of the research progress and challenges in the hallucination problem of NLG. The survey is organized into two big divisions: (i) a general overview of metrics, mitigation methods, and future directions; (ii) task-specific research progress for hallucinations in a large set of downstream tasks: abstractive summarization, dialogue generation, generative question answering, data-to-text generation, and machine translation. This survey could facilitate collaborative efforts among researchers in these tasks.