Retrieval-Augmented Generation for Natural Language Processing: A Survey

Authors:
  • SWShangyu Wu
    Shangyu Wu
  • YXYing Xiong
    Ying Xiong
  • YCYufei Cui
    Yufei Cui
  • HWHaolun Wu
    Haolun Wu
  • CCCan Chen
    Can Chen
  • YYYe Yuan
    Ye Yuan
  • LHLianming Huang
    Lianming Huang
  • XLXue Liu
    Xue Liu
  • TKTei-Wei Kuo
    Tei-Wei Kuo
  • NGNan Guan
    Nan Guan
  • CXChun Jason Xue
    Chun Jason Xue
Preprint
Publisher website
Google Scholar
Abstract
Large language models (LLMs) have demonstrated great success in various fields, benefiting from their huge amount of parameters that store knowledge. However, LLMs still suffer from several key issues, such as hallucination problems, knowledge update issues, and lacking domain-specific expertise. The appearance of retrieval-augmented generation (RAG), which leverages an external knowledge database to augment LLMs, makes up those drawbacks of LLMs. This paper reviews all significant techniques of RAG, especially in the retriever and the retrieval fusions. Besides, tutorial codes are provided for implementing the representative techniques in RAG. This paper further discusses the RAG training, including RAG with/without datastore update. Then, we introduce the application of RAG in representative natural language processing tasks and industrial scenarios. Finally, this paper discusses the future directions and challenges of RAG for promoting its development.

This publication has 21 references indexed in Scilit: