Metacognitive Retrieval-Augmented Large Language Models
Open Access
- 13 May 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1453-1463
Abstract
Retrieval-augmented generation have become central in natural language processing due to their efficacy in generating factual content. While traditional methods employ single-time retrieval, more recent approaches have shifted towards multi-time retrieval for multi-hop reasoning tasks. However, these strategies are bound by predefined reasoning steps, potentially leading to inaccuracies in response generation. This paper introduces MetaRAG, an approach that combines the retrieval-augmented generation process with metacognition. Drawing from cognitive psychology, metacognition allows an entity to self-reflect and critically evaluate its cognitive processes. By integrating this, MetaRAG enables the model to monitor, evaluate, and plan its response strategies, enhancing its introspective reasoning abilities. Through a three-step metacognitive regulation pipeline, the model can identify inadequacies in initial cognitive responses and fixes them. Empirical evaluations show that MetaRAG significantly outperforms existing methods.Keywords
All Related Versions
Funding Information
- National Key R&D Program of China (2022ZD0120103)
- National Natural Science Foundation of China (62272467)
- Beijing Natural Science Foundation (L233008)
This publication has 23 references indexed in Scilit:
- UniGen: A Unified Generative Framework for Retrieval and Question Answering with Large Language ModelsProceedings of the AAAI Conference on Artificial Intelligence, 2024
- TRUE: Re-evaluating Factual Consistency Evaluation2022
- GLM: General Language Model Pretraining with Autoregressive Blank Infilling2022
- SpanBERT: Improving Pre-training by Representing and Predicting SpansTransactions of the Association for Computational Linguistics, 2020
- Encoding History with Context-aware Representation Learning for Personalized Search2020
- Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps2020
- On Faithfulness and Factuality in Abstractive Summarization2020
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019
- The Probabilistic Relevance Framework: BM25 and BeyondFoundations and Trends® in Information Retrieval, 2009
- Metacognitive theoriesEducational Psychology Review, 1995