MindMap: Constructing Evidence Chains for Multi-Step Reasoning in Large Language Models
Authors:
Open Access
- 24 March 2024
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (17) , 19270-19278
Abstract
Large language models (LLMs) have demonstrated remarkable performance in various natural language processing tasks. However, they still face significant challenges in automated reasoning, particularly in scenarios involving multi-step reasoning. In this paper, we focus on the logical reasoning problem. The main task is to answer a question based on a set of available facts and rules. A lot of work has focused on guiding LLMs to think logically by generating reasoning paths, ignoring the structure among available facts. In this paper, we propose a simple approach MindMap by introducing evidence chains for supporting reasoning. An evidence chain refers to a set of facts that involve the same subject. In this way, we can organize related facts together to avoid missing important information. MindMap can be integrated with existing reasoning framework, such as Chain-of-Thought (CoT) and Selection-Inference (SI), by letting the model select relevant evidence chains instead of independent facts. The experimental results on the bAbI and ProofWriterOWA datasets demonstrate the effectiveness of MindMap.It can significantly improve CoT and SI, especially in multi-step reasoning tasks.Keywords
This publication has 6 references indexed in Scilit:
- Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and BeyondACM Transactions on Knowledge Discovery From Data, 2024
- Graph of Thoughts: Solving Elaborate Problems with Large Language ModelsProceedings of the AAAI Conference on Artificial Intelligence, 2024
- FOLIO: Natural Language Reasoning with First-Order Logic2024
- A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity2023
- Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers2023
- Reward is enoughArtificial Intelligence, 2021