Graph of Logic: Enhancing LLM Reasoning with Graphs and Symbolic Logic
Authors:
- 15 December 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5926-5935
Abstract
Large language models (LLMs) have been shown to struggle with complex logical reasoning tasks due to the inherent ambiguity and complexity of natural language. These challenges are further amplified when processing large and diverse datasets, increasing the likelihood of unfaithful reasoning and predictive hallucinations. However, LLMs can provide accurate responses when queries are clear and direct. Symbolic logic provides precise, well-defined rules that can help overcome ambiguity and support reasoning. In this work, we leverage symbolic logic’s precision to enhance LLMs’ logical reasoning capabilities by introducing the Graph of Logic (GoL) framework. GoL combines the power of graph structures with the strengths of LLMs and symbolic logic. GoL utilizes the precise rules of symbolic logic to infer new facts and detect LLM hallucinations effectively on complex datasets. Furthermore, GoL utilizes graph structures to support scalability for large datasets and tackle long dependencies, enabling efficient handling of complex reasoning tasks. We conduct extensive experiments across seven benchmark datasets, encompassing various types of reasoning. These include deductive, inductive, and abductive reasoning, each testing distinct aspects of logical inference. The experimental results demonstrate GoL’s advantage in improving the reasoning capabilities of LLMs. GoL outperforms the baselines with an average margin of 18.18% for the GPT-3.5 and GPT-4 models, outperforming the baselines for all datasets for the GPT-3.5 model and six out of seven datasets for the GPT-4 model1.Keywords
Funding Information
- National Science Foundation
This publication has 21 references indexed in Scilit:
- Are Large Language Models the New Interface for Data Pipelines?2024
- To prompt or not to prompt: Navigating the use of Large Language Models for integrating and modeling heterogeneous dataData & Knowledge Engineering, 2024
- Graph of Thoughts: Solving Elaborate Problems with Large Language ModelsProceedings of the AAAI Conference on Artificial Intelligence, 2024
- Large Language Models in Education: Vision and Opportunities2023
- Reasoning with Language Model Prompting: A Survey2023
- LAMBADA: Backward Chaining for Automated Reasoning in Natural Language2023
- AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts2022
- PromptChainer: Chaining Large Language Model Prompts through Visual Programming2022
- Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning2022
- Analytical Reasoning of Text2022