Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning
Preprint
Abstract
Large Language Models (LLMs) have shown human-like reasoning abilities but still struggle with complex logical problems. This paper introduces a novel framework, Logic-LM, which integrates LLMs with symbolic solvers to improve logical problem-solving. Our method first utilizes LLMs to translate a natural language problem into a symbolic formulation. Afterward, a deterministic symbolic solver performs inference on the formulated problem. We also introduce a self-refinement module, which utilizes the symbolic solver's error messages to revise symbolic formalizations. We demonstrate Logic-LM's effectiveness on five logical reasoning datasets: ProofWriter, PrOntoQA, FOLIO, LogicalDeduction, and AR-LSAT. On average, Logic-LM achieves a significant performance boost of 39.2% over using LLM alone with standard prompting and 18.4% over LLM with chain-of-thought prompting. Our findings suggest that Logic-LM, by combining LLMs with symbolic logic, offers a promising avenue for faithful logical reasoning. Code and data are publicly available at https://github.com/teacherpeterpan/Logic-LLM.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Weakly Supervised Neural Symbolic Learning for Cognitive TasksProceedings of the AAAI Conference on Artificial Intelligence, 2022
- Fifty Years of Prolog and BeyondTheory and Practice of Logic Programming, 2022
- Neural probabilistic logic programming in DeepProbLogArtificial Intelligence, 2021
- ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language2021
- The Lean Theorem Prover (System Description)2015
- Z3: An Efficient SMT Solver2008
- Markov logic networksMachine Learning, 2006
- Expert systems in production planning and scheduling: A state-of-the-art surveyJournal of Intelligent Manufacturing, 2002
- Isabelle1994
- A Machine-Oriented Logic Based on the Resolution PrincipleJournal of the ACM, 1965