Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts
Authors:
- Jürgen MüllerJMJürgen Müller
Preprint
Abstract
The field of natural language processing (NLP) has witnessed significant progress in recent years, with a notable focus on improving large language models' (LLM) performance through innovative prompting techniques. Among these, prompt engineering coupled with structures has emerged as a promising paradigm, with designs such as Chain-of-Thought, Tree of Thoughts, or Graph of Thoughts, in which the overall LLM reasoning is guided by a structure such as a graph. As illustrated with numerous examples, this paradigm significantly enhances the LLM's capability to solve numerous tasks, ranging from logical or mathematical reasoning to planning or creative writing. To facilitate the understanding of this growing field and pave the way for future developments, we devise a general blueprint for effective and efficient LLM reasoning schemes. For this, we conduct an in-depth analysis of the prompt execution pipeline, clarifying and clearly defining different concepts. We then build the first taxonomy of structure-enhanced LLM reasoning schemes. We focus on identifying fundamental classes of harnessed structures, and we analyze the representations of these structures, algorithms executed with these structures, and many others. We refer to these structures as reasoning topologies, because their representation becomes to a degree spatial, as they are contained within the LLM context. Our study compares existing prompting schemes using the proposed taxonomy, discussing how certain design choices lead to different patterns in performance and cost. We also outline theoretical underpinnings, relationships between prompting and other parts of the LLM ecosystem such as knowledge bases, and the associated research challenges. Our work will help to advance future prompt engineering techniques.Keywords
All Related Versions
This publication has 47 references indexed in Scilit:
- GraphLLM: Boosting Graph Reasoning Ability of Large Language ModelIEEE Transactions on Big Data, 2025
- ControlLLM: Augment Language Models with Tools by Searching on Graphs2024
- LLMs for knowledge graph construction and reasoning: recent capabilities and future opportunitiesWorld Wide Web, 2024
- Tree of Uncertain Thoughts Reasoning for Large Language Models2024
- Integrating Graphs With Large Language Models: Methods and ProspectsIEEE Intelligent Systems, 2024
- Single-cell biological network inference using a heterogeneous graph transformerNature Communications, 2023
- Graph neural networks: A review of methods and applicationsAI Open, 2020
- Processing data where it makes sense: Enabling in-memory computationMicroprocessors and Microsystems, 2019
- Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question AnsweringInternational Journal of Computer Vision, 2018
- Geometric Deep Learning: Going beyond Euclidean dataIEEE Signal Processing Magazine, 2017