Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?
Preprint
Abstract
Recent progress in LLMs discussion suggests that multi-agent discussion improves the reasoning abilities of LLMs. In this work, we reevaluate this claim through systematic experiments, where we propose a novel group discussion framework to enrich the set of discussion mechanisms. Interestingly, our results show that a single-agent LLM with strong prompts can achieve almost the same performance as the best existing discussion approach on a wide range of reasoning tasks and backbone LLMs. We observe that the multi-agent discussion performs better than a single agent only when there is no demonstration in the prompt. Further study reveals the common interaction mechanisms of LLMs during the discussion.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Graph of Thoughts: Solving Elaborate Problems with Large Language ModelsProceedings of the AAAI Conference on Artificial Intelligence, 2024
- Measuring and Narrowing the Compositionality Gap in Language Models2023
- Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement2023
- Large Language Models are Better Reasoners with Self-Verification2023
- Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models2023
- MathPrompter: Mathematical Reasoning using Large Language Models2023
- LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learning2022
- Successive Prompting for Decomposing Complex Questions2022
- Memory-assisted prompt editing to improve GPT-3 after deployment2022
- Mechanism Design with Collusion and CorrelationEconometrica, 2000