Towards philosophical reasoning with agentic LLMs: Socratic method for scientific assistance
Authors:
- Hassan HarbHHHassan Harb
- Yunkai SunYSYunkai Sun
- Mustafa UnalMUMustafa Unal
- Abhishek AggarwalAAAbhishek Aggarwal
- Chiara BissolottiCBChiara Bissolotti
- Isik Su BuyukerIBIsik Su Buyuker
- Sungil HongSHSungil Hong
- Lateef JolaosoLJLateef Jolaoso
- Bratin SenguptaBratin Sengupta
- Michael StuhrMSMichael Stuhr
- Zhenzhen YangZhenzhen Yang
Open Access
- 24 December 2025
- journal article
- Published by IOP Publishing in Machine Learning: Science and Technology
- Vol. 6 (4) , 045073
Abstract
As large language models (LLMs) become central tools in science, improving their reasoning capabilities is critical for meaningful and trustworthy applications. We introduce a Socratic agent for scientific reasoning, implemented through a structured system prompt that guides LLMs via classical principles of inquiry. Unlike typical prompt engineering or retrieval-based methods, our approach leverages definition, analogy, hypothesis elimination, and other Socratic techniques to generate more coherent, critical, and domain-aware responses. We evaluate the agent across diverse scientific domains and benchmark it on the abstraction and reasoning corpus challenge dataset, achieving 97.15% under a fixed prompting protocol and without fine-tuning or external tools. Expert evaluation shows improved reasoning depth, clarity, and adaptability over conventional LLM outputs, suggesting that structured prompting rooted in philosophical reasoning can improve the scientific utility of language models.Keywords
Funding Information
- Argonne National Laboratory
- Basic Energy Sciences (C-STEEL)
This publication has 26 references indexed in Scilit:
- A brain-inspired agentic architecture to improve planning with LLMsNature Communications, 2025
- The Hitchhiker's Guide to Socratic Methods in Prompting Large Language Models for Chemistry ApplicationsChemRxiv, 2025
- A review of large language models and autonomous agents in chemistryChemical Science, 2024
- Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching2024
- Artificial intelligence enabled ChatGPT and large language models in drug target discovery, drug discovery, and developmentMolecular Therapy Nucleic Acids, 2023
- 14 examples of how LLMs can transform materials science and chemistry: a reflection on a large language model hackathonDigital Discovery, 2023
- Do Large Language Models Understand Chemistry? A Conversation with ChatGPTJournal of Chemical Information and Modeling, 2023
- Assessment of chemistry knowledge in large language models that generate codeDigital Discovery, 2023
- Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering2019
- Socratic Method and Socratic Truth2005