A Framework for Designing an AI Chatbot to Support Scientific Argumentation
Open Access
- 8 November 2025
- journal article
- research article
- Published by MDPI AG in Education Sciences
- Vol. 15 (11) , 1507
Abstract
As large language models (LLMs) are increasingly used to support learning, there is a growing need for a principled framework to guide the design of LLM-based tools and resources that are pedagogically effective and contextually responsive. This study proposes a framework by examining how prompt engineering can enhance the quality of chatbot responses to support middle school students’ scientific reasoning and argumentation. Drawing on learning theories and established frameworks for scientific argumentation, we employed a design-based research approach to iteratively refine system prompts and evaluate LLM-generated responses across diverse student input scenarios. Our analysis highlights how different prompt configurations affect the relevance and explanatory depth of chatbot feedback. We report findings from the iterative refinement process, along with an analysis of the quality of responses generated by each version of the chatbot. The outcomes indicate how different prompt configurations influence the coherence, relevance, and explanatory processes of LLM responses. The study contributes a set of critical design principles for developing theory-aligned prompts that enable LLM-based chatbots to meaningfully support students in constructing and revising scientific arguments. These principles offer broader implications for designing LLM applications across varied educational domains.Keywords
Funding Information
- Institute of Education Sciences (R305A240356)
This publication has 42 references indexed in Scilit:
- Managing uncertainty in scientific argumentationScience Education, 2019
- How concept familiarity and experience with scientific argumentation are related to the way groups participate in an episode of argumentationJournal of Research in Science Teaching, 2018
- Validation of automated scoring of science assessmentsJournal of Research in Science Teaching, 2016
- Teachers’ Pedagogical Content Knowledge of Scientific Argumentation: The Impact of Professional Development on K-12 TeachersScience Education, 2013
- Science teachers and scientific argumentation: Trends in views and practiceJournal of Research in Science Teaching, 2012
- Argument‐Driven Inquiry as a way to help students learn how to participate in scientific argumentation and craft written arguments: An exploratory studyScience Education, 2010
- Classroom communities' adaptations of the practice of scientific argumentationScience Education, 2010
- A learning progression for scientific argumentation: Understanding student work and designing supportive instructional contextsScience Education, 2010
- Scientific explanations: Characterizing and evaluating the effects of teachers' instructional practices on student learningJournal of Research in Science Teaching, 2007
- Enhancing the quality of argumentation in school scienceJournal of Research in Science Teaching, 2004