When Generative AI Meets Socratic Method: Investigating Programming Learning Dynamics Through Behaviours, Interaction Qualities and Perceptions
Authors:
Top Cited Papers
- 17 February 2026
- journal article
- Published by Wiley in Journal of Computer Assisted Learning
- Vol. 42 (2)
Abstract
Background: The integration of generative artificial intelligence (GAI) tools like GPT into programming education offers transformative potential through personalised guidance and instant feedback, yet risks fostering overreliance and superficial learning due to their tendency to deliver direct, context‐free answers. Objectives: This quasi‐experimental study addresses this gap by proposing a Socratic questioning framework to optimise GAI‐facilitated programming instruction, emphasising critical thinking over passive solution retrieval. Methods: We compared two pedagogical approaches: GAI‐Scaffolded Learning (GSL), where GPT employs structured Socratic dialogue to guide problem‐solving and GAI‐Direct Learning (GDL), which provides immediate answers without guided inquiry. This research collected learners' programming behaviours, interactions data with GPT from screen recordings and platform log data and perceptions data. This research further utilised multiple learning analytics approaches (i.e., click stream analysis, lag‐sequential analysis, epistemic network analysis [ENA] and statistics) to compare learners' programming behaviours, interaction patterns and perceptions under two approaches. Results and Conclusions: Through an analysis of 80 college students' programming behaviours, interaction qualities and perceptions, we found some intriguing results. First, GSL engaged in cyclical, reflective practices (debugging, Socratic questioning, console use), while GDL prioritised rapid fixes via trial‐and‐error with GPT code, risking superficial mimicry and over‐reliance on external resources. Second, ENA highlighted GSL's deeper engagement through interconnected feedback, emotional support and iterative inquiry, reducing frustration and sustaining persistence and GDL interactions focused on surface‐level queries, lacking scaffolding for emotional/heuristic integration. Third, GSL maintained positive attitudes due to structured prompts aligning expectations and easing cognitive load. GDL attitudes declined from mismatched expectations and frustration. Implications: Based on these findings, the study proposes pedagogical and developmental implications for future design and development of AI‐augmented curricula, providing actionable insights for educators seeking to harness GAI's potential while nurturing critical thinking in programming education.Keywords
Funding Information
- National Natural Science Foundation of China (62307011)
This publication has 38 references indexed in Scilit:
- ChatGPT: Empowering lifelong learning in the digital age of higher educationEducation and Information Technologies, 2023
- Augmented intelligence in programming learning: Examining student views on the use of ChatGPT for programming learningComputers in Human Behavior: Artificial Humans, 2023
- The effect of generative artificial intelligence (AI)-based tool use on students' computational thinking skills, programming self-efficacy and motivationComputers and Education: Artificial Intelligence, 2023
- Examining Science Education in ChatGPT: An Exploratory Study of Generative Artificial IntelligenceJournal of Science Education and Technology, 2023
- Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policyInternational Journal of Information Management, 2023
- Collaborative learning with block-based programming: investigating human-centered artificial intelligence in educationBehaviour & Information Technology, 2022
- Developing K-12 students’ programming ability: A systematic literature reviewEducation and Information Technologies, 2022
- Factors influencing programming self-efficacy: an empirical study in the context of Mainland ChinaAsia Pacific Journal of Education, 2021
- Clickstream for learning analytics to assess students’ behavior with ScratchFuture Generation Computer Systems, 2019
- Cognitive load theory, learning difficulty, and instructional designLearning and Instruction, 1994