StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT Interactions
- 23 September 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 31 (1) , 908-918
Abstract
The integration of Large Language Models (LLMs), especially ChatGPT, into education is poised to revolutionize students' learning experiences by introducing innovative conversational learning methodologies. To empower students to fully leverage the capabilities of ChatGPT in educational scenarios, understanding students' interaction patterns with ChatGPT is crucial for instructors. However, this endeavor is challenging due to the absence of datasets focused on student-ChatGPT conversations and the complexities in identifying and analyzing the evolutional interaction patterns within conversations. To address these challenges, we collected conversational data from 48 students interacting with ChatGPT in a master's level data visualization course over one semester. We then developed a coding scheme, grounded in the literature on cognitive levels and thematic analysis, to categorize students' interaction patterns with ChatGPT. Furthermore, we present a visual analytics system, StuGPTViz, that tracks and compares temporal patterns in student prompts and the quality of ChatGPT's responses at multiple scales, revealing significant pedagogical insights for instructors. We validated the system's effectiveness through expert interviews with six data visualization instructors and three case studies. The results confirmed StuGPTViz's capacity to enhance educators' insights into the pedagogical value of ChatGPT. We also discussed the potential research opportunities of applying visual analytics in education and developing AI-driven personalized learning solutions.Keywords
All Related Versions
Funding Information
- ITF (PRP/017/22FX)
- RGC GRF (16218724)
This publication has 65 references indexed in Scilit:
- ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design2024
- Impact of AI assistance on student agencyComputers & Education, 2023
- ChatGPT: The cognitive effects on learning and memoryBrain‐X, 2023
- Open AI in Education, the Responsible and Ethical Use of ChatGPT Towards Lifelong Learning2023
- GPT-3-Driven Pedagogical Agents to Train Children’s Curious Question-Asking SkillsInternational Journal of Artificial Intelligence in Education, 2023
- Involving Teachers in the Data-Driven Improvement of Intelligent Tutors: A Prototyping Study2023
- ChatGPT: Jack of all trades, master of noneInformation Fusion, 2023
- Large language models in education: A focus on the complementary relationship between human teachers and ChatGPTEducation and Information Technologies, 2023
- An integrated framework for course adapted student learning analytics dashboardComputers in Human Behavior, 2019
- Open Learner Models2010