Exploring teacher-student interaction through multimodal large language models: an empirical investigation
Authors:
Open Access
- 6 February 2026
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 16 (1)
Abstract
Teacher–student interaction is central to classroom learning, yet traditional observation and machine-learning approaches often remain inefficient and subjective. This study explores the use of multimodal large language models (MLLMs) for systematic analysis of classroom dynamics. We fine-tuned VisualGLM-6B on 2,380 annotated images from 30 classroom videos, covering five interaction types: guided, collaborative, questioning, independent, and exhibitive. LoRA-based fine-tuning combined with prompt engineering was employed to enhance interpretability and domain-specific accuracy. Model performance was assessed through confusion matrices, BERTScore, and expert comparisons. The fine-tuned model achieved 82% overall accuracy, performing best on guided, independent, and exhibitive interactions, while collaborative and questioning types remained challenging. Compared with expert annotation, the model provided more structured and interpretable outputs, though occasional misclassifications and hallucinations persisted. These findings demonstrate the feasibility of applying MLLMs for efficient, objective analysis of teacher–student interactions and highlight future directions such as incorporating audio inputs and larger datasets to further advance educational research methodologies.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62177032)
This publication has 27 references indexed in Scilit:
- Future of education in the era of generative artificial intelligence: Consensus among Chinese scholars on applications of ChatGPT in schoolsFuture in Educational Research, 2023
- Using multimodal learning analytics to model students’ learning behavior in animated programming classroomEducation and Information Technologies, 2023
- A Student-Teacher Multimodal Interaction Analysis System for Classroom Observation2023
- Hybrid analysis of the learner’s online behavior based on learning styleEducation and Information Technologies, 2023
- Analysis and Improvement of Classroom Teaching Based on Artificial Intelligence2022
- Streamlit Use Cases2022
- Student Behavioral Analysis Using Computer Vision2021
- Predicting learner’s performance through video sequences viewing behavior analysis using educational data-miningEducation and Information Technologies, 2021
- Clustering Constructed Responses for Formative Assessment in Comprehension SEEDINGLecture Notes in Computer Science, 2014
- Pedagogical content knowledge as reflected in teacher–student interactions: Analysis of two video casesJournal of Research in Science Teaching, 2012