Engagement Recognition in Online Learning Based on an Improved Video Vision Transformer
Authors:
- 18 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Abstract
Online learning has gained wide attention and application due to its flexibility and convenience. However, due to the separation of time and space, the level of students' engagement is not easily informed by teachers, which affects the effectiveness of teaching. Automatic detection of students' engagement is an effective way to solve this problem. It can help teachers obtain timely feedback from students and adjust the teaching schedule. In this paper, transformer is first applied in engagement recognition and a novel network based on an improved video vision transformer (ViViT) is proposed to detect student engagement. A new transformer encoder, named Transformer Encoder with Low Complexity (TELC) is proposed. It adopts unit force operated attention (UFO-attention) to eliminate the nonlinearity of the original self-attention in standard ViViT and Patch Merger to fuse the input patches, which allows the network to significantly reduce computational complexity while improving performance. The proposed method is evaluated on the Dataset for Affective States in E-learning Environments (DAiSEE) and achieves an accuracy of 63.91% in the four-level classification task, which is superior to state-of-the-art methods. The experimental results demonstrate the effectiveness of our method, which is more suitable for the practical application of online learning.Keywords
This publication has 16 references indexed in Scilit:
- EmotiW 2019: Automatic Emotion, Engagement and Cohesion Prediction Tasks2019
- An Novel End-to-end Network for Automatic Student Engagement Recognition2019
- Fine-grained Engagement Recognition in Online Learning Environment2019
- Automatic Recognition of Children Engagement from Facial Video Using Convolutional Neural NetworksIEEE Transactions on Affective Computing, 2018
- Focal Loss for Dense Object Detection2017
- Deep Networks with Stochastic Depth2016
- Joint Face Detection and Alignment Using Multitask Cascaded Convolutional NetworksIEEE Signal Processing Letters, 2016
- An Instructional Design Framework for Fostering Student Engagement in Online Learning EnvironmentsTechTrends, 2016
- Rethinking the Inception Architecture for Computer Vision2016
- A crowdsourced approach to student engagement recognition in e-learning environments2016