The Impact of Attribute Noise on the Automated Estimation of Collaboration Quality Using Multimodal Learning Analytics in Authentic Classrooms
Open Access
- 20 June 2024
- journal article
- research article
- Published by Society for Learning Analytics Research in Journal of Learning Analytics
- Vol. 11 (2) , 73-90
Abstract
Multimodal learning analytics (MMLA) research has shown the feasibility of building automated models of collaboration quality using artificial intelligence (AI) techniques (e.g., supervised machine learning (ML)), thus enablingthe development of monitoring and guiding tools for computer-supported collaborative learning (CSCL). However, the practical applicability and performance of these automated models in authentic settings remains largely an under-researched area. In such settings, the quality of data features or attributes is often affected by noise, which is referred to as attribute noise. This paper undertakes a systematic exploration of the impact of attribute noise on the performance of different collaboration-quality estimation models. Moreover, we also perform a comparative analysis of different ML algorithms in terms of their capability of dealing with attribute noise. We employ four ML algorithms that have often been used for collaboration-quality estimation tasks due to their high performance: random forest, naive Bayes, decision tree, and AdaBoost. Our results show that random forest and decision tree outperformed other algorithms for collaboration-quality estimation tasks in the presence of attribute noise. The study contributes to the MMLA (and learning analytics (LA) in general) and CSCL fields by illustrating how attribute noise impacts collaboration-quality model performance and which ML algorithms seem to be more robust to noise and thus more likely to perform well in authentic settings. Our research outcomes offer guidance to fellow researchers and developers of (MM)LA systems employing AI techniques with multimodal data to model collaboration-related constructs in authentic classroom settings.Keywords
This publication has 37 references indexed in Scilit:
- Studying teacher withitness in the wild: comparing a mirroring and an alerting & guiding dashboard for collaborative learningInternational Journal of Computer-Supported Collaborative Learning, 2023
- Exploring Teachers’ Perceptions of Artificial Intelligence as a Tool to Support their Practice in Estonian K-12 EducationInternational Journal of Artificial Intelligence in Education, 2022
- Neurophysiological Measurements in Higher Education: A Systematic Literature ReviewInternational Journal of Artificial Intelligence in Education, 2022
- Are MOOC Learning Analytics Results Trustworthy? With Fake Learners, They Might Not Be!International Journal of Artificial Intelligence in Education, 2019
- A Multimodal Analysis of MakingInternational Journal of Artificial Intelligence in Education, 2017
- Capturing and analyzing verbal and physical collaborative learning interactions at an enriched interactive tabletopInternational Journal of Computer-Supported Collaborative Learning, 2013
- Real-time mutual gaze perception enhances collaborative learning and collaboration qualityInternational Journal of Computer-Supported Collaborative Learning, 2013
- Analyzing Collaborative Interactions Across Domains and Settings: An Adaptable Rating Scheme2010
- A study of the effect of different types of noise on the precision of supervised learning techniquesArtificial Intelligence Review, 2010
- A rating scheme for assessing the quality of computer-supported collaboration processesInternational Journal of Computer-Supported Collaborative Learning, 2007