Data Mining Techniques in Analyzing Process Data: A Didactic
Authors:
Open Access
- 23 November 2018
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Psychology
- Vol. 9, 2231
Abstract
Due to increasing use of technology-enhanced educational assessment, data mining methods have been explored to analyse process data in log files from such assessment. However, most studies were limited to one data mining technique under one specific scenario. The current study demonstrates the usage of four frequently used supervised techniques, including Classification and Regression Trees (CART), gradient boosting, random forest, support vector machine (SVM), and two unsupervised methods, Self-organizing Map (SOM) and k-means, fitted to one assessment data. The USA sample (N = 426) from the 2012 Program for International Student Assessment (PISA) responding to problem-solving items is extracted to demonstrate the methods. After concrete feature generation and feature selection, classifier development procedures are implemented using the illustrated techniques. Results show satisfactory classification accuracy for all the techniques. Suggestions for the selection of classifiers are presented based on the research questions, the interpretability and the simplicity of the classifiers. Interpretations for the results from both supervised and unsupervised learning methods are provided.Keywords
This publication has 16 references indexed in Scilit:
- Modelling Conditional Dependence Between Response Time and AccuracyPsychometrika, 2016
- An NCME Instructional Module on Data Mining Methods for Classification and RegressionEducational Measurement: Issues and Practice, 2016
- Using Networks to Visualize and Analyze Process Data for Educational AssessmentJournal of Educational Measurement, 2016
- Statistical Methods for Assessments in Simulations and Serious GamesETS Research Report Series, 2014
- Improving Construct Validity Yields Better Models of Systematic Inquiry, Even with Less Information2012
- A Hierarchical Framework for Modeling Speed and Accuracy on Test ItemsPsychometrika, 2007
- The meaning and use of the area under a receiver operating characteristic (ROC) curve.Radiology, 1982
- A Cluster Separation MeasureIEEE Transactions on Pattern Analysis and Machine Intelligence, 1979
- A cluster separation measure.1979
- The Measurement of Observer Agreement for Categorical DataBiometrics, 1977