Diagnostic Classification Models for Actionable Feedback in Education: Effects of Sample Size and Assessment Length
Open Access
- 22 February 2022
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Education
- Vol. 7
Abstract
E-learning is increasingly used to support student learning in higher education. This results in huge amounts of item response data containing valuable information about students’ strengths and weaknesses that can be used to provide effective feedback to both students and teachers. However, in current practice, feedback in e-learning is often given in the form of a simple proportion of correctly solved items rather than diagnostic, actionable feedback. Diagnostic classification models (DCMs) provide opportunities to model the item response data from formative assessments in online learning environments and to obtain diagnostic information to improve teaching and learning. This simulation study explores the demands on the data structure (i.e., assessment length, respondent sample size) to apply log-linear DCMs to empirical data. Thereby we provide guidance to educational practitioners on how many items need to be administered to how many students in order to accurately assess skills at different levels of specificity using DCMs. In addition, effects of misspecification of the dimensionality of the assessed skills on model fit indices are explored. Results show that detecting these misspecifications statistically with DCMs can be problematic. Recommendations and implications for educational practice are discussed.Keywords
This publication has 53 references indexed in Scilit:
- Model selection and psychological theory: A discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC).Psychological Methods, 2012
- A Note on Comparing Examinee Classification Methods for Cognitive Diagnosis ModelsEducational and Psychological Measurement, 2011
- Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees’ Knowledge and Skills in Mathematics: An Operational Implementation of Cognitive Diagnostic AssessmentInternational Journal of Testing, 2010
- Cognitive Modeling of Performance on Diagnostic Achievement Tests: A Philosophical Analysis and Justification2007
- The Power of FeedbackReview of Educational Research, 2007
- Multimodel InferenceSociological Methods & Research, 2004
- Self‐Regulation and Learning2003
- Assessment and Classroom LearningAssessment in Education: Principles, Policy & Practice, 1998
- Estimating the Dimension of a ModelThe Annals of Statistics, 1978
- A new look at the statistical model identificationIEEE Transactions on Automatic Control, 1974