Effect of different scoring approaches upon credit assignment when using Multiple True‐False items in dental undergraduate examinations
Authors:
- 22 June 2018
- journal article
- research article
- Published by Wiley in European Journal of Dental Education
- Vol. 22 (4) , E669-E678
Abstract
Introduction: Various scoring approaches for Multiple True‐False (MTF) items exist. This study aimed at comparing scoring results obtained with different scoring approaches and to assess the effect of item cues on each scoring approaches’ result.Materials and Methods: Different scoring approaches (MTF, Count‐2, Count‐3, “Vorkauf‐Method,” PS50, Dichotomized MTF, “Blasberg‐Method,” Multiple response (MR), Correction for Guessing, “Ripkey‐Method,” Morgan‐Method, Balanced Scoring Method) were retrospectively applied to all MTF items used within electronic examinations of undergraduate dental students at the University Medical Center Göttingen in the winter term 2016/2017 (1297 marking events). Item quality was evaluated regarding formal parameters such as presence of cues and correctness of content. Differences between scoring results of all scoring approaches and the differences between each methods’ scoring results of items with and without cues were calculated by Wilcoxon rank sum tests (P < .05).Results: Average scoring results per item highly differed between the scoring approaches and ranged from 0.46 (MR) to 0.92 (Dichotomized MTF). Presence of cues leads to significantly higher scoring in case of all scoring approaches (P < .001; +0.14 on average). However, effect of cues differed amongst scoring approaches and ranged from +0.04 (Dichotomized MTF) to +0.20 (MR).Conclusion: Scoring of MTF items is complex. The data presented in this manuscript may help educators make informed choices about scoring algorithms.Keywords
This publication has 10 references indexed in Scilit:
- Multiple true–false items: a comparison of scoring algorithmsAdvances in Health Sciences Education, 2017
- Contemplation on marking scheme for Type X multiple choice questions, and an illustration of a practically applicable schemeIndian Journal of Pharmacology, 2016
- Assessment in Medical EducationNew England Journal of Medicine, 2007
- The Power of Testing Memory: Basic Research and Implications for Educational PracticePerspectives on Psychological Science, 2006
- A “new” item format for assessing aspects of clinical competenceAcademic Medicine, 1996
- A Brief Report on a Comparison of Six Scoring Methods for Multiple True-False ItemsEducational and Psychological Measurement, 1993
- The Multiple True‐False Item Format: A Status ReviewEducational Measurement: Issues and Practice, 1992
- Multiple True-False Items: A Study of Interitem Correlations, Scoring Alternatives, and Reliability EstimationJournal of Educational Measurement, 1988
- THE RELATIVE MERITS OF MULTIPLE TRUE‐FALSE ACHIEVEMENT TESTSJournal of Educational Measurement, 1982
- MCQ: An interactive computer program for multiple-choice self-testingBiochemical Education, 1979