Evaluating the Quality of Higher Education Instructor-Constructed Multiple-Choice Tests: Impact on Student Grades
Open Access
- 2 June 2017
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Education
- Vol. 2
Abstract
Multiple-choice questions (MCQs) are commonly used in higher education assessment tasks because they can be easily and accurately scored, while giving good coverage of instructional content in a short time. However, studies that have evaluated the quality of MCQs used in higher education assessments have found many flawed items, resulting in misleading insights about student performance and contaminating important decisions. Thus, MCQs need to be evaluated statistically to ensure high quality items are used as the basis of inferences. This study evaluated the quality of 100 instructor-written MCQs used in an undergraduate midterm test (50 items) and final exam (50 items), making up 50% of the course grade, using the responses of 380 students enrolled in one 1st-year undergraduate general education course. Item difficulty, discrimination, and chance properties were determined using Classical Test Theory and Item Response Theory statistical item analysis models. The two-parameter logistic model consistently had the best fit to the data. The impact on overall course grades between the original raw score model and the IRT 2PL model showed 70% of students would receive the same grade (i.e., D to A), but only one-third would get the same mark using the standard augmented grade scale (i.e., A+ to D-). The analyses show that higher education institutions need to ensure MCQs are evaluated before student grading decisions are made.Keywords
This publication has 38 references indexed in Scilit:
- Rasch Analysis in the Human Sciences2014
- Implications of multiple-choice testing in nursing educationNurse Education Today, 2012
- Examination of the Quality of Multiple-choice Items on Classroom TestsThe Canadian Journal for the Scholarship of Teaching and Learning, 2011
- The Validity of Examination Essays in Higher Education: Issues and ResponsesHigher Education Quarterly, 2010
- Assessing the Quality of Multiple-Choice Test ItemsNurse Educator, 2010
- Assessment of learning with multiple-choice questionsNurse Education in Practice, 2005
- Measuring the Mind2005
- Multiple‐choice and true/false tests: myths and misapprehensionsAssessment & Evaluation in Higher Education, 2005
- Multimodel InferenceSociological Methods & Research, 2004
- DOES THE RASCH MODEL REALLY WORK FOR MULTIPLE CHOICE ITEMS? NOT IF YOU LOOK CLOSELYJournal of Educational Measurement, 1986