Anomalous Responses on Confidence-Scored Tests
Authors:
- Robert M. RippeyRRRobert M. Rippey
- 1 March 1985
- journal article
- Published by SAGE Publications in Evaluation & the Health Professions
- Vol. 8 (1) , 109-119
Abstract
Confidence testing allows subjects to respond to each option of a multiple-choice question with their perceived likelihood that it is the correct one. Scoring procedures subsequently provide measures not only of knowledge, but of the individual's "confidence" or realism in the assessment of whether the response is correct. There is an underlying assumption, however, of a significant correlation between assigned probabilities and actual likelihood for choosing a correct response, for we would naturally assume that subjects would, in general, be more certain when correct and less certain when wrong. Significant negative correlation or correlation not different from zero suggests bizarre responses in which the subject becomes more certain of the answer as mastery declines. Such anomalous responses were isolated from 530 tests of clinical knowledge administered to highly motivated medical students. These were rare (less than 3%0). Thus a crucial assumption underlying the theory of confidence testing is validated and a useful method of identifying outliers is devised.Keywords
This publication has 7 references indexed in Scilit:
- Adjusting Confidence Tests for RealismEvaluation & the Health Professions, 1982
- Interactive Confidence Test Scoring and InterpretationEducational and Psychological Measurement, 1978
- A COMPARISON OF SEVERAL METHODS OF ASSESSING PARTIAL KNOWLEDGE IN MULTIPLE‐CHOICE TESTS: I. SCORING PROCEDURES*Journal of Educational Measurement, 1975
- The Use of Confidence Testing In Objective TestsReview of Educational Research, 1972
- A COMPARISON OF THE VALIDITIES OF CONVENTIONAL CHOICE TESTING AND VARIOUS CONFIDENCE MARKING PROCEDURESJournal of Educational Measurement, 1971
- A COMPARISON OF FIVE DIFFERENT SCORING FUNCTIONS FOR CONFIDENCE TESTS1Journal of Educational Measurement, 1970
- Admissible Probability Measurement ProceduresPsychometrika, 1966