AUTOMATED SCORING OF SHORT‐ANSWER OPEN‐ENDED GRE® SUBJECT TEST ITEMS
Authors:
Open Access
- 1 June 2008
- journal article
- research article
- Published by Wiley in ETS Research Report Series
- Vol. 2008 (1)
Abstract
This report describes the development, administration, and scoring of open‐ended variants of GRE® Subject Test items in biology and psychology. These questions were administered in a Web‐based experiment to registered examinees of the respective Subject Tests. The questions required a short answer of 1‐3 sentences, and responses were automatically scored by natural language processing methods, using the c‐rater™ scoring engine, immediately after participants submitted their responses. Participants received immediate feedback on the correctness of their answers, and an opportunity to revise their answers. Subsequent human scoring of the responses allowed an evaluation of the quality of automated scoring. This report focuses on the success of the automated scoring process. A separate report describes the feedback and revision results.Keywords
This publication has 8 references indexed in Scilit:
- EFFECT OF IMMEDIATE FEEDBACK AND REVISION ON PSYCHOMETRIC PROPERTIES OF OPEN‐ENDED GRE® SUBJECT TEST ITEMSETS Research Report Series, 2008
- C-rater: Automated Scoring of Short-Answer QuestionsComputers and the Humanities, 2003
- A comprehensive meta-analysis of the predictive validity of the Graduate Record Examinations: Implications for graduate student selection and performance.Psychological Bulletin, 2001
- How Not to Pick a Physicist?Science, 1996
- USING THE FREE‐RESPONSE SCORING TOOL TO AUTOMATICALLY SCORE THE FORMULATING‐HYPOTHESES ITEMETS Research Report Series, 1994
- A Comparison of Quantitative Questions in Open‐Ended and Multiple‐Choice FormatsJournal of Educational Measurement, 1992
- The Measurement of Observer Agreement for Categorical DataBiometrics, 1977
- A New Procedure for Assessing Reliability of Scoring EEG Sleep RecordingsAmerican Journal of EEG Technology, 1971