Item Diffficulty Modeling of Paragraph Comprehension Items
Authors:
- 1 September 2006
- journal article
- research article
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 30 (5) , 394-411
Abstract
Recent assessment research joining cognitive psychology and psychometric theory has introduced a new technology, item generation. In algorithmic item generation, items are systematically created based on specific combinations of features that underlie the processing required to correctly solve a problem. Reading comprehension items have been more difficult to model than other item types due to the complexities of quantifying text. However, recent developments in artificial intelligence for text analysis permit quantitative indices to represent cognitive sources of difficulty. The current study attempts to identify generative components for the Graduate Record Examination paragraph comprehension items through the cognitive decomposition of item difficulty. Text comprehension and decision processes accounted for a significant amount of the variance in item difficulties. The decision model variables contributed significantly to variance in item difficulties, whereas the text representation variables did not. Implications for score interpretation and future possibilities for item generation are discussed. Index terms: difficulty modeling, construct validity, comprehension tests, item generationKeywords
This publication has 26 references indexed in Scilit:
- Improving Construct Validity With Cognitive Psychology PrinciplesJournal of Educational Measurement, 2001
- Generating Items During Testing: Psychometric Issues and ModelsPsychometrika, 1999
- Using New Technology to Improve AssessmentEducational Measurement: Issues and Practice, 1999
- Validity and Automad Scoring: It's Not Only the ScoringEducational Measurement: Issues and Practice, 1998
- A cognitive design system approach to generating valid tests: Application to abstract reasoning.Psychological Methods, 1998
- Applications of Cognitive Design Systems to Test Development1994
- A Generative Approach to the Modeling of Isomorphic Hidden-Figure ItemsApplied Psychological Measurement, 1991
- The Effects of Surface Structure Variables on Performance in Reading Comprehension TestsReading Research Quarterly, 1981
- Levels of processing: A framework for memory researchJournal of Verbal Learning and Verbal Behavior, 1972
- How to Construct Achievement Tests to Assess ComprehensionReview of Educational Research, 1972