Large Field Surveys and Short Items Sets: Estimating Reasoning Skills among Distinct Samples in Malawi and Zambia
Open Access
- 20 October 2023
- journal article
- research article
- Published by University of Arizona in Journal of Methods and Measurement in the Social Sciences
- Vol. 14 (1)
Abstract
Raven's Progressive Matrices measure logical reasoning and are often included in large multi-topic surveys in low and middle-income countries. The matrices are image-based items that do not require formal knowledge of language or math to complete. As such, they are attractive items to measure logical reasoning in international development contexts. Many of these large field surveys include short item Raven's sets because space is too limited to fit a full suite. However, short sets can result in restricted variation in terms of test scores. In this paper, we use a nominal response model (NRM) form of item response theory (IRT) to uncover hidden variation in right and wrong answers using short-item Raven's tests from two large field surveys in Malawi and Zambia. We also analyze relationships between a set of other variables, comparing performance of different versions of the logical reasoning scores as both independent and dependent variables, checking the validity of the new scores. The new NRM-estimated logical reasoning scores follow a more normal distribution in both samples. Validity checks suggest that when relationships are less strong, NRM-estimated scores can capture more nuance than summed scores or even 2 parameter logistic IRT-estimated scores. NRM can uncover differences that are not apparent when using simple summed scores.Keywords
This publication has 29 references indexed in Scilit:
- RETRACTED: No glove, no love: General intelligence predicts increased likelihood of condom use in response to HIV threatPersonality and Individual Differences, 2020
- Self-confidence and strategic behaviorExperimental Economics, 2017
- Measurements of Intelligence in sub-Saharan Africa: Perspectives Gathered from Research in MaliCurrent Psychology, 2017
- Summary for Policymakers2014
- Height, skills, and labor market outcomes in MexicoJournal of Development Economics, 2014
- Schooling and basic aspects of intelligence: A natural quasi-experiment in MalawiJournal of Applied Developmental Psychology, 2009
- Sexual Experiences of Adolescents with Low Cognitive Abilities in the U.SJournal of Developmental and Physical Disabilities, 2005
- Sex differences on the progressive matrices: A meta-analysisIntelligence, 2004
- The Raven's Progressive Matrices: Change and Stability over Culture and TimeCognitive Psychology, 2000
- Estimating Item Parameters and Latent Ability when Responses are Scored in Two or More Nominal CategoriesPsychometrika, 1972