Same Test, Better Scores: Boosting the Reliability of Short Online Intelligence Recruitment Tests with Nested Logit Item Response Theory Models
Open Access
- 10 July 2019
- journal article
- research article
- Published by MDPI AG in Journal of Intelligence
- Vol. 7 (3) , 17
Abstract
Assessing job applicants’ general mental ability online poses psychometric challenges due to the necessity of having brief but accurate tests. Recent research (Myszkowski & Storme, 2018) suggests that recovering distractor information through Nested Logit Models (NLM; Suh & Bolt, 2010) increases the reliability of ability estimates in reasoning matrix-type tests. In the present research, we extended this result to a different context (online intelligence testing for recruitment) and in a larger sample ( job applicants). We found that the NLMs outperformed the Nominal Response Model (Bock, 1970) and provided significant reliability gains compared with their binary logistic counterparts. In line with previous research, the gain in reliability was especially obtained at low ability levels. Implications and practical recommendations are discussed.
Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- Making sense of Cronbach's alphaInternational Journal of Medical Education, 2011
- A Nested Logit Approach for Investigating Distractors as Causes of Differential Item FunctioningJournal of Educational Measurement, 2011
- Role of test motivation in intelligence testingProceedings of the National Academy of Sciences, 2011
- Standardized Conditional SEM : A Case for Conditional ReliabilityApplied Psychological Measurement, 2007
- IRT Models for Ability-Based GuessingApplied Psychological Measurement, 2006
- When High-Powered People FailPsychological Science, 2005
- General Mental Ability in the World of Work: Occupational Attainment and Job Performance.Journal of Personality and Social Psychology, 2004
- Internet Recruitment and Selection: Kissing Frogs to find PrincesInternational Journal of Selection and Assessment, 2000
- Applicants’ Perceptions of Selection Procedures and Decisions: A Critical Review and Agenda for the FutureJournal of Management, 2000
- Advances in Item Response Theory and Applications: An IntroductionApplied Psychological Measurement, 1982