Building on Shaky Foundations: A Critical Review of Criterion Validity Evidence in Personnel Selection
Open Access
- 19 October 2026
- journal article
- review article
- Published by Wiley in International Journal of Selection and Assessment
- Vol. 34 (4)
Abstract
We reviewed the most recent meta‐analytic literature on the criterion validity of predictors of job performance used in personnel selection. The overreliance on concurrent incumbent studies leads to meta‐analytic estimates based on data collected in conditions with non‐representative stakes and an inability to estimate and correct for range restriction based on credible information, obstructing the estimation of operational validity in actual selection contexts. Therefore, we argue that we do not need more meta‐analyses. First, we need primary studies that actually provide estimates of operational criterion validity. On a positive note, we observed increasing attention in meta‐analyses to heterogeneity of results, publication bias and open science. There is room for improvement in terms of attention to the quality and operationalization of predictor and performance measures, and reflection on the quality of the evidence. Given these issues, our stance is that the current body of research does not allow drawing firm conclusions about the criterion validity of predictors of job performance in actual, operational selection settings. To move forward, we provide recommendations for reviewers, editors, and authors of primary studies and meta‐analyses on predictors used in personnel selection.Keywords
This publication has 52 references indexed in Scilit:
- Shaping the future of industrial-organizational psychology: The transformative potential of research collaborationsIndustrial and Organizational Psychology, 2025
- Meta-analytic validity of cognitive ability for hands-on military job proficiencyIntelligence, 2024
- Cognitive Ability and Job Performance: Meta-analytic Evidence for the Validity of Narrow Cognitive AbilitiesJournal of Business and Psychology, 2022
- Prestige and relevance of the scholarly journals: Impressions of SIOP membersIndustrial and Organizational Psychology, 2020
- Principles for the Validation and Use of Personnel Selection ProceduresIndustrial and Organizational Psychology, 2018
- Interest congruence and performance: Revisiting recent meta-analytic findingsJournal of Vocational Behavior, 2017
- Meta-analytic Reviews in the Organizational Sciences: Two Meta-analytic Schools on the Way to MARS (the Meta-analytic Reporting Standards)Journal of Business and Psychology, 2013
- A Meta-Analytic Study of General Mental Ability Validity for Different Occupations in the European Community.Journal of Applied Psychology, 2003
- A note on concurrent and predictive validity designs: A critical reanalysis.Journal of Applied Psychology, 1982
- Concurrent and predictive validity designs: A critical reanalysis.Journal of Applied Psychology, 1981