Wes Bonifay
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Wes Bonifay has a total of 32 co-authors. Here are the top co-authors by number of shared publications:
Ashley L. Watts
Vanderbilt University, Nashville, TN, USA
4 shared publicationsWolfgang Wiedermann
University of Missouri, Columbia, USA
3 shared publications- Hanamori F. Skoblow
University of Missouri
2 shared publications Hyejin Shim
Laureate Institute for Brain Research, Tulsa, USA
2 shared publicationsJason Dana
Yale School of Management, Yale University
2 shared publicationsLi Cai
Department of Education, University of California, Los Angeles
2 shared publications
Latest Publications
- Open Access
- 21 May 2025
- Published by Cambridge University Press (CUP) in Psychometrika
- Vol. 90 (3) , 1039-1066
Random Item Response Data Generation Using a Limited-Information Approach: Applications to Assessing Model Complexity
Fitting propensity (FP) analysis quantifies model complexity but has been impeded in item response theory (IRT) due to the computational infeasibility of uniformly and randomly sampling multinomial item response patterns under a full-information approach. We adopt a limited-information (LI) approach, wherein we generate data only up to the lower-order margins of the complete item response patterns. We present an algorithm that builds upon classical work on sampling contingency tables with fixed margins by implementing a Sequential Importance Sampling algorithm to Quickly and Uniformly Obtain Contingency tables (SISQUOC). Theoretical justification and comprehensive validation demonstrate the effectiveness of the SISQUOC algorithm for IRT and offer insights into sampling from the complete data space defined by the lower-order margins. We highlight the efficiency and simplicity of the LI approach for generating large and uniformly random datasets of dichotomous and polytomous items. We further present an iterative proportional fitting procedure to reconstruct joint multinomial probabilities after LI-based data generation, facilitating FP evaluation using traditional estimation strategies. We illustrate the proposed approach by examining the FP of the graded response model and generalized partial credit model, with results suggesting that their functional forms express similar degrees of configural complexity.
- 21 May 2025
- Published by Cambridge University Press (CUP) in Psychometrika
- Vol. 90 (3) , 1039-1066
Open AccessRandom Item Response Data Generation Using a Limited-Information Approach: Applications to Assessing Model Complexity
Fitting propensity (FP) analysis quantifies model complexity but has been impeded in item response theory (IRT) due to the computational infeasibility of uniformly and randomly sampling multinomial item response patterns under a full-information approach. We adopt a limited-information (LI) approach, wherein we generate data only up to the lower-order margins of the complete item response patterns. We present an algorithm that builds upon classical work on sampling contingency tables with fixed margins by implementing a Sequential Importance Sampling algorithm to Quickly and Uniformly Obtain Contingency tables (SISQUOC). Theoretical justification and comprehensive validation demonstrate the effectiveness of the SISQUOC algorithm for IRT and offer insights into sampling from the complete data space defined by the lower-order margins. We highlight the efficiency and simplicity of the LI approach for generating large and uniformly random datasets of dichotomous and polytomous items. We further present an iterative proportional fitting procedure to reconstruct joint multinomial probabilities after LI-based data generation, facilitating FP evaluation using traditional estimation strategies. We illustrate the proposed approach by examining the FP of the graded response model and generalized partial credit model, with results suggesting that their functional forms express similar degrees of configural complexity.
- 19 May 2025
- Published by Springer Nature in Behavior Research Methods
- Vol. 57 (6) , 1-25
Parsimonious item response theory modeling with the cauchit link: Revisiting the rationale of the four-parameter logistic model
Application of the four-parameter logistic model (4PLM) in item response theory (IRT) research is contentious due to the complexities of estimating the asy
- 19 May 2025
- Published by Springer Nature in Behavior Research Methods
- Vol. 57 (6) , 1-25
Parsimonious item response theory modeling with the cauchit link: Revisiting the rationale of the four-parameter logistic model
Application of the four-parameter logistic model (4PLM) in item response theory (IRT) research is contentious due to the complexities of estimating the asy
- 3 March 2025
- Published by Taylor & Francis in Exceptionality
- Vol. 33 (4) , 263-276
Distribution Overlap to Estimate Effects in Meta-Analyses of Group Differences for Students with Learning Disabilities
- 3 March 2025
- Published by Taylor & Francis in Exceptionality
- Vol. 33 (4) , 263-276
Distribution Overlap to Estimate Effects in Meta-Analyses of Group Differences for Students with Learning Disabilities
- 10 February 2025
- Published by American Psychological Association (APA) in Psychological Methods
Reassessing the fitting propensity of factor models.
Model complexity is a critical consideration when evaluating a statistical model. To quantify complexity, one can examine fitting propensity (FP), or the ability of the model to fit well to diverse patterns of data. The scant foundational research on FP has focused primarily on proof of concept rather than practical application. To address this oversight, the present work joins a recently published study in examining the FP of models that are commonly applied in factor analysis. We begin with a historical account of statistical model evaluation, which refutes the notion that complexity can be fully understood by counting the number of free parameters in the model. We then present three sets of analytic examples to better understand the FP of exploratory and confirmatory factor analysis models that are widely used in applied research. We characterize our findings relative to previously disseminated claims about factor model FP. Finally, we provide some recommendations for future research on FP in latent variable modeling. (PsycInfo Database Record (c) 2025 APA, all rights reserved).
- 10 February 2025
- Published by American Psychological Association (APA) in Psychological Methods
Reassessing the fitting propensity of factor models.
Model complexity is a critical consideration when evaluating a statistical model. To quantify complexity, one can examine fitting propensity (FP), or the ability of the model to fit well to diverse patterns of data. The scant foundational research on FP has focused primarily on proof of concept rather than practical application. To address this oversight, the present work joins a recently published study in examining the FP of models that are commonly applied in factor analysis. We begin with a historical account of statistical model evaluation, which refutes the notion that complexity can be fully understood by counting the number of free parameters in the model. We then present three sets of analytic examples to better understand the FP of exploratory and confirmatory factor analysis models that are widely used in applied research. We characterize our findings relative to previously disseminated claims about factor model FP. Finally, we provide some recommendations for future research on FP in latent variable modeling. (PsycInfo Database Record (c) 2025 APA, all rights reserved).
- 14 March 2024
- Published by SAGE Publications in Assessment
- Vol. 32 (2) , 170-189
Good Fit Is Weak Evidence of Replication: Increasing Rigor Through Prior Predictive Similarity Checking
Replication provides a confrontation of psychological theory, not only in experimental research, but also in model-based research. Goodness of fit (GOF) of the original model to the replication data is routinely provided as meaningful evidence of replication. We demonstrate, however, that GOF obscures important differences between the original and replication studies. As an alternative, we present Bayesian prior predictive similarity checking: a tool for rigorously evaluating the degree to which the data patterns and parameter estimates of a model replication study resemble those of the original study. We apply this method to original and replication data from the National Comorbidity Survey. Both data sets yielded excellent GOF, but the similarity checks often failed to support close or approximate empirical replication, especially when examining covariance patterns and indicator thresholds. We conclude with recommendations for applied research, including registered reports of model-based research, and provide extensive annotated R code to facilitate future applications of prior predictive similarity checking.
- 14 March 2024
- Published by SAGE Publications in Assessment
- Vol. 32 (2) , 170-189
Good Fit Is Weak Evidence of Replication: Increasing Rigor Through Prior Predictive Similarity Checking
Replication provides a confrontation of psychological theory, not only in experimental research, but also in model-based research. Goodness of fit (GOF) of the original model to the replication data is routinely provided as meaningful evidence of replication. We demonstrate, however, that GOF obscures important differences between the original and replication studies. As an alternative, we present Bayesian prior predictive similarity checking: a tool for rigorously evaluating the degree to which the data patterns and parameter estimates of a model replication study resemble those of the original study. We apply this method to original and replication data from the National Comorbidity Survey. Both data sets yielded excellent GOF, but the similarity checks often failed to support close or approximate empirical replication, especially when examining covariance patterns and indicator thresholds. We conclude with recommendations for applied research, including registered reports of model-based research, and provide extensive annotated R code to facilitate future applications of prior predictive similarity checking.
- Open Access
- 22 January 2024
- Published by Center for Open Science in PsyArXiv
Good fit is weak evidence of replication: Increasing rigor through prior predictive similarity checking
Jerzy Neyman declared, “Models become plausible by repetition.” What he called repetition is now known as model replication and entails applying the same statistical model to data from different studies. Typically, researchers rely on goodness-of-fit – indexing the degree to which the model represents the data – as evidence of successful replication. But we demonstrate that goodness-of-fit testing fails to capture important (dis)similarities between studies. As an alternative, we propose prior predictive similarity checking: a method for rigorous comparison of the data patterns and/or model parameters from different studies. We illustrate this method using psychopathology data from the National Comorbidity Survey (NCS) and its purported replication, the NCS-R. Both data sets yielded excellent fit, but often failed to pass our more demanding similarity checking technique. We conclude with recommendations for applied model replication research.
- 22 January 2024
- Published by Center for Open Science in PsyArXiv
Open AccessGood fit is weak evidence of replication: Increasing rigor through prior predictive similarity checking
Jerzy Neyman declared, “Models become plausible by repetition.” What he called repetition is now known as model replication and entails applying the same statistical model to data from different studies. Typically, researchers rely on goodness-of-fit – indexing the degree to which the model represents the data – as evidence of successful replication. But we demonstrate that goodness-of-fit testing fails to capture important (dis)similarities between studies. As an alternative, we propose prior predictive similarity checking: a method for rigorous comparison of the data patterns and/or model parameters from different studies. We illustrate this method using psychopathology data from the National Comorbidity Survey (NCS) and its purported replication, the NCS-R. Both data sets yielded excellent fit, but often failed to pass our more demanding similarity checking technique. We conclude with recommendations for applied model replication research.
- Open Access
- 15 September 2023
- Published by Center for Open Science in PsyArXiv
A critical evaluation of the p-factor literature
The p-factor is relatively novel construct that is thought to explain and maybe even cause variation in all forms of psychopathology. Since its “discovery” in 2012, hundreds of studies have been dedicated to the extraction and validation of statistical instantiations of the p-factor, called general factors of psychopathology. In this Perspective, we outline 5 major challenges in the p-factor literature, namely that it has: (1) mistakenly equated good model fit with validity; (2) sought to corroborate weak p-factor theories through underspecified construct validation efforts; (3) produced poorly replicated general factors of psychopathology; (4) violated assumptions of latent variable models; and (5) reified general factors of psychopathology as latent, causal entities, in turn neglecting alternative models that do not incorporate a p-factor and are entirely incompatible with the notion that a single dimension adequately summarizes variation in all forms of psychopathology. Each of these challenges raise questions about substantive interpretations of the p-factor (i.e., negative emotionality, thought disorder), undermining the field’s confidence that the p-factor is a real, latent entity, or that GFPs are useful summaries of psychopathology variation. We conclude with ways to move forward, in the spirit of strengthening the p-factor literature and improving our ability to classify, treat, and prevent psychopathology across the lifespan.
- 15 September 2023
- Published by Center for Open Science in PsyArXiv
Open AccessA critical evaluation of the p-factor literature
The p-factor is relatively novel construct that is thought to explain and maybe even cause variation in all forms of psychopathology. Since its “discovery” in 2012, hundreds of studies have been dedicated to the extraction and validation of statistical instantiations of the p-factor, called general factors of psychopathology. In this Perspective, we outline 5 major challenges in the p-factor literature, namely that it has: (1) mistakenly equated good model fit with validity; (2) sought to corroborate weak p-factor theories through underspecified construct validation efforts; (3) produced poorly replicated general factors of psychopathology; (4) violated assumptions of latent variable models; and (5) reified general factors of psychopathology as latent, causal entities, in turn neglecting alternative models that do not incorporate a p-factor and are entirely incompatible with the notion that a single dimension adequately summarizes variation in all forms of psychopathology. Each of these challenges raise questions about substantive interpretations of the p-factor (i.e., negative emotionality, thought disorder), undermining the field’s confidence that the p-factor is a real, latent entity, or that GFPs are useful summaries of psychopathology variation. We conclude with ways to move forward, in the spirit of strengthening the p-factor literature and improving our ability to classify, treat, and prevent psychopathology across the lifespan.
- 3 August 2023
- Published by Springer Nature in Behavior Research Methods
- Vol. 56 (5) , 4385-4402
Parsimonious item response theory modeling with the negative log-log link: The role of inflection point shift
In item response theory (IRT) modeling, the magnitude of the lower and upper asymptote parameters determines the degree to which the inflection point shift
- 3 August 2023
- Published by Springer Nature in Behavior Research Methods
- Vol. 56 (5) , 4385-4402
Parsimonious item response theory modeling with the negative log-log link: The role of inflection point shift
In item response theory (IRT) modeling, the magnitude of the lower and upper asymptote parameters determines the degree to which the inflection point shift
- 19 July 2023
- Published by SAGE Publications in Perspectives on Psychological Science
- Vol. 19 (1) , 223-243
Better Accuracy for Better Science . . . Through Random Conclusions
Conducting research with human subjects can be difficult because of limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a baseline for interpreting effect-size estimates, in turn producing more stringent thresholds for hypothesis testing and for statistical-power calculations. An examination of recent meta-analyses in psychology, neuroscience, and medicine confirms that, even if all considered effects are real, results involving small effects are indeed indistinguishable from random conclusions.
- 19 July 2023
- Published by SAGE Publications in Perspectives on Psychological Science
- Vol. 19 (1) , 223-243
Better Accuracy for Better Science . . . Through Random Conclusions
Conducting research with human subjects can be difficult because of limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a baseline for interpreting effect-size estimates, in turn producing more stringent thresholds for hypothesis testing and for statistical-power calculations. An examination of recent meta-analyses in psychology, neuroscience, and medicine confirms that, even if all considered effects are real, results involving small effects are indeed indistinguishable from random conclusions.
- Open Access
- 2 May 2022
- Published by Center for Open Science in PsyArXiv
Better Accuracy for Better Science...Through Random Conclusions
The difficulty of conducting research with human subjects often entails limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a baseline for interpreting effect size estimates, in turn producing more stringent thresholds for hypothesis testing and statistical power calculations. An examination of recent meta-analyses in psychology, neuroscience, and medicine confirms that, even if all considered effects are real, results involving small effects are indeed indistinguishable from random conclusions.
- 2 May 2022
- Published by Center for Open Science in PsyArXiv
Open AccessBetter Accuracy for Better Science...Through Random Conclusions
The difficulty of conducting research with human subjects often entails limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a baseline for interpreting effect size estimates, in turn producing more stringent thresholds for hypothesis testing and statistical power calculations. An examination of recent meta-analyses in psychology, neuroscience, and medicine confirms that, even if all considered effects are real, results involving small effects are indeed indistinguishable from random conclusions.