Robert A. Cribbie
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Robert A. Cribbie has a total of 225 co-authors. Here are the top co-authors by number of shared publications:
- Alyssa Counsell
Toronto Metropolitan University
6 shared publications Nataly Beribisky
Quantitative Methods Program, Department of Psychology, York University, Toronto, Ontario, Canada
5 shared publicationsAlyssa Counsell
Department of Psychology Department of Psychology York University York University Toronto ON Canada
3 shared publicationsChantal A. Arpin-Cribbie
Department of Psychology, Laurentian University
3 shared publicationsChantal Ragoonanan
Department of Psychology Department of Psychology York University York University Toronto ON Canada
3 shared publications- Sherry L. Grace
Faculty of Health, York University, Toronto, Canada
3 shared publications
Latest Publications
- Open Access
- 29 October 2025
- Published by Linnaeus University in Meta-Psychology
- Vol. 9
A Multi-faceted Mess: A Review of Statistical Power Analysis in Psychology Journal Articles
Many bodies recommend that a sample planning procedure, such as traditional NHST a priori power analysis, is conducted during the planning stages of a study. Power analysis allows the researcher to estimate how many participants are required in order to detect a minimally meaningful effect size at a specific level of power and Type I error rate. However, there are several drawbacks to the procedure that render it “a mess.” Specifically, the identification of the minimally meaningful effect size is very challenging, the procedure is not precision oriented, and does not guide the researcher to collect as many participants as feasibly possible. In this study, we explore how these three theoretical issues are reflected in applied psychological research in order to better understand whether these issues are concerns in practice. To investigate how power analysis is currently used, this study reviewed the reporting of 443 power analyses in high impact Psychology journals in 2016 and 2017 using Google Scholar. It was found that researchers rarely use the minimally meaningful effect size as a rationale for the chosen effect in a power analysis. Further, precision-based approaches and collecting the maximum sample size feasible are almost never used in tandem with power analyses. In light of these findings, we offer that researchers should focus on tools beyond traditional power analysis when sample planning, such as collecting the maximum sample size feasible.
- 29 October 2025
- Published by Linnaeus University in Meta-Psychology
- Vol. 9
Open AccessA Multi-faceted Mess: A Review of Statistical Power Analysis in Psychology Journal Articles
Many bodies recommend that a sample planning procedure, such as traditional NHST a priori power analysis, is conducted during the planning stages of a study. Power analysis allows the researcher to estimate how many participants are required in order to detect a minimally meaningful effect size at a specific level of power and Type I error rate. However, there are several drawbacks to the procedure that render it “a mess.” Specifically, the identification of the minimally meaningful effect size is very challenging, the procedure is not precision oriented, and does not guide the researcher to collect as many participants as feasibly possible. In this study, we explore how these three theoretical issues are reflected in applied psychological research in order to better understand whether these issues are concerns in practice. To investigate how power analysis is currently used, this study reviewed the reporting of 443 power analyses in high impact Psychology journals in 2016 and 2017 using Google Scholar. It was found that researchers rarely use the minimally meaningful effect size as a rationale for the chosen effect in a power analysis. Further, precision-based approaches and collecting the maximum sample size feasible are almost never used in tandem with power analyses. In light of these findings, we offer that researchers should focus on tools beyond traditional power analysis when sample planning, such as collecting the maximum sample size feasible.
- 1 September 2025
- Published by American Psychological Association (APA) in Scholarship of Teaching and Learning in Psychology
- Vol. 11 (3) , 376-397
Perfectionism, anxiety sensitivity, and negative reactions following a failed statistics test: A vulnerability–stress model.
- 1 September 2025
- Published by American Psychological Association (APA) in Scholarship of Teaching and Learning in Psychology
- Vol. 11 (3) , 376-397
Perfectionism, anxiety sensitivity, and negative reactions following a failed statistics test: A vulnerability–stress model.
- 28 July 2025
- Published by American Psychological Association (APA) in Canadian Psychology / Psychologie canadienne
- Vol. 67 (4) , 301-310
Multiplicity control in psychological research: A systematic review of empirical practices.
- 28 July 2025
- Published by American Psychological Association (APA) in Canadian Psychology / Psychologie canadienne
- Vol. 67 (4) , 301-310
Multiplicity control in psychological research: A systematic review of empirical practices.
- Open Access
- 19 December 2024
- Published by Wiley in International Journal of Psychology
- Vol. 60 (1) , e13271
Navigating the Bayes maze: The psychologist's guide to Bayesian statistics, a hands‐on tutorial with R code
Bayesian statistics has gained substantial popularity in the social sciences, particularly in psychology. Despite its growing prominence in the psychological literature, many researchers remain unacquainted with Bayesian methods and their advantages. This tutorial addresses the needs of curious applied psychology researchers and introduces Bayesian analysis as an accessible and powerful tool. We begin by comparing Bayesian and frequentist approaches, redefining fundamental terms from both perspectives with practical illustrations. Our exploration of Bayesian statistics includes Bayes's Theorem, likelihood, prior and posterior distributions, various prior types, and Markov‐Chain Monte Carlo (MCMC) methods, supplemented by graphical aids for clarity. To bridge theory and practice, we employ a psychological research example with real, open data. We analyse the data using both frequentist and Bayesian approaches, providing R code and comprehensive supporting information, and emphasising best practices for interpretation and reporting. We discuss and demonstrate how to interpret parameter estimates and credible intervals, among other essential topics. Throughout, we maintain an accessible and user‐friendly language, focusing on practical implications, intuitive examples, and actionable recommendations.
- 19 December 2024
- Published by Wiley in International Journal of Psychology
- Vol. 60 (1) , e13271
Open AccessNavigating the Bayes maze: The psychologist's guide to Bayesian statistics, a hands‐on tutorial with R code
Bayesian statistics has gained substantial popularity in the social sciences, particularly in psychology. Despite its growing prominence in the psychological literature, many researchers remain unacquainted with Bayesian methods and their advantages. This tutorial addresses the needs of curious applied psychology researchers and introduces Bayesian analysis as an accessible and powerful tool. We begin by comparing Bayesian and frequentist approaches, redefining fundamental terms from both perspectives with practical illustrations. Our exploration of Bayesian statistics includes Bayes's Theorem, likelihood, prior and posterior distributions, various prior types, and Markov‐Chain Monte Carlo (MCMC) methods, supplemented by graphical aids for clarity. To bridge theory and practice, we employ a psychological research example with real, open data. We analyse the data using both frequentist and Bayesian approaches, providing R code and comprehensive supporting information, and emphasising best practices for interpretation and reporting. We discuss and demonstrate how to interpret parameter estimates and credible intervals, among other essential topics. Throughout, we maintain an accessible and user‐friendly language, focusing on practical implications, intuitive examples, and actionable recommendations.
- Open Access
- 8 November 2024
- Published by Toronto Metropolitan University Library and Archives
Testing for negligible interaction: A coherent and robust approach
Researchers often want to demonstrate a lack of interaction between two categorical predictors on an outcome. To justify a lack of interaction, researchers typically accept the null hypothesis of no interaction from a conventional analysis of variance (ANOVA). This method is inappropriate as failure to reject the null hypothesis does not provide statistical evidence to support a lack of interaction. This study proposes a bootstrap-based intersection–union test for negligible interaction that provides coherent decisions between the omnibus test and post hoc interaction contrast tests and is robust to violations of the normality and variance homogeneity assumptions. Further, a multiple comparison strategy for testing interaction contrasts following a non-significant omnibus test is proposed. Our simulation study compared the Type I error control, omnibus power and per-contrast power of the proposed approach to the non-centrality-based negligible interaction test of Cheng and Shao (2007, Statistica Sinica, 17, 1441). For 2 × 2 designs, the empirical Type I error rates of the Cheng and Shao test were very close to the nominal α level when the normality and variance homogeneity assumptions were satisfied; however, only our proposed bootstrapping approach was satisfactory under non-normality and/or variance heterogeneity. In general a × b designs, although the omnibus Cheng and Shao test, as expected, is the most powerful, it is not robust to assumption violation and results in incoherent omnibus and interaction contrast decisions that are not possible with the intersection–union approach.
- 8 November 2024
- Published by Toronto Metropolitan University Library and Archives
Open AccessTesting for negligible interaction: A coherent and robust approach
Researchers often want to demonstrate a lack of interaction between two categorical predictors on an outcome. To justify a lack of interaction, researchers typically accept the null hypothesis of no interaction from a conventional analysis of variance (ANOVA). This method is inappropriate as failure to reject the null hypothesis does not provide statistical evidence to support a lack of interaction. This study proposes a bootstrap-based intersection–union test for negligible interaction that provides coherent decisions between the omnibus test and post hoc interaction contrast tests and is robust to violations of the normality and variance homogeneity assumptions. Further, a multiple comparison strategy for testing interaction contrasts following a non-significant omnibus test is proposed. Our simulation study compared the Type I error control, omnibus power and per-contrast power of the proposed approach to the non-centrality-based negligible interaction test of Cheng and Shao (2007, Statistica Sinica, 17, 1441). For 2 × 2 designs, the empirical Type I error rates of the Cheng and Shao test were very close to the nominal α level when the normality and variance homogeneity assumptions were satisfied; however, only our proposed bootstrapping approach was satisfactory under non-normality and/or variance heterogeneity. In general a × b designs, although the omnibus Cheng and Shao test, as expected, is the most powerful, it is not robust to assumption violation and results in incoherent omnibus and interaction contrast decisions that are not possible with the intersection–union approach.
- Open Access
- 8 November 2024
- Published by Toronto Metropolitan University Library and Archives
Testing for negligible interaction: A coherent and robust approach
Researchers often want to demonstrate a lack of interaction between two categorical predictors on an outcome. To justify a lack of interaction, researchers typically accept the null hypothesis of no interaction from a conventional analysis of variance (ANOVA). This method is inappropriate as failure to reject the null hypothesis does not provide statistical evidence to support a lack of interaction. This study proposes a bootstrap-based intersection–union test for negligible interaction that provides coherent decisions between the omnibus test and post hoc interaction contrast tests and is robust to violations of the normality and variance homogeneity assumptions. Further, a multiple comparison strategy for testing interaction contrasts following a non-significant omnibus test is proposed. Our simulation study compared the Type I error control, omnibus power and per-contrast power of the proposed approach to the non-centrality-based negligible interaction test of Cheng and Shao (2007, Statistica Sinica, 17, 1441). For 2 × 2 designs, the empirical Type I error rates of the Cheng and Shao test were very close to the nominal α level when the normality and variance homogeneity assumptions were satisfied; however, only our proposed bootstrapping approach was satisfactory under non-normality and/or variance heterogeneity. In general a × b designs, although the omnibus Cheng and Shao test, as expected, is the most powerful, it is not robust to assumption violation and results in incoherent omnibus and interaction contrast decisions that are not possible with the intersection–union approach.
- 8 November 2024
- Published by Toronto Metropolitan University Library and Archives
Open AccessTesting for negligible interaction: A coherent and robust approach
Researchers often want to demonstrate a lack of interaction between two categorical predictors on an outcome. To justify a lack of interaction, researchers typically accept the null hypothesis of no interaction from a conventional analysis of variance (ANOVA). This method is inappropriate as failure to reject the null hypothesis does not provide statistical evidence to support a lack of interaction. This study proposes a bootstrap-based intersection–union test for negligible interaction that provides coherent decisions between the omnibus test and post hoc interaction contrast tests and is robust to violations of the normality and variance homogeneity assumptions. Further, a multiple comparison strategy for testing interaction contrasts following a non-significant omnibus test is proposed. Our simulation study compared the Type I error control, omnibus power and per-contrast power of the proposed approach to the non-centrality-based negligible interaction test of Cheng and Shao (2007, Statistica Sinica, 17, 1441). For 2 × 2 designs, the empirical Type I error rates of the Cheng and Shao test were very close to the nominal α level when the normality and variance homogeneity assumptions were satisfied; however, only our proposed bootstrapping approach was satisfactory under non-normality and/or variance heterogeneity. In general a × b designs, although the omnibus Cheng and Shao test, as expected, is the most powerful, it is not robust to assumption violation and results in incoherent omnibus and interaction contrast decisions that are not possible with the intersection–union approach.
- Open Access
- 13 July 2023
- Published by Wiley in British Journal of Mathematical and Statistical Psychology
- Vol. 77 (1) , 103-129
Evaluating the performance of existing and novel equivalence tests for fit indices in structural equation modelling
It has been suggested that equivalence testing (otherwise known as negligible effect testing) should be used to evaluate model fit within structural equation modelling (SEM). In this study, we propose novel variations of equivalence tests based on the popular root mean squared error of approximation and comparative fit index fit indices. Using Monte Carlo simulations, we compare the performance of these novel tests to other existing equivalence testing‐based fit indices in SEM, as well as to other methods commonly used to evaluate model fit. Results indicate that equivalence tests in SEM have good Type I error control and display considerable power for detecting well‐fitting models in medium to large sample sizes. At small sample sizes, relative to traditional fit indices, equivalence tests limit the chance of supporting a poorly fitting model. We also present an illustrative example to demonstrate how equivalence tests may be incorporated in model fit reporting. Equivalence tests in SEM also have unique interpretational advantages compared to other methods of model fit evaluation. We recommend that equivalence tests be utilized in conjunction with descriptive fit indices to provide more evidence when evaluating model fit.
- 13 July 2023
- Published by Wiley in British Journal of Mathematical and Statistical Psychology
- Vol. 77 (1) , 103-129
Open AccessEvaluating the performance of existing and novel equivalence tests for fit indices in structural equation modelling
It has been suggested that equivalence testing (otherwise known as negligible effect testing) should be used to evaluate model fit within structural equation modelling (SEM). In this study, we propose novel variations of equivalence tests based on the popular root mean squared error of approximation and comparative fit index fit indices. Using Monte Carlo simulations, we compare the performance of these novel tests to other existing equivalence testing‐based fit indices in SEM, as well as to other methods commonly used to evaluate model fit. Results indicate that equivalence tests in SEM have good Type I error control and display considerable power for detecting well‐fitting models in medium to large sample sizes. At small sample sizes, relative to traditional fit indices, equivalence tests limit the chance of supporting a poorly fitting model. We also present an illustrative example to demonstrate how equivalence tests may be incorporated in model fit reporting. Equivalence tests in SEM also have unique interpretational advantages compared to other methods of model fit evaluation. We recommend that equivalence tests be utilized in conjunction with descriptive fit indices to provide more evidence when evaluating model fit.
- Open Access
- 5 January 2023
- Published by Ubiquity Press, Ltd. in Journal of Open Psychology Data
- Vol. 11 (1)
Data from an International Multi-Centre Study of Statistics and Mathematics Anxieties and Related Variables in University Students (the SMARVUS Dataset)
This large, international dataset contains survey responses from N = 12,570 students from 100 universities in 35 countries, collected in 21 languages. We measured anxieties (statistics, mathematics, test, trait, social interaction, performance, creativity, intolerance of uncertainty, and fear of negative evaluation), self-efficacy, persistence, and the cognitive reflection test, and collected demographics, previous mathematics grades, self-reported and official statistics grades, and statistics module details. Data reuse potential is broad, including testing links between anxieties and statistics/mathematics education factors, and examining instruments’ psychometric properties across different languages and contexts. Data and metadata are stored on the Open Science Framework website [https://osf.io/mhg94/].
- 5 January 2023
- Published by Ubiquity Press, Ltd. in Journal of Open Psychology Data
- Vol. 11 (1)
Open AccessData from an International Multi-Centre Study of Statistics and Mathematics Anxieties and Related Variables in University Students (the SMARVUS Dataset)
This large, international dataset contains survey responses from N = 12,570 students from 100 universities in 35 countries, collected in 21 languages. We measured anxieties (statistics, mathematics, test, trait, social interaction, performance, creativity, intolerance of uncertainty, and fear of negative evaluation), self-efficacy, persistence, and the cognitive reflection test, and collected demographics, previous mathematics grades, self-reported and official statistics grades, and statistics module details. Data reuse potential is broad, including testing links between anxieties and statistics/mathematics education factors, and examining instruments’ psychometric properties across different languages and contexts. Data and metadata are stored on the Open Science Framework website [https://osf.io/mhg94/].
- 1 October 2021
- Published by American Psychological Association (APA) in Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement
- Vol. 53 (4) , 480-488
Incidence and interpretation of statistical suppression in psychological research.
- 1 October 2021
- Published by American Psychological Association (APA) in Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement
- Vol. 53 (4) , 480-488
Incidence and interpretation of statistical suppression in psychological research.
- Open Access
Denouncing the use of field-specific effect size distributions to inform magnitude
An effect size (ES) provides valuable information regarding the magnitude of effects, with the interpretation of magnitude being the most important. Interpreting ES magnitude requires combining information from the numerical ES value and the context of the research. However, many researchers adopt popular benchmarks such as those proposed by Cohen. More recently, researchers have proposed interpreting ES magnitude relative to the distribution of observed ESs in a specific field, creating unique benchmarks for declaring effects small, medium or large. However, there is no valid rationale whatsoever for this approach. This study was carried out in two parts: (1) We identified articles that proposed the use of field-specific ES distributions to interpret magnitude (primary articles); and (2) We identified articles that cited the primary articles and classified them by year and publication type. The first type consisted of methodological papers. The second type included articles that interpreted ES magnitude using the approach proposed in the primary articles. There has been a steady increase in the number of methodological and substantial articles discussing or adopting the approach of interpreting ES magnitude by considering the distribution of observed ES in that field, even though the approach is devoid of a theoretical framework. It is hoped that this research will restrict the practice of interpreting ES magnitude relative to the distribution of ES values in a field and instead encourage researchers to interpret such by considering the specific context of the study.
Open AccessDenouncing the use of field-specific effect size distributions to inform magnitude
An effect size (ES) provides valuable information regarding the magnitude of effects, with the interpretation of magnitude being the most important. Interpreting ES magnitude requires combining information from the numerical ES value and the context of the research. However, many researchers adopt popular benchmarks such as those proposed by Cohen. More recently, researchers have proposed interpreting ES magnitude relative to the distribution of observed ESs in a specific field, creating unique benchmarks for declaring effects small, medium or large. However, there is no valid rationale whatsoever for this approach. This study was carried out in two parts: (1) We identified articles that proposed the use of field-specific ES distributions to interpret magnitude (primary articles); and (2) We identified articles that cited the primary articles and classified them by year and publication type. The first type consisted of methodological papers. The second type included articles that interpreted ES magnitude using the approach proposed in the primary articles. There has been a steady increase in the number of methodological and substantial articles discussing or adopting the approach of interpreting ES magnitude by considering the distribution of observed ES in that field, even though the approach is devoid of a theoretical framework. It is hoped that this research will restrict the practice of interpreting ES magnitude relative to the distribution of ES values in a field and instead encourage researchers to interpret such by considering the specific context of the study.