Tsz Keung Wong

Data exclusion is common in psychology and can potentially impact statistical results and conclusions when applied as a p-hacking strategy. We studied the impact of data exclusion on p values in the framework of Null Hypothesis Significance Testing and developed a simulation-based methodology, along with a Shiny app, to assess the robustness of statistically significant results under different mechanisms of data exclusion. Additionally, we applied our method to 77 t tests in a sample of 16 articles published in the Journal of Personality and Social Psychology in 2014. We found that statistical conclusions (i.e., rejection of H0 given α = .05) are robust to randomized data exclusion (RDE) if the reported two-sided p value ≤ .01, and to selecting the best of ten subsets after selective randomized data exclusion (SRDE) if the reported two-sided p value ≤ .001. A small portion of articles reported sample sizes per group before (1.3%) and after data exclusion (2.6%) among 76 independent samples t tests, complicating direct robustness analyses. Of the statistically significant t test results reported in the articles, 71% were robust to RDE and 56% were robust to SRDE. Besides detailed study preregistration to prevent potential bias from data exclusion, we recommend that authors explicitly report the reasons for exclusion and provide detailed information about the sample sizes before and after data exclusion.

The Bayes factor is often proposed as a superior replacement to p values in testing null hypotheses for various reasons, with the availability of many user-friendly and easily accessible statistical software tools facilitating the use of Bayesian tests. Meanwhile, Bayes factor design analysis (BFDA), the counterpart of power analysis, is also proposed to ensure the maximum efficiency and informativeness of a study. Despite tools for conducting BFDA being limited and mostly relying on Monte Carlo methodology, methods based on root-finding algorithms have been recently developed (e.g., Pawel and Held, 2025), overcoming weaknesses of simulation approaches. This paper builds on these advancements by presenting a method generalizing the existing approach for conducting BFDA for sample size determination in t tests. The major advantage of the current method is that it does not assume normality of the effect size estimate, allowing more flexibility in the specification of the design and analysis priors. We developed and showcase a user-friendly Shiny app for facilitating the use of BFDA, illustrated with an empirical example. Furthermore, using our method, we explore the operating characteristics of the Bayes factors using various priors.

Most researchers receive formal training in frequentist statistics during their undergraduate studies. In particular, hypothesis testing is usually rooted on the null hypothesis significance testing paradigm and its p‐value. Null hypothesis Bayesian testing and its so‐called Bayes factor are now becoming increasingly popular. Although the Bayes factor is often introduced as being the Bayesian counterpart to the p‐value, its computation, use, and interpretation are quite distinct from the p‐value. There is now evidence confirming that the application of the Bayes factor in applied research is ill‐devised. To improve the current status quo, we have created a Shiny/R app called the Bayes factor, which offers a dynamic tutorial for learning all the basics about the Bayes factor. In this paper, we explain how the app works and we offer suggestions on how to use it in class or self‐study settings. The app is freely available at https://statsedge.org/shiny/LearnBF/.

Hypothesis testing is often used for inference in the social sciences. In particular, null hypothesis significance testing (NHST) and its p value have been ubiquitous in published research for decades. Much more recently, null hypothesis Bayesian testing (NHBT) and its Bayes factor have also started to become more commonplace in applied research. Following preliminary work by Wong and colleagues, we investigated how, and to what extent, researchers misapply the Bayes factor in applied psychological research by means of a literature study. Based on a final sample of 167 articles, our results indicate that, not unlike NHST and the [ ] value, the use of NHBT and the Bayes factor also shows signs of misconceptions. We consider the root causes of the identified problems and provide suggestions to improve the current state of affairs. This article is aimed to assist researchers in drawing the best inferences possible while using NHBT and the Bayes factor in applied research.

The aim of this study is to investigate whether there is a potential mismatch between the usability of a statistical tool and psychology researchers’ expectation of it. Bayesian statistics is often promoted as an ideal substitute for frequentists statistics since it coincides better with researchers’ expectations and needs. A particular incidence of this is the proposal of replacing Null Hypothesis Significance Testing (NHST) by Null Hypothesis Bayesian Testing (NHBT) using the Bayes factor. In this paper, it is studied to what extent the usability and expectations of NHBT match well. First, a study of the reporting practices in 73 psychological publications was carried out. It was found that eight Questionable Reporting and Interpreting Practices (QRIPs) occur more than once among the practitioners when doing NHBT. Specifically, our analysis provides insight into possible mismatches and their occurrence frequencies. A follow-up survey study has been conducted to assess such mismatches. The sample (N = 108) consisted of psychology researchers, experts in methodology (and/or statistics), and applied researchers in fields other than psychology. The data show that discrepancies exist among the participants. Interpreting the Bayes Factor as posterior odds and not acknowledging the notion of relative evidence in the Bayes Factor are arguably the most concerning ones. The results of the paper suggest that a shift of statistical paradigm cannot solve the problem of misinterpretation altogether if the users are not well acquainted with the tools.