Improving Inferences About Null Effects With Bayes Factors and Equivalence Tests
Open Access
- 6 June 2018
- journal article
- research article
- Published by Oxford University Press (OUP) in The Journals of Gerontology, Series B: Psychological Sciences and Social Sciences
- Vol. 75 (1) , 45-57
Abstract
Researchers often conclude an effect is absent when a null-hypothesis significance test yields a nonsignificant p value. However, it is neither logically nor statistically correct to conclude an effect is absent when a hypothesis test is not significant. We present two methods to evaluate the presence or absence of effects: Equivalence testing (based on frequentist statistics) and Bayes factors (based on Bayesian statistics). In four examples from the gerontology literature, we illustrate different ways to specify alternative models that can be used to reject the presence of a meaningful or predicted effect in hypothesis tests. We provide detailed explanations of how to calculate, report, and interpret Bayes factors and equivalence tests. We also discuss how to design informative studies that can provide support for a null model or for the absence of a meaningful effect. The conceptual differences between Bayes factors and equivalence tests are discussed, and we also note when and why they might lead to similar or different inferences in practice. It is important that researchers are able to falsify predictions or can quantify the support for predicted null effects. Bayes factors and equivalence tests provide useful statistical tools to improve inferences about null effects.Keywords
All Related Versions
Funding Information
- Netherlands Organization for Scientific Research (452-17-013)
This publication has 36 references indexed in Scilit:
- Long-term Associations Between Physical Frailty and Performance in Specific Cognitive DomainsThe Journals of Gerontology, Series B: Psychological Sciences and Social Sciences, 2018
- When power analyses based on pilot data are biased: Inaccurate effect size estimators and follow-up biasJournal of Experimental Social Psychology, 2018
- Four reasons to prefer Bayesian analyses over significance testingPsychonomic Bulletin & Review, 2017
- Why Psychologists Should by Default Use Welch’s t-test Instead of Student’s t-testInternational Review of Social Psychology, 2017
- How Bayes factors change scientific practiceJournal of Mathematical Psychology, 2016
- Using Bayes to get the most out of non-significant resultsFrontiers in Psychology, 2014
- When to use the B onferroni correctionOphthalmic and Physiological Optics, 2014
- Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyondAIMS Neuroscience, 2014
- Statistics notes: Absence of evidence is not evidence of absenceBMJ, 1995
- Scientific Reporting and Personal Probabilities: Student's HypothesisJournal of the Royal Statistical Society Series B: Statistical Methodology, 1973