Safe testing
Authors:
Top Cited Papers
Open Access
- 7 March 2024
- journal article
- research article
- Published by Oxford University Press (OUP) in Journal of the Royal Statistical Society Series B: Statistical Methodology
- Vol. 86 (5) , 1091-1128
Abstract
We develop the theory of hypothesis testing based on the e-value, a notion of evidence that, unlike the p-value, allows for effortlessly combining results from several studies in the common scenario where the decision to perform a new study may depend on previous outcomes. Tests based on e-values are safe, i.e. they preserve Type-I error guarantees, under such optional continuation. We define growth-rate optimality (GRO) as an analogue of power in an optional continuation context, and we show how to construct GRO e-variables for general testing problems with composite null and alternative, emphasizing models with nuisance parameters. GRO e-values take the form of Bayes factors with special priors. We illustrate the theory using several classic examples including a one-sample safe t-test and the 2×2 contingency table. Sharing Fisherian, Neymanian and Jeffreys-Bayesian interpretations, e-values may provide a methodology acceptable to adherents of all three schools.Keywords
This publication has 44 references indexed in Scilit:
- Generic E-variables for exact sequential -sample tests that allow for optional stoppingJournal of Statistical Planning and Inference, 2023
- Estimating means of bounded random variables by bettingJournal of the Royal Statistical Society Series B: Statistical Methodology, 2023
- Log-optimal anytime-valid E-valuesInternational Journal of Approximate Reasoning, 2022
- Valid sequential inference on probability forecast performanceBiometrika, 2021
- Abandon Statistical SignificanceThe American Statistician, 2019
- Redefine statistical significanceNature Human Behaviour, 2017
- Asymptotically optimal data analysis for rejecting local realismPhysical Review A, 2011
- Uniform test of algorithmic randomness over a general spaceTheoretical Computer Science, 2005
- Robust utility maximization for complete and incomplete market modelsFinance and Stochastics, 2005
- Information projections revisitedIEEE Transactions on Information Theory, 2003