On the Hauck-Donner Effect in Wald Tests: Detection, Tipping Points, and Parameter Space Characterization
Preprint
Abstract
The Wald test remains ubiquitous in statistical practice despite shortcomings such as its inaccuracy in small samples and lack of invariance under reparameterization. This paper develops on another but lesser-known shortcoming called the Hauck--Donner effect (HDE) whereby a Wald test statistic is not monotonely increasing as a function of increasing distance between the parameter estimate and the null value. Resulting in an upward biased $p$-value and loss of power, the aberration can lead to very damaging consequences such as in variable selection. The HDE afflicts many types of regression models and corresponds to estimates near the boundary of the parameter space. This article presents several new results, and its main contributions are to (i) propose a very general test for detecting the HDE, regardless of its underlying cause; (ii) fundamentally characterize the HDE by pairwise ratios of Wald and Rao score and likelihood ratio test statistics for 1-parameter distributions; (iii) show that the parameter space may be partitioned into an interior encased by 5 HDE severity measures (faint, weak, moderate, strong, extreme); (iv) prove that a necessary condition for the HDE in a 2 by 2 table is a log odds ratio of at least 2; (v) give some practical guidelines about HDE-free hypothesis testing. Overall, practical post-fit tests can now be conducted potentially to any model estimated by iteratively reweighted least squares, such as the generalized linear model (GLM) and Vector GLM (VGLM) classes, the latter which encompasses many popular regression models.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- On the Hauck–Donner Effect in Wald Tests: Detection, Tipping Points, and Parameter Space CharacterizationJournal of the American Statistical Association, 2021
- High-Dimensional Inference: Confidence Intervals, $p$-Values and R-Software hdiStatistical Science, 2015
- Improved Estimation in Cumulative Link ModelsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2013
- Blindly Using Wald's Test Can Miss Rare Disease‐Causal Variants in Case‐Control Association StudiesAnnals of Human Genetics, 2012
- Odds are, it's wrong: Science fails to face the shortcomings of statisticsScience News, 2010
- How Can the Score Test Be Inconsistent?The American Statistician, 2007
- A Matrix Handbook for Statisticians2007
- Modeling Survival Data: Extending the Cox Model2000
- Partial Separation in Logistic DiscriminationJournal of the Royal Statistical Society Series B: Statistical Methodology, 1989
- The Analysis of Sensory Data by Generalized Linear ModelBiometrical Journal, 1989