Testing Weak Nulls in Matched Observational Studies
Open Access
- 18 August 2022
- journal article
- research article
- Published by Oxford University Press (OUP) in Biometrics
- Vol. 79 (3) , 2196-2207
Abstract
We develop sensitivity analyses for the sample average treatment effect in matched observational studies while allowing unit-level treatment effects to vary. The methods may be applied to studies using any optimal without-replacement matching algorithm. In contrast to randomized experiments and to paired observational studies, we show for general matched designs that over a large class of test statistics, any procedure bounding the worst-case expectation while allowing for arbitrary effect heterogeneity must be unnecessarily conservative if treatment effects are actually constant across individuals. We present a sensitivity analysis which bounds the worst-case expectation while allowing for effect heterogeneity, and illustrate why it is generally conservative if effects are constant. An alternative procedure is presented that is asymptotically sharp if treatment effects are constant, and that is valid for testing the sample average effect under additional restrictions which may be deemed benign by practitioners. Simulations demonstrate that this alternative procedure results in a valid sensitivity analysis for the weak null hypothesis under a host of reasonable data-generating processes. The procedures allow practitioners to assess robustness of estimated sample average treatment effects to hidden bias while allowing for effect heterogeneity in matched observational studies.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Bounded, efficient and doubly robust estimation with inverse weightingBiometrika, 2010
- The sensitivity of linear regression coefficients’ confidence limits to the omission of a confounderThe Annals of Applied Statistics, 2010
- Matching Methods for Causal Inference: A Review and a Look ForwardStatistical Science, 2010
- On Estimation of the Survivor Average Causal Effect in Observational Studies When Important Confounders Are Missing Due to DeathBiometrics, 2009
- Sensitivity analysis for trend tests: application to the risk of radiation exposureBiostatistics, 2005
- Full Matching in an Observational Study of Coaching for the SATJournal of the American Statistical Association, 2004
- Sensitivity to Exogeneity Assumptions in Program EvaluationAmerican Economic Review, 2003
- Asymptotic Separability in Sensitivity AnalysisJournal of the Royal Statistical Society Series B: Statistical Methodology, 2000
- Studentized permutation tests for non-i.i.d. hypotheses and the generalized Behrens-Fisher problemStatistics & Probability Letters, 1997
- Statistical Problems in Agricultural ExperimentationJournal of the Royal Statistical Society Series B: Statistical Methodology, 1935