Testing weak nulls in matched observational studies
Preprint
Abstract
We develop sensitivity analyses for weak nulls in matched observational studies while allowing unit-level treatment effects to vary. In contrast to randomized experiments, we show for general matched designs that over a large class of test statistics, any valid sensitivity analysis for the weak null must be unnecessarily conservative if Fisher's sharp null of no treatment effect for any individual also holds. We present a sensitivity analysis valid for the weak null, and illustrate why it is conservative if the sharp null holds through connections to inverse probability weighted estimators. An alternative procedure is presented that is asymptotically sharp if treatment effects are constant, and is valid for the weak null under additional assumptions which may be deemed reasonable by practitioners. Simulations demonstrate that this alternative procedure results in a valid sensitivity analysis for the weak null hypothesis under many reasonable data-generating processes. With binary outcomes, a modification of the alternative procedure results in a test that is valid over the entirety of the weak null without further restrictions. The methods may be applied to matched observational studies constructed using any optimal without-replacement matching algorithm, allowing practitioners to assess robustness to hidden bias while allowing for treatment effect heterogeneity.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Randomization Tests for Weak Null Hypotheses in Randomized ExperimentsJournal of the American Statistical Association, 2020
- Studentized Sensitivity Analysis for the Sample Average Treatment Effect in Paired Observational StudiesJournal of the American Statistical Association, 2019
- Sensitivity Analysis for Inverse Probability Weighting Estimators via the Percentile BootstrapJournal of the Royal Statistical Society Series B: Statistical Methodology, 2019
- On Mitigating the Analytical Limitations of Finely Stratified ExperimentsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2018
- Shape-constrained partial identification of a population mean under unknown probabilities of sample selectionBiometrika, 2017
- Randomization Inference and Sensitivity Analysis for Composite Null Hypotheses With Binary Outcomes in Matched Observational StudiesJournal of the American Statistical Association, 2017
- Impact of Multiple Matched Controls on Design Sensitivity in Observational StudiesBiometrics, 2013
- Interval estimation of population means under unknown but bounded probabilities of sample selectionBiometrika, 2012
- Asymptotic Separability in Sensitivity AnalysisJournal of the Royal Statistical Society Series B: Statistical Methodology, 2000
- A Characterization of Optimal Designs for Observational StudiesJournal of the Royal Statistical Society Series B: Statistical Methodology, 1991