Statistical Power for Estimating Treatment Effects Using Difference-in-Differences and Comparative Interrupted Time Series Designs with Variation in Treatment Timing
Preprint
Abstract
This article develops new closed-form variance expressions for power analyses for commonly used difference-in-differences (DID) and comparative interrupted time series (CITS) panel data estimators. The main contribution is to incorporate variation in treatment timing into the analysis. The power formulas also account for other key design features that arise in practice: autocorrelated errors, unequal measurement intervals, and clustering due to the unit of treatment assignment. We consider power formulas for both cross-sectional and longitudinal models and allow for covariates. An illustrative power analysis provides guidance on appropriate sample sizes. The key finding is that accounting for treatment timing increases required sample sizes. Further, DID estimators have considerably more power than standard CITS and ITS estimators. An available Shiny R dashboard performs the sample size calculations for the considered estimators.Keywords
All Related Versions
This publication has 34 references indexed in Scilit:
- Design-based analysis in Difference-In-Differences settings with staggered adoptionJournal of Econometrics, 2022
- Estimating dynamic treatment effects in event studies with heterogeneous treatment effectsJournal of Econometrics, 2021
- Panel data and experimental designJournal of Development Economics, 2020
- Simulation-based power and sample size calculation for designing interrupted time series analyses of count outcomes in evaluation of health policy interventionsContemporary Clinical Trials Communications, 2020
- Interrupted Time Series Design2014
- Beyond baseline and follow-up: The case for more T in experimentsJournal of Development Economics, 2012
- Simulation-based power calculation for designing interrupted time series analyses of health policy interventionsJournal of Clinical Epidemiology, 2011
- Is regression adjustment supported by the Neyman model for causal inference?Journal of Statistical Planning and Inference, 2010
- On regression adjustments to experimental dataAdvances in Applied Mathematics, 2008
- Repeated measures in clinical trials: Analysis using mean summary statistics and its implications for designStatistics in Medicine, 1992