Covariate Balance in Simple, Stratified and Clustered Comparative Studies
Authors:
- Ben B. HansenBHBen B. Hansen
- Jake BowersJBJake Bowers
Open Access
- 1 May 2008
- journal article
- Published by Institute of Mathematical Statistics in Statistical Science
- Vol. 23 (2) , 219-236
Abstract
In randomized experiments, treatment and control groups should be roughly the same—balanced—in their distributions of pretreatment variables. But how nearly so? Can descriptive comparisons meaningfully be paired with significance tests? If so, should there be several such tests, one for each pretreatment variable, or should there be a single, omnibus test? Could such a test be engineered to give easily computed p-values that are reliable in samples of moderate size, or would simulation be needed for reliable calibration? What new concerns are introduced by random assignment of clusters? Which tests of balance would be optimal? To address these questions, Fisher’s randomization inference is applied to the question of balance. Its application suggests the reversal of published conclusions about two studies, one clinical and the other a field experiment in political participation.Keywords
All Related Versions
This publication has 37 references indexed in Scilit:
- The essential role of balance tests in propensity‐matched observational studies: Comments on ‘A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003’ by Peter Austin, Statistics in MedicineStatistics in Medicine, 2008
- A Lego System for Conditional InferenceThe American Statistician, 2006
- Propensity Score Matching and Variations on the Balancing TestSSRN Electronic Journal, 2006
- Comment: Randomized Confidence Intervals and the Mid-P ApproachStatistical Science, 2005
- Do Get-Out-the-Vote Calls Reduce Turnout? The Importance of Statistical Methods for Field ExperimentsAmerican Political Science Review, 2005
- Comparing completely and stratified randomized designs in cluster randomized trials when the stratifying factor is cluster size: a simulation studyStatistics in Medicine, 2004
- Sample size calculations for ordered categorical dataStatistics in Medicine, 1993
- Reducing Bias in Observational Studies Using Subclassification on the Propensity ScoreJournal of the American Statistical Association, 1984
- On Simpson's Paradox and the Sure-Thing PrincipleJournal of the American Statistical Association, 1972
- Probability Inequalities for Sums of Bounded Random VariablesJournal of the American Statistical Association, 1963