A Theory of Experimenters: Robustness, Randomization, and Balance
Authors:
- 1 April 2020
- journal article
- research article
- Published by American Economic Association in American Economic Review
- Vol. 110 (4) , 1206-1230
Abstract
This paper studies the problem of experiment design by an ambiguity-averse decision-maker who trades off subjective expected performance against robust performance guarantees. This framework accounts for real-world experimenters’ preference for randomization. It also clarifies the circumstances in which randomization is optimal: when the available sample size is large and robustness is an important concern. We apply our model to shed light on the practice of rerandomization, used to improve balance across treatment and control groups. We show that rerandomization creates a trade-off between subjective performance and robust performance guarantees. However, robust performance guarantees diminish very slowly with the number of rerandomizations. This suggests that moderate levels of rerandomization usefully expand the set of acceptable compromises between subjective performance and robustness. Targeting a fixed quantile of balance is safer than targeting an absolute balance objective. (JEL C90, D81)Keywords
This publication has 36 references indexed in Scilit:
- THE 2009 LAWRENCE R. KLEIN LECTURE: DIVERSIFIED TREATMENT UNDER AMBIGUITY*International Economic Review, 2009
- In Pursuit of Balance: Randomization in Practice in Development Field ExperimentsAmerican Economic Journal: Applied Economics, 2009
- Statistical Treatment Rules for Heterogeneous PopulationsEconometrica, 2004
- Worms: Identifying Impacts on Education and Health in the Presence of Treatment ExternalitiesEconometrica, 2003
- Learning and Strategic PricingEconometrica, 1996
- Bias Reduction Using Mahalanobis-Metric Matching1980
- Using Multivariate Matched Sampling and Regression Adjustment to Control Bias in Observational StudiesJournal of the American Statistical Association, 1979
- Prospect Theory: An Analysis of Decision under RiskEconometrica, 1979
- Robust Estimation of a Location ParameterThe Annals of Mathematical Statistics, 1964
- Equivalent Comparisons of ExperimentsThe Annals of Mathematical Statistics, 1953