Design Considerations in Multisite Randomized Trials Probing Moderated Treatment Effects
Open Access
- 14 October 2020
- journal article
- research article
- Published by American Educational Research Association (AERA) in Journal of Educational and Behavioral Statistics
- Vol. 46 (5) , 527-559
Abstract
Past research has demonstrated that treatment effects frequently vary across sites (e.g., schools) and that such variation can be explained by site-level or individual-level variables (e.g., school size or gender). The purpose of this study is to develop a statistical framework and tools for the effective and efficient design of multisite randomized trials (MRTs) probing moderated treatment effects. The framework considers three core facets of such designs: (a) Level 1 and Level 2 moderators, (b) random and nonrandomly varying slopes (coefficients) of the treatment variable and its interaction terms with the moderators, and (c) binary and continuous moderators. We validate the formulas for calculating statistical power and the minimum detectable effect size difference with simulations, probe its sensitivity to model assumptions, execute the formulas in accessible software, demonstrate an application, and provide suggestions in designing MRTs probing moderated treatment effects.Keywords
Funding Information
- National Science Foundation (DRL- 1913563)
This publication has 25 references indexed in Scilit:
- PowerUp!: A Tool for Calculating Minimum Detectable Effect Sizes and Minimum Required Sample Sizes for Experimental and Quasi-Experimental Design StudiesJournal of Research on Educational Effectiveness, 2013
- Power Analysis in Two-Level Unbalanced DesignsThe Journal of Experimental Education, 2010
- An Examination of the Precision and Technical Accuracy of the First Wave of Group-Randomized Trials Funded by the Institute of Education SciencesEducational Evaluation and Policy Analysis, 2009
- Empirical Benchmarks for Interpreting Effect Sizes in ResearchChild Development Perspectives, 2008
- The Power of the Test for Treatment Effects in Three-Level Block Randomized DesignsJournal of Research on Educational Effectiveness, 2008
- Intraclass Correlation Values for Planning Group-Randomized Trials in EducationEducational Evaluation and Policy Analysis, 2007
- Using Covariates to Improve Precision for Studies That Randomize Schools to Evaluate Educational InterventionsEducational Evaluation and Policy Analysis, 2007
- Power and Sample Size in Multilevel Linear Models2005
- Statistical power and optimal design for multisite randomized trials.Psychological Methods, 2000
- Minimum Detectable EffectsEvaluation Review, 1995