Critical Issues and Common Pitfalls in Designing and Conducting Impact Studies in Education
Authors:
- 1 September 2010
- journal article
- Published by American Educational Research Association (AERA) in Educational Evaluation and Policy Analysis
- Vol. 32 (3) , 351-371
Abstract
Drawing on our five years of experience developing WWC evidence standards and reviewing studies against those standards as well as current literature on the design of impact studies, we highlight in this paper some of the most critical issues and common pitfalls in designing and conducting impact studies in education, and provide practical guidance on addressing these issues and avoiding common pitfalls. This paper serves as a quick reference guide for researchers who design and conduct impact studies in education and related fields. Information presented in this paper may also help research users make better informed judgments about the quality of the research and the credibility of the evidence produced from the research.Keywords
This publication has 43 references indexed in Scilit:
- Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within‐study comparisonsJournal of Policy Analysis and Management, 2008
- Using Covariates to Improve Precision for Studies That Randomize Schools to Evaluate Educational InterventionsEducational Evaluation and Policy Analysis, 2007
- Standards for Reporting on Empirical Social Science Research in AERA PublicationsEducational Researcher, 2006
- The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and ElaborationAnnals of Internal Medicine, 2001
- Multiple comparisons: philosophies and illustrationsAmerican Journal of Physiology-Regulatory, Integrative and Comparative Physiology, 2000
- Minimum Detectable EffectsEvaluation Review, 1995
- Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple TestingJournal of the Royal Statistical Society Series B: Statistical Methodology, 1995
- Effects of Misspecification of the Propensity Score on Estimators of Treatment Effect1993
- What is coefficient alpha? An examination of theory and applications.Journal of Applied Psychology, 1993
- A Multiple Comparison Procedure for Comparing Several Treatments with a ControlJournal of the American Statistical Association, 1955