Subgroup Analysis in Social Experiments: Measuring Program Impacts Based on Post-Treatment Choice
Author:
- 1 June 2003
- journal article
- Published by SAGE Publications in American Journal of Evaluation
- Vol. 24 (2) , 157-187
Abstract
A fundamental question within the field of program evaluation is “Do social programs work?” Although experiments allow us to answer this question with certainty, they have some limitations. Experiments generate mean program impacts and even mean impacts by subgroup, but they often leave unexplored the impacts on subgroups determined by treatment use. This work proposes a methodology for analyzing the impacts of social programs on previously unexamined subgroups. Rather than using a single trait to define subgroups—which is currently the dominant method of subgroup analysis—the proposed approach estimates the impact of programs on subgroups identified by a post-treatment choice while still maintaining the integrity of the experimental research design. Analysis of data from the experimental evaluation of New York State’s Child Assistance Program (CAP) provides an application of the proposed technique.Keywords
This publication has 17 references indexed in Scilit:
- Causal Effects in Nonexperimental Studies: Reevaluating the Evaluation of Training ProgramsJournal of the American Statistical Association, 1999
- Accounting for Dropouts in Evaluations of Social ProgramsReview of Economics and Statistics, 1998
- The Distributional Impacts of Social ProgramsEvaluation Review, 1997
- Does Mandatory Basic Education Improve Achievement Test Scores of Afdc Recipients?Evaluation Review, 1997
- Learning about Treatment Effects from Experiments with Random Assignment of TreatmentsThe Journal of Human Resources, 1996
- AGE, CRIMINAL CAREERS, AND POPULATION HETEROGENEITY: SPECIFICATION AND ESTIMATION OF A NONPARAMETRIC, MIXED POISSON MODEL*Criminology, 1993
- Subgroup Impacts of Large-Scale Welfare Employment ProgramsReview of Economics and Statistics, 1993
- Alternative methods for evaluating the impact of interventions1985
- Alternative methods for evaluating the impact of interventionsJournal of Econometrics, 1985
- Accounting for No-Shows in Experimental Evaluation DesignsEvaluation Review, 1984