Integrating a Comparison Group Design into a Theory of Change Evaluation: The Case of the Urban Health Initiative
Authors:
- Beth C. WeitzmanBWBeth C. Weitzman
- Diana SilverDSDiana Silver
- Keri-Nicole DillmanKDKeri-Nicole Dillman
- 1 December 2002
- journal article
- research article
- Published by SAGE Publications in American Journal of Evaluation
- Vol. 23 (4) , 371-385
Abstract
This paper describes how we strengthened the theory of change approach to evaluating a complex social initiative by integrating it with a quasi-experimental, comparison group design. We also demonstrate the plausibility of selecting a credible comparison group through the use of cluster analysis, and describe our work in validating that analysis with additional measures. The integrated evaluation design relies on two points of comparison: (1) program theory to program experience; and (2) program cities to comparison cities. We describe how we are using this integrated design to evaluate the Robert Wood Johnson Foundation’s Urban Health Initiative, an effort that aims to improve health and safety outcomes for children and youth in five distressed urban areas through a process of citywide, multi-sector planning and changed public and private systems. We also discuss how the use of two research frameworks and multiple methods can enrich our ability to test underlying assumptions and evaluate overall program effects. Using this integrated approach has provided evidence that the earliest phases of this initiative are unfolding as the theory would predict, and that the comparison cities are not undergoing a similar experience to those in UHI. Despite many remaining limitations, this integrated evaluation can provide greater confidence in assessing whether future changes in health and safety outcomes may have resulted from the Urban Health Initiative (UHI).Keywords
This publication has 8 references indexed in Scilit:
- Setting an Agenda for Local ActionPolicy Studies Journal, 2002
- What We Argue about When We Argue about RegionalismJournal of Urban Affairs, 2001
- The false choice between theory‐based evaluation and experimentationNew Directions for Evaluation, 2000
- Program theory evaluation: Practice, promise, and problemsNew Directions for Evaluation, 2000
- Using program theory to replicate successful programsNew Directions for Evaluation, 2000
- How Can Theory-Based Evaluation Make Greater Headway?Evaluation Review, 1997
- Measuring Community Indicators1997
- Regional Politics: America in a Post-City Age1996