The mixing of methods: A three-step process for improving rigour in impact evaluations
Author:
- 22 January 2012
- journal article
- Published by SAGE Publications in Evaluation
- Vol. 18 (1) , 5-25
Abstract
This article describes a systematic process that is helpful in improving impact evaluation assignments, within restricted budgets and timelines. It involves three steps: a rethink of the key questions of the evaluation to develop more relevant, specific questions; a way of designing a mix of research methods to generate evidence that supports more valid conclusions; and a step that aims to make evaluation outputs more useful. The approach is illustrated through two examples: one on measuring income impacts in an irrigated horticulture programme in Nepal, Zambia and Ethiopia; and another on the assessment of changes in organizational capacities for collective marketing by smallholders in Bolivia. The article demonstrates that this simple, straightforward and structured three-step process helped to reduce the tendency to one-method designs. Enhanced critical reflection within the team allowed for greater sensitivity to validity threats and the creativity to find ways to handle them.Keywords
This publication has 18 references indexed in Scilit:
- Evaluating aid: An adolescent domain of practiceEvaluation, 2011
- Development impacts of value chain interventions: how to collect credible evidence and draw valid conclusions in impact evaluations?Journal on Chain and Network Science, 2011
- A Contribution to Current Debates in Impact EvaluationEvaluation, 2010
- Impact Assessments in Finance and Private Sector Development: What Have We Learned and What Should We Learn?The World Bank Research Observer, 2009
- Matching impact evaluation design to the nature of the intervention and the purpose of the evaluationJournal of Development Effectiveness, 2009
- Using Programme Theory to Evaluate Complicated and Complex Aspects of InterventionsEvaluation, 2008
- Toward a Pluralistic Vision of MethodologyPolitical Analysis, 2006
- Theory-Based Evaluation and Types of ComplexityEvaluation, 2004
- How Can Theory-Based Evaluation Make Greater Headway?Evaluation Review, 1997
- Assessing the Case for Social ExperimentsJournal of Economic Perspectives, 1995