Exploring influences on evaluation practice: a case study of a national physical activity programme
Authors:
Open Access
- 16 February 2021
- journal article
- research article
- Published by Springer Nature in International Journal of Behavioral Nutrition and Physical Activity
- Vol. 18 (1) , 1-24
Abstract
Background: Interventions to improve physical activity behaviour are a core part of public health policy and practice. It is essential that we evaluate these interventions and use the evidence to inform decisions to improve population health. Evaluation of ‘real-world’ interventions provide an opportunity to generate practice-relevant evidence, however these interventions are difficult to evaluate. Various guidelines have been developed to facilitate evaluation, but evidence about their effectiveness in practice is limited. To explore influences on evaluation practice in an applied context, we conducted a case study of Sport England’s ‘Get Healthy Get Active’ (GHGA) programme. This was a national programme that funded 33 projects that were delivered and evaluated across England. The programme was chosen as it was designed to generate evidence on the role of sport in increasing physical activity and improving health. The study aimed to explore and appraise whether strategies intended to facilitate project evaluation, including funder requirements to use a standardised evaluation framework and specific data collection methods, were effective in generating evidence that enabled the programme to meet its aims.Methods: We applied a collective case study design involving 35 semi-structured interviews, and documentary analysis of multiple sources of evidence from 23 physical activity projects funded by GHGA. We applied thematic and framework analysis. We developed a logic model and mapped actual outcomes against intended outcomes. A narrative synthesis is provided. We discuss implications for the effective commissioning and evaluation of public health interventions.Results: We identified five main themes of influences on evaluation practices that can act as barriers and facilitators to good practice: programme and project design; evaluation design; partnerships; resources; and organisational structures and systems. These influences are context-specific and operate through a complex set of interactions.Conclusion: Developing a better understanding of how influences on evaluation practice can act as facilitators or barriers is vital to help close current gaps in the evidence-based practice cycle. Critically, organisational structures and systems are needed to facilitate collaborative decision making; integration of projects and evaluation across partners organisations; transfer of knowldege and insights between stakeholders; and more rapid feedback and dissemination.Keywords
This publication has 48 references indexed in Scilit:
- Using the framework method for the analysis of qualitative data in multi-disciplinary health researchBMC Medical Research Methodology, 2013
- Challenges to evaluating complex interventions: a content analysis of published papersBMC Public Health, 2013
- Strengthening evaluation to capture the breadth of public health practice: ideal vs. realJournal of Public Health, 2012
- The case study approachBMC Medical Research Methodology, 2011
- Public health evaluation in the twenty-first century: time to see the wood as well as the treesJournal of Public Health, 2010
- Realistic Evaluation and Case StudiesEvaluation, 2009
- Developing and evaluating complex interventions: the new Medical Research Council guidanceBMJ, 2008
- Pathways to “Evidence-Informed” Policy and Practice: A Framework for ActionPLOS Medicine, 2005
- Evaluating the public health impact of health promotion interventions: the RE-AIM framework.American Journal of Public Health, 1999
- Realistic EvaluationThe British Journal of Sociology, 1998