Gill Westhorp

It is increasingly recognised that ‘big problems’ in health and social care require well-designed solutions and robust evaluation, including attention to costs as well as outcomes. This is especially important in the pervasive environment of resource scarcity. Decisions need to be made that consider not only whether and how an intervention works but also affordability and efficiency in different circumstances and populations. As researchers tackle these complex issues, interest in bringing together realist and economic evaluation is growing. Responding to this, we discuss how using programme theory within a novel realist economic evaluation approach can be used in articulating how interventions may (or may not) bring about cost-effective change. We identify what initial programme theory in realist economic evaluation might look like in practice, drawing on data from three pilot realist economic evaluation studies.

Over a quarter of adults in Europe have limited health literacy (LHL). LHL hampers their ability to find, understand, and implement medical information, re

Realist evaluation is a theory-based approach to evaluation which examines what works in what respects and to what extent, for whom, in what contexts and how. It focuses on identifying underlying mechanisms which cause outcomes and the contextual factors necessary for their operation. Despite realist evaluation being ‘agnostic’ about methods, qualitative approaches have been heavily favoured in practice. This paper argues that the use of surveys can considerably strengthen standard approaches to realist analysis while still being consistent with the philosophical underpinnings of realist evaluation. It provides examples drawn from various research and evaluation projects. The paper demonstrates that when program theory is well-developed, quantitative data collected through surveys can be used to examine all three elements of realist hypothesis testing: contexts, mechanisms and outcomes. The paper concludes with five principles for the design and analysis of survey data in realist evaluations.

Billions of dollars continue to be provided in foreign aid each year. However, few scholarly studies have examined whether the outcomes from foreign aid interventions are sustained after donor funding has ceased. This paper examines current approaches to assessing this issue before arguing that a realist evaluation approach is ideally suited to understand why and how sustained outcomes are—or are not—achieved. It contributes to the existing literature by presenting three new frameworks to examine the sustainability of outcomes in international development as well as some Context‐Mechanism‐Outcome statements. Implications for governments, communities, households and donor/implementing organisations are discussed.

Background: Shared Decision-making (SDM), a medical decision-making model, was popularized in the late 1980s in reaction to then predominate paternalistic decision-making, aiming to better meet the needs of patients. Extensive research has been conducted internationally examining the benefits of SDM implementation; however, existing theory on how SDM works, for whom, in which circumstances, and why is limited. While literature has shown positive patient, health care provider, and system benefits (SDM outputs), further research is required to understand the nuances of this type of decision-making. As such, we set out to address: “In which situations, how, why, and for whom does SDM between patients and health care providers contribute to improved engagement in the Shared Decision-making process?” Methods: To achieve our study goals we conducted a seven-step realist synthesis process, which included: (1) preliminary program theory development, (2) search strategy development, (3) selection and appraisal of literature in accordance with realist methodology, (4) data extraction, (5) identification of relevant formal theories, (6) data analysis and synthesis, and (7) formation of a revised program theory with the input of stakeholders. This process was done in accordance with RAMESES guidelines and publication standards for a realist synthesis. Expert consultations were also held to ensure consistency within the SDM literature. Results: Through our realist synthesis, we developed a program theory of SDM which includes three contexts (pre-existing relationship, difficulty of decision, and system support), eight mechanism sets (anxiety, trust, perception of other party capacity, perception of time, self-efficacy, world view, perception of capacity to external support, and recognition of decision), and one outcome (engagement in SDM). Conclusions: As far as the authors of this paper are aware, this paper is the first to begin unpacking how SDM works, for whom, in which circumstances, and why. By examining key mechanism sets and exploring how they facilitate or inhibit SDM, we have produced a program theory that may assist health care professionals, policy makers, and patients. While further research is suggested to further unpack the concepts identified within this paper, this provides an initial understanding into the theory behind SDM. Registration: PROSPERO: CRD42017062609.

Background: Many of the problems confronting policy- and decision-makers, evaluators and researchers today are complex, as are the interventions designed to tackle them. Their success depends both on individuals’ responses and on the wider context of people’s lives. Realist evaluation tries to make sense of these complex interventions. It is a form of theory-driven evaluation, based on realist philosophy, that aims to understand why these complex interventions work, how, for whom, in what context and to what extent.Objectives: Our objectives were to develop (a) quality standards, (b) reporting standards, (c) resources and training materials, (d) information and resources for patients and other lay participants and (e) to build research capacity among those interested in realist evaluation.Methods: To develop the quality and reporting standards, we undertook a thematic review of the literature, supplemented by our content expertise and feedback from presentations and workshops. We synthesised findings into briefing materials for realist evaluations for the Delphi panel (a structured method using experts to develop consensus). To develop our resources and training materials, we drew on our experience in developing and delivering education materials, feedback from the Delphi panel, the RAMESES JISCMail e-mail list, training workshops and feedback from training sessions. To develop information and resources for patients and other lay participants in realist evaluation, we convened a group consisting of patients and the public. We built research capacity by running workshops and training sessions.Results: Our literature review identified 152 realist evaluations, and when 37 of these had been analysed we were able to develop our briefing materials for the Delphi panel. The Delphi panel comprised 35 members from 27 organisations across six countries and five disciplines. Within three rounds, the panels had reached a consensus on 20 key reporting standards. The quality standards consist of eight criteria for realist evaluations. We developed resources and training materials for 15 theoretical and methodological topics. All resources are available online (www.ramesesproject.org). We provided methodological support to 17 projects and presentations or workshops to help build research capacity in realist evaluations to 29 organisations. Finally, we produced a generic patient information leaflet for lay participants in realist evaluations.Limitations: Our project had ambitious goals that created a substantial workload, leading to the need to prioritise objectives. For example, we truncated the literature review and focused on standards and training material development.Conclusions: Although realist evaluation holds much promise, misunderstandings and misapplications of it are common. We hope that our project’s outputs and activities will help to address these problems. Our resources are the start of an iterative journey of refinement and development of better resources for realist evaluations. The RAMESES II project seeks not to produce the last word on these issues, but to capture current expertise and establish an agreed state of the science. Much methodological development is needed in realist evaluation but this can take place only if there is a sufficient pool of highly skilled realist evaluators. Capacity building is the next key step in realist evaluation.Funding: The National Institute for Health Research Health Services and Delivery Research programme.

This case demonstrates the integration of realist action research and co-design to address the complex social problem of long-term reliance on welfare benefits. Realist action research combines a realist philosophy of science and the questions that flow from it with an action research cycle. Realist approaches to evaluation and planning seek to explain for whom in what contexts and how impacts are generated or might be generated. Action research seeks to solve real world problems, trialling solutions until a ‘best fit’ solution is reached. The article describes the principles underpinning the methodology and the research cycles through which the project worked – situation analysis; prioritizing; co-design; trialling and further refining ideas for change. It demonstrates the development and testing of program theory for one service innovation. It also reflects on the experience and potential benefits of this approach.

In this paper, we respond to a paper by Jamal and colleagues published in Trials in October 2015 and take an opportunity to continue the much-needed debate about what applied scientific realism is. The paper by Jamal et al. is useful because it exposes the challenges of combining a realist evaluation approach (as developed by Pawson and Tilley) with the randomised controlled trial (RCT) design. We identified three fundamental differences that are related to paradigmatic differences in the treatment of causation between post-positivist and realist logic: (1) the construct of mechanism, (2) the relation between mediators and moderators on one hand and mechanisms and contexts on the other hand, and (3) the variable-oriented approach to analysis of causation versus the configurational approach. We show how Jamal et al. consider mechanisms as observable, external treatments and how their approach reduces complex causal processes to variables. We argue that their proposed RCT design cannot provide a truly realist understanding. Not only does the proposed realist RCT design not deal with the RCT’s inherent inability to “unpack” complex interventions, it also does not enable the identification of the dynamic interplay among the intervention, actors, context, mechanisms and outcomes, which is at the core of realist research. As a result, the proposed realist RCT design is not, as we understand it, genuinely realist in nature.