Co-creating a program theory and evaluability assessment for an Irish single-session, synchronous chat-based youth mental health intervention: implications for outcome evaluation
Open Access
- 17 March 2026
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Digital Health
- Vol. 8, 1673317
Abstract
Introduction: Single-session online synchronous chat offers immediate, anonymous, single-session support for young people. However, the drop-in format attracts a diverse population with urgent and varied needs, creating challenges for evaluation. Standardized outcome measures may not capture short-term changes, and randomized controlled trials may be ethically inappropriate. These constraints point to the value of theory-based evaluation approaches rooted in implementation science, which can better capture short-term change, contextual complexity, and real-world variation in service delivery. Methods: This study applied a theory-driven evaluability assessment to Jigsaw Live Chat, an Irish single-session, synchronous chat-based mental health service for youth. A situational analysis, review of the literature, and an internal data review established the intervention context and examined existing evaluative approaches within the literature. Two co-design workshops with staff ( N = 13) identified contextual barriers, preconditions and assumptions for intervention effectiveness, core components, mechanisms of change, and intended outcomes. These elements were synthesized into a program theory, which informed the development of recommendations for evaluation. Results: Several contextual factors were identified as crucial to the success of Jigsaw Live Chat, including cross-collaboration, integration, and buy-in across youth mental health services. Staff defined a range of core components ( N = 12) of the intervention, detailing how these components might influence various outcomes such as accessibility, ease of use, perceived usefulness, help-seeking intention, and decreases in immediate distress and overwhelm among young people. The broader impacts of the service were developed, reflecting the potential contribution of synchronous chat within a wider, integrated youth mental health care system nationally. These findings were integrated into a causal program theory model, and tailored indicators were proposed to support feasible and meaningful evaluation. Discussion: This study presents one of the first published examples of an iterative, collaborative evaluability assessment of a single-session synchronous chat-based mental health support service. The findings clarify how such interventions operate within specific organizational and sociocultural contexts and provide the foundation for future adaptation and evaluation aimed at improving service delivery and outcomes for young people.Keywords
This publication has 64 references indexed in Scilit:
- Idiographic patient reported outcome measures (I‐PROMs) for routine outcome monitoring in psychological therapies: Position paperJournal of Clinical Psychology, 2022
- Designing and scaling up integrated youth mental health careWorld Psychiatry, 2022
- The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual realityWorld Psychiatry, 2021
- Exploring hope and expectations in the youth mental health online counselling environmentComputers in Human Behavior, 2016
- A naturalistic study of the effects of synchronous online chat counselling on young people's psychological distress, life satisfaction and hopeCounselling and Psychotherapy Research, 2015
- Feasibility, Acceptability, and Preliminary Efficacy of a Live-Chat Social Media Intervention to Reduce HIV Risk Among Young Men Who Have Sex With MenAIDS and Behavior, 2014
- The Behavior Change Technique Taxonomy (v1) of 93 Hierarchically Clustered Techniques: Building an International Consensus for the Reporting of Behavior Change InterventionsAnnals of Behavioral Medicine, 2013
- Bridging Research and PracticeAmerican Journal of Preventive Medicine, 2012
- Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challengesAdministration and Policy in Mental Health and Mental Health Services Research, 2008
- A Brief Measure for Assessing Generalized Anxiety DisorderArchives of internal medicine (1960), 2006