Development of the ASSESS tool: a comprehenSive tool to Support rEporting and critical appraiSal of qualitative, quantitative, and mixed methods implementation reSearch outcomes
Authors:
Top Cited Papers
Open Access
- 28 March 2022
- journal article
- research article
- Published by Springer Nature in Implementation Science Communications
- Vol. 3 (1) , 1-13
Abstract
Background: Several tools to improve reporting of implementation studies for evidence-based decision making have been created; however, no tool for critical appraisal of implementation outcomes exists. Researchers, practitioners, and policy makers lack tools to support the concurrent synthesis and critical assessment of outcomes for implementation research. Our objectives were to develop a comprehensive tool to (1) describe studies focused on implementation that use qualitative, quantitative, and/or mixed methodologies and (2) assess risk of bias of implementation outcomes. Methods: A hybrid consensus-building approach combining Delphi Group and Nominal Group techniques (NGT) was modeled after comparative methodologies for developing health research reporting guidelines and critical appraisal tools. First, an online modified NGT occurred among a small expert panel (n = 5), consisting of literature review, item generation, round robin with clarification, application of the tool to various study types, voting, and discussion. This was followed by a larger e-consensus meeting and modified Delphi process with implementers and implementation scientists (n = 32). New elements and elements of various existing tools, frameworks, and taxonomies were combined to produce the ASSESS tool. Results: The 24-item tool is applicable to a broad range of study designs employed in implementation science, including qualitative studies, randomized-control trials, non-randomized quantitative studies, and mixed methods studies. Two key features are a section for assessing bias of the implementation outcomes and sections for describing the implementation strategy and intervention implemented. An accompanying explanation and elaboration document that identifies and describes each of the items, explains the rationale, and provides examples of reporting and appraising practice, as well as templates to allow synthesis of extracted data across studies and an instructional video, has been prepared. Conclusions: The comprehensive, adaptable tool to support both reporting and critical appraisal of implementation science studies including quantitative, qualitative, and mixed methods assessment of intervention and implementation outcomes has been developed. This tool can be applied to a methodologically diverse and growing body of implementation science literature to support reviews or meta-analyses that inform evidence-based decision-making regarding processes and strategies for implementation.Keywords
Funding Information
- National Center for Advancing Translational Sciences (TL1 TR001447)
This publication has 26 references indexed in Scilit:
- Protocol: Adaptive Implementation of Effective Programs Trial (ADEPT): cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders programImplementation Science, 2014
- Bridging the gap between the economic evaluation literature and daily practice in occupational health: a qualitative study among decision-makers in the healthcare sectorImplementation Science, 2013
- Factors influencing the implementation of fall-prevention programmes: a systematic review and synthesis of qualitative studiesImplementation Science, 2012
- Effectiveness-implementation Hybrid DesignsMedical Care, 2012
- Assessment of a Novel Hybrid Delphi and Nominal Groups Technique to Evaluate Quality IndicatorsHealth Services Research, 2011
- Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research AgendaAdministration and Policy in Mental Health and Mental Health Services Research, 2010
- Guidance for Developers of Health Research Reporting GuidelinesPLOS Medicine, 2010
- Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challengesAdministration and Policy in Mental Health and Mental Health Services Research, 2008
- A comparison of formal consensus methods used for developing clinical guidelinesJournal of Health Services Research & Policy, 2006
- Judgment under Uncertainty: Heuristics and BiasesScience, 1974