Quantitative measures of health policy implementation determinants and outcomes: a systematic review
Authors:
Open Access
- 19 June 2020
- journal article
- review article
- Published by Springer Nature in Implementation Science
- Vol. 15 (1) , 1-17
Abstract
Background: Public policy has tremendous impacts on population health. While policy development has been extensively studied, policy implementation research is newer and relies largely on qualitative methods. Quantitative measures are needed to disentangle differential impacts of policy implementation determinants (i.e., barriers and facilitators) and outcomes to ensure intended benefits are realized. Implementation outcomes include acceptability, adoption, appropriateness, compliance/fidelity, feasibility, penetration, sustainability, and costs. This systematic review identified quantitative measures that are used to assess health policy implementation determinants and outcomes and evaluated the quality of these measures. Methods: Three frameworks guided the review: Implementation Outcomes Framework (Proctor et al.), Consolidated Framework for Implementation Research (Damschroder et al.), and Policy Implementation Determinants Framework (Bullock et al.). Six databases were searched: Medline, CINAHL Plus, PsycInfo, PAIS, ERIC, and Worldwide Political. Searches were limited to English language, peer-reviewed journal articles published January 1995 to April 2019. Search terms addressed four levels: health, public policy, implementation, and measurement. Empirical studies of public policies addressing physical or behavioral health with quantitative self-report or archival measures of policy implementation with at least two items assessing implementation outcomes or determinants were included. Consensus scoring of the Psychometric and Pragmatic Evidence Rating Scale assessed the quality of measures. Results: Database searches yielded 8417 non-duplicate studies, with 870 (10.3%) undergoing full-text screening, yielding 66 studies. From the included studies, 70 unique measures were identified to quantitatively assess implementation outcomes and/or determinants. Acceptability, feasibility, appropriateness, and compliance were the most commonly measured implementation outcomes. Common determinants in the identified measures were organizational culture, implementation climate, and readiness for implementation, each aspects of the internal setting. Pragmatic quality ranged from adequate to good, with most measures freely available, brief, and at high school reading level. Few psychometric properties were reported. Conclusions: Well-tested quantitative measures of implementation internal settings were under-utilized in policy studies. Further development and testing of external context measures are warranted. This review is intended to stimulate measure development and high-quality assessment of health policy implementation outcomes and determinants to help practitioners and researchers spread evidence-informed policies to improve population health. Registration: Not registeredKeywords
Funding Information
- Foundation for Barnes-Jewish Hospital (4798, ICTS Research 615735 from Washington University in St. Louis Institute of Clinical and Translational Science Pilot Program, NIH/National Center for Advancing Translational Sciences (NCATS) grant UL1 TR002345)
- National Cancer Institute (P50CA244431)
- Centers for Disease Control and Prevention (U48DP006395-01-00)
- National Institute of Mental Health (R01MH106510)
- National Institute of Diabetes and Digestive and Kidney Diseases (P30DK020579)
This publication has 92 references indexed in Scilit:
- Policy to implementation: evidence-based practice in community mental health – study protocolImplementation Science, 2013
- Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measuresImplementation Science, 2013
- Health worker perspectives on user fee removal in ZambiaHuman Resources for Health, 2012
- Comparing the effectiveness of using generic and specific search terms in electronic databases to identify health outcomes for a systematic review: a prospective comparative study of literature search methodsBMJ Open, 2012
- An ecological quantification of the relationships between water, sanitation and infant, child, and maternal mortalityEnvironmental Health, 2012
- Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research AgendaAdministration and Policy in Mental Health and Mental Health Services Research, 2010
- A cross-sectional study of the number and frequency of terms used to refer to knowledge translation in a body of health literature in 2006: a Tower of Babel?Implementation Science, 2010
- Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instrumentsQuality of Life Research, 2009
- Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation scienceImplementation Science, 2009
- National quality improvement policies and strategies in European healthcare systemsQuality and Safety in Health Care, 2009