Improving the reporting of research impact assessments: a systematic review of biomedical funder research impact assessments
Open Access
- 22 July 2024
- journal article
- research article
- Published by Oxford University Press (OUP) in Research Evaluation
- Vol. 33
Abstract
The field of research impact assessment (RIA) has seen remarkable growth over the past three decades. Increasing numbers of RIA frameworks have been developed and applied by research funders and new technologies can capture some research impacts automatically. However, RIAs are too different to draw comparable conclusions about what type of methods, data or processes are best suited to assess research impacts of different kinds, or how funders should most efficiently implement RIAs. To usher in the next era of RIA and mature the field, future RIA methodologies should become more transparent, standardized and easily implementable. Key to these efforts is an improved understanding of how to practically implement and report on RIA at the funder-level. Our aim is to address this gap through two major contributions. First, we identify common items across existing best practice guidelines for RIA, creating a preliminary reporting checklist for standardized RIA reporting. Next, we systematically reviewed studies examining funders’ assessment of biomedical grant portfolios to examine how funders reported the results of their RIAs across the checklist, as well as the operational steps funders took to perform their RIA and the variation in how funders implemented the same RIA frameworks. We compare evidence on current RIA practices with the reporting checklist to identify good practice for RIA reporting, gaps in the evidence base for future research, and recommendations for future effective RIA.Keywords
This publication has 63 references indexed in Scilit:
- Development of a novel and more holistic approach for assessing impact in health and medical research: the Research Impact Assessment FrameworkAustralian Health Review, 2023
- Re-thinking research impact: voice, context and power at the interface of science, policy and practiceSustainability Science, 2022
- PRISMA2020 : An R package and Shiny app for producing PRISMA 2020‐compliant flow diagrams, with interactivity for optimised digital transparency and Open SynthesisCampbell Systematic Reviews, 2022
- Insights from the evaluations of the NIH Centers for Accelerated Innovation and Research Evaluation and Commercialization Hubs programsJournal of Clinical and Translational Science, 2021
- Evaluation of the public health achievements made by projects supported by a federal contract mechanism at the Centers for Disease Control and Prevention (CDC), USAEvaluation and Program Planning, 2021
- Exploring research impact models: A systematic scoping reviewResearch Evaluation, 2021
- Evaluating impact from research: A methodological frameworkResearch Policy, 2021
- Using the payback framework to evaluate the outcomes of pilot projects supported by the Georgia Clinical and Translational Science AllianceJournal of Clinical and Translational Science, 2020
- Assessing research impact: Australian Research Council criteria and the case of Family Wellbeing researchEvaluation and Program Planning, 2019
- Exploring the effectiveness, efficiency and equity (3e’s) of research and research impact assessmentPalgrave Communications, 2016