Assessing research impact in academic clinical medicine: a study using Research Excellence Framework pilot impact indicators
Open Access
- 23 December 2012
- journal article
- Published by Springer Nature in BMC Health Services Research
- Vol. 12 (1) , 478
Abstract
Background: Funders of medical research the world over are increasingly seeking, in research assessment, to complement traditional output measures of scientific publications with more outcome-based indicators of societal and economic impact. In the United Kingdom, the Higher Education Funding Council for England (HEFCE) developed proposals for the Research Excellence Framework (REF) to allocate public research funding to higher education institutions, inter alia, on the basis of the social and economic impact of their research. In 2010, it conducted a pilot exercise to test these proposals and refine impact indicators and criteria. Methods: The impact indicators proposed in the 2010 REF impact pilot exercise are critically reviewed and appraised using insights from the relevant literature and empirical data collected for the University of Oxford’s REF pilot submission in clinical medicine. The empirical data were gathered from existing administrative sources and an online administrative survey carried out by the university’s Medical Sciences Division among 289 clinical medicine faculty members (48.1% response rate). Results: The feasibility and scope of measuring research impact in clinical medicine in a given university are assessed. Twenty impact indicators from seven categories proposed by HEFCE are presented; their strengths and limitations are discussed using insights from the relevant biomedical and research policy literature. Conclusions: While the 2010 pilot exercise has confirmed that the majority of the proposed indicators have some validity, there are significant challenges in operationalising and measuring these indicators reliably, as well as in comparing evidence of research impact across different cases in a standardised manner. It is suggested that the public funding agencies, medical research charities, universities, and the wider medical research community work together to develop more robust methodologies for capturing and describing impact, including more valid and reliable impact indicators.Keywords
This publication has 87 references indexed in Scilit:
- Knowledge sources, patent protection, and commercialization of pharmaceutical innovationsResearch Policy, 2010
- Measuring knowledge spillovers: What patents, licenses and publications reveal about innovation diffusionResearch Policy, 2009
- From apprentice to colleague: The metamorphosis of Early Career ResearchersHigher Education, 2007
- If star scientists do not patent: The effect of productivity, basicness and impact on the decision to patent in the academic worldResearch Policy, 2007
- Collaboration uncovered: Exploring the adequacy of measuring university-industry collaboration through co-authorship and fundingScientometrics, 2006
- Traces of Prior Art: An analysis of non-patent references found in patent documentsScientometrics, 2006
- Resources, capabilities, risk capital and the creation of university spin-out companiesResearch Policy, 2005
- Industry funding and university professors’ research performanceResearch Policy, 2005
- Academic careers, patents, and productivity: industry experience as scientific and technical human capitalResearch Policy, 2005
- The historical development of clinical therapeutic trialsJournal of Chronic Diseases, 1959