Initial comparative analysis of model and peer review process for ERC starting grant proposals
- 8 August 2013
- journal article
- Published by Oxford University Press (OUP) in Research Evaluation
- Vol. 22 (4) , 248-257
Abstract
This article discusses a statistical modelling approach for inferring attributes of ‘frontier research’ in peer-reviewed research proposals submitted to the European Research Council (ERC). The first step briefly revisit the underlying concept, approach, and the defined indicators to proxy aspects of frontier research, which are based on proposal texts as well as scientometric and bibliometric data of ERC grant applications. Based on the set of indicators, the second step models the decision probability of a proposal to be accepted. It compares outcomes between the model and peer review decision, with the goal to determine the influence of different frontier research aspects in the peer review process. In an initial attempt, we demonstrate and discuss in a proof-of-concept approach a data sample of about 10% of all proposals submitted to the ERC for starting grants in the year 2009. It shows the feasibility and usefulness of the modelling approach. Ultimately, the concept is aiming at testing new methods for monitoring the effectiveness of peer review processes, by taking a scientometric perspective of research proposals that reach beyond publication and citation statistics.Keywords
This publication has 15 references indexed in Scilit:
- Research funding: Making the cutNature, 2010
- Scientometric indicators: peer-review, bibliometric methods and conflict of interestsScientometrics, 2010
- The European Research Council Takes FlightCell, 2009
- Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability.American Psychologist, 2008
- Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A ReviewScientometrics, 2006
- Impartial Judgment by the “Gatekeepers” of Science: Fallibility and Accountability in the Peer Review ProcessAdvances in Health Sciences Education, 2003
- The counting houseNature, 2002
- Analysis of distribution of the age of citations in theoretical population geneticsScientometrics, 1997
- A bibliometric study on ageing and reception processes of scientific literatureJournal of Information Science, 1995
- Grants Peer Review in Theory and PracticeEvaluation Review, 1994