Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Elizabeth Gadd has a total of 55 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Can ChatGPT evaluate research environments? Evidence from REF2021

    by Kayvan Kousha, Mike Thelwall, Elizabeth Gadd


  • AI in Research Assessment, A Global Symposium

    by Mike Thelwall, Maura Hiney, Diego Lozano, Julia Melkers, Elizabeth Gadd

    The Research Evaluation Group (INORMS) and the REACH Network are pleased to invite you to a Global Symposium on “AI in Research Assessment”, taking place 11 February 2026. This event brings together leading voices in research policy and evaluation, to explore one of the most urgent questions facing the scholarly community: **Can AI assess research?** Artificial intelligence promises to reshape how research is evaluated. Emerging AI tools can generate detailed, human-like assessment - opening exciting opportunities to reduce administrative burden, support evaluation at scale, and uncover new insights. At the same time, these technologies raise concerns around accuracy, transparency, equity, and the protection of research integrity. The symposium arrives at a moment when the global research community is already rethinking evaluation practices. As assessment evolves beyond narrow bibliometric indicators, researchers and institutions increasingly recognize the importance of mentorship, open science, societal and economic contributions, and collaborative work. How might AI support these responsible approaches - and under what safeguards? Our aim is to engage critically with both the promise and the pitfalls of AI-assisted evaluation, asking: • When can AI meaningfully contribute to research assessment? • What standards, governance, and oversight are required? • Which tasks should remain firmly in the hands of peer-review? **AI in Research Assessment: Friend or Foe for Research Integrity?** Dr. Maura Hiney The interplay between research assessment and research integrity is complex. Robust research integrity practices ensure trust and quality in the direct products of research (outputs) and the subsequent use and effects of these outputs (outcomes and impacts). In such an environment, honest and fair assessment is supported. However, when research integrity fails, assessors may be basing their judgements on biased, poor-quality, or untrustworthy outputs and outcomes. In addition, assessing individuals whose research practices are poor or who are willing to cheat to advance their careers may skew the assessment system in their favour and disadvantage those striving for quality in their research. Adding AI tools to an already complex relationship raises the question of whether using such tools in research assessment will help or hinder research integrity practices. **Can Large Language Models Be Used Responsibly in Research Assessment?** Dr. Mike Thelwall Large Language Models (LLMs) are already being used in research assessment by at least one funder, as well as by some conferences and journals. In all cases they support but do not replace human judgment. Recent evidence also suggests that LLMs are moderately good at assessing the quality of published academic research, even based only on titles and abstracts. Given the huge burden of various forms of peer and expert review in academia, it is tempting to suggest that shortcuts and timesaving must be possible. This is already happening because LLMs are secretly used by many reviewers to replace their opinion, even if the journal, conference, or funder tells them not to. This practice has probably increased the risk that flawed research passes peer review and good research is blocked. More generally, all uses of LLMs in research assessment must be considered carefully from various perspectives, including biases, perverse incentives, and the practical outcomes of greater reliance on AI. For example, if AI becomes more important, will authors prioritize studies that they expect ChatGPT to like, such as those singing the praises of AI or with exaggerated value claims? **Bridging Research and AI for Impact Narratives** Dr. Diego Emilio Lozano Qualitative peer review for researchers is inherently time-consuming, demanding substantial effort for reviewer training and process standardization to ensure robust and unbiased evaluations. To address this, the University of Monterrey (UDEM) conducted a control exercise: 80 researchers were evaluated by five independent peer reviewers to mitigate human bias and establish a reliable average score. Subsequently, we tested an AI tool as a potential replacement for multiple human evaluators. While the AI excelled at generating critical textual analysis, it was ultimately found to be incapable of delivering a reliable, direct quantitative assessment. Our final, successful approach redefined the AI’s role: the tool now rapidly analyzes complex research documents and generates structured, focused validation questions for the human evaluator. This transformation shifts the evaluator's task from deep review to efficient verification. This AI-augmented workflow significantly accelerates the joint validation of impact narratives. The conclusive model positions the AI not as a substitute, but as a critical support mechanism, projecting a sustainable reduction in required human evaluators from five to just two. **Panel Session Includes** Dr. Elizabeth (Lizzie) Gadd, Loughborough University Dr. Julia Melkers, Arizona State University

  • Open Access

    Can ChatGPT evaluate research environments? Evidence from REF2021

    by Kayvan Kousha, Mike Thelwall, Elizabeth Gadd

    UK academic departments are evaluated partly on the statements that they write about the value of their research environments for the Research Excellence Framework (REF) periodic assessments. These statements mix qualitative narratives and quantitative data, typically requiring time-consuming and difficult expert judgements to assess. This article investigates whether Large Language Models (LLMs) can support the process or validate the results, using the UK REF2021 unit-level environment statements as a test case. Based on prompts mimicking the REF guidelines, ChatGPT 4o-mini scores correlated positively with expert scores in almost all 34 (field-based) Units of Assessment (UoAs). ChatGPT's scores had moderate to strong positive Spearman correlations with REF expert scores in 32 out of 34 UoAs: 14 UoAs above 0.7 and a further 13 between 0.6 and 0.7. Only two UoAs had weak or no significant associations (Classics and Clinical Medicine). From further tests for UoA34, multiple LLMs had significant positive correlations with REF2021 environment scores (all p < .001), with ChatGPT 5 performing best (r=0.81; $ρ$=0.82), followed by ChatGPT-4o-mini (r=0.68; $ρ$=0.67) and Gemini Flash 2.5 (r=0.67; $ρ$=0.69). If LLM-generated scores for environment statements are used in future to help reduce workload, support more consistent interpretation, and complement human review then caution must be exercised because of the potential for biases, inaccuracy in some cases, and unwanted systemic effects. Even the strong correlations found here seem unlikely to be judged close enough to expert scores to fully delegate the assessment task to LLMs.

  • Open Access

    Criteria for Bibliographic Databases in a Well-Functioning Scholarly Communication and Research Assessment Ecosystem

    by Ginny Barbour, Caitlin Carter, Jonny Coates - Asapbio, Kelly D. Cobey, Katherine S. Corker, Elizabeth Gadd, Bianca Kramer, Rebecca Lawrence, Eva Méndez, Cameron Neylon, et al.

    There is much to celebrate when it comes to recent developments in scholarly communication and research assessment. The scholarly communication landscape shows many promising initiatives, including a growing use of preprint services, open access repositories, and open peer review platforms, as well as an increasing interest in more equitable scholarly publishing models such as diamond open access and Subscribe to Open.

  • Open Access

    Criteria for Bibliographic Databases in a Well-Functioning Scholarly Communication and Research Assessment Ecosystem

    by Ginny Barbour, Caitlin Carter, Jonny Coates - Asapbio, Kelly D. Cobey, Katherine S. Corker, Elizabeth Gadd, Bianca Kramer, Rebecca Lawrence, Eva Méndez, Cameron Neylon, et al.

    There is much to celebrate when it comes to recent developments in scholarly communication and research assessment. The scholarly communication landscape shows many promising initiatives, including a growing use of preprint services, open access repositories, and open peer review platforms, as well as an increasing interest in more equitable scholarly publishing models such as diamond open access and Subscribe to Open.


  • Open Access

    The SCOPE framework – implementing ideals of responsible research assessment

    by Laura Himanen, Erica Conte, Marianne Gauffriau, Tanja Strøm, Baron Wolf, Elizabeth Gadd

    Background Research and researchers are heavily evaluated, and over the past decade it has become widely acknowledged that the consequences of evaluating the research enterprise and particularly individual researchers are considerable. This has resulted in the publishing of several guidelines and principles to support moving towards more responsible research assessment (RRA). To ensure that research evaluation is meaningful, responsible, and effective the International Network of Research Management Societies (INORMS) Research Evaluation Group created the SCOPE framework enabling evaluators to deliver on existing principles of RRA. SCOPE bridges the gap between principles and their implementation by providing a structured five-stage framework by which evaluations can be designed and implemented, as well as evaluated. Methods SCOPE is a step-by-step process designed to help plan, design, and conduct research evaluations as well as check effectiveness of existing evaluations. In this article, four case studies are presented to show how SCOPE has been used in practice to provide value-based research evaluation. Results This article situates SCOPE within the international work towards more meaningful and robust research evaluation practices and shows through the four case studies how it can be used by different organisations to develop evaluations at different levels of granularity and in different settings. Conclusions The article demonstrates that the SCOPE framework is rooted firmly in the existing literature. In addition, it is argued that it does not simply translate existing principles of RRA into practice, but provides additional considerations not always addressed in existing RRA principles and practices thus playing a specific role in the delivery of RRA. Furthermore, the use cases show the value of SCOPE across a range of settings, including different institutional types, sizes, and missions.


  • Open Access

    The SCOPE framework – implementing the ideals of responsible research assessment

    by Laura Himanen, Erica Conte, Marianne Gauffriau, Tanja Strøm, Baron Wolf, Elizabeth Gadd

    Background:Research and researchers are heavily evaluated, and over the past decade it has become apparent that the consequences of evaluating the research enterprise and particularly individual researchers are considerable. This has resulted in the publishing of several guidelines and principles to support moving towards more responsible research assessment (RRA). To ensure that research evaluation is meaningful, responsible, and effective the International Network of Research Management Societies (INORMS) Research Evaluation Group created the SCOPE framework enabling evaluators to deliver on existing principles of RRA. SCOPE bridges the gap between principles and their implementation by providing a structured five-stage framework by which evaluations can be designed and implemented, as well as evaluated. Methods:SCOPE is a step-by-step process designed to help plan, design, and conduct research evaluations as well as check effectiveness of existing evaluations. In this article, four case studies are presented to show how SCOPE has been used in practice to provide value-based research evaluation. Results:This article situates SCOPE within the international work towards more meaningful and robust research evaluation practices and shows through the four case studies how it can be used by different organisations to develop evaluations at different levels of granularity and in different settings. Conclusions:The article demonstrates that the SCOPE framework is rooted firmly in the existing literature. In addition, it is argued that it does not simply translate existing principles of RRA into practice, but provides additional considerations not always addressed in existing RRA principles and practices thus playing a specific role in the delivery of RRA. Furthermore, the use cases show the value of SCOPE across a range of settings, including different institutional types, sizes, and missions.


  • Open Access

    Mis-Measuring Our Universities: Why Global University Rankings Don’t Add Up

    by Elizabeth Gadd

    Draws parallels between the problematic use of GDP to evaluate economic success with the use of global university rankings to evaluate university success. Inspired by Kate Raworth’sDoughnut Economics,this perspective argues that the pursuit of growth as measured by such indicators creates universities that ‘grow’ up the rankings rather than those which ‘thrive’ or ‘mature.’ Such growth creates academic wealth divides within and between countries, despite the direction of growth as inspired by the rankings not truly reflecting universities’ critical purpose or contribution. Highlights the incompatibility between universities’ alignment with socially responsible practices and continued engagement with socially irresponsible ranking practices. Proposes four possible ways of engendering change in the university rankings space. Concludes by calling on leaders of ‘world-leading’ universities to join together to ‘lead the world’ in challenging global university rankings, and to set their own standards for thriving and maturing universities.


  • Open Access

    Developing a Method for Evaluating Global University Rankings

    by Elizabeth Gadd, Richard Holmes, Justin Shearer