An AI4Research Scoring Benchmark: Characterizing and Calibrating LLM Evaluation of Automated Research Protocols
Authors:
- Yang YichenYYYang Yichen
- Lin DianqingLDLin Dianqing
- Chaolu MengCMChaolu Meng
Preprint
- 6 September 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
Automated research (AI4Research) increasingly delegates both generation and evaluation of scientific content to large language models (LLMs). Evaluation of AI-generated experimental protocols still relies on LLM-as-judge scoring, whose systematic inflation threatens the reliability of research automation. To characterize this problem, we construct a benchmark that stress-tests three families of mainstream LLM usage: fifteen agent reasoning frameworks, five reinforcement-learning strategies (PPO, GRPO, GSPO, DAPO, and ODP), and retrieval-augmented generation with dense, sparse, and hybrid retrieval. We contribute a methodology-centric dataset of 4,946 peer-reviewed machine-learning papers annotated through a six-stage progressive scheme with Long Chain-of-Thought formatting, and fine-tune 14 representative LLMs as generators. We further test the benchmark on biomedical research problems, a knowledge-intensive non-ML domain, and find that score inflation is amplified precisely where factual grounding is weakest. Across all settings we characterize five manifestations of score inflation, including self-evaluation bias and score-quality decoupling. Grounded in these findings, we propose a staged protocol-generation framework combining persistent memory files, stepwise generation, and dual-judge plus human-expert cross-validation. This framework reduces the mean deviation between LLM-judge and human-expert scores to 10.3 points, less than half the 21 to 26 point deviation of every other approach, yielding protocols that approach expert judgment.Keywords
This publication has 19 references indexed in Scilit:
- Toward expert-level medical question answering with large language modelsNature Medicine, 2025
- Large language models in medical and healthcare fields: applications, advances, and challengesArtificial Intelligence Review, 2024
- Fast, sensitive detection of protein homologs using deep dense retrievalNature Biotechnology, 2024
- Solving olympiad geometry without human demonstrationsNature, 2024
- Mathematical discoveries from program search with large language modelsNature, 2023
- Scientific discovery in the age of artificial intelligenceNature, 2023
- Reproducible, scalable, and shareable analysis pipelines with bioinformatics workflow managersNature Methods, 2021
- Highly accurate protein structure prediction with AlphaFoldNature, 2021
- Ensemble deep learning in bioinformaticsNature Machine Intelligence, 2020
- Functional genomic hypothesis generation and experimentation by a robot scientistNature, 2004