Abstract
Automated research (AI4Research) increasingly delegates both generation and evaluation of scientific content to large language models (LLMs). Evaluation of AI-generated experimental protocols still relies on LLM-as-judge scoring, whose systematic inflation threatens the reliability of research automation. To characterize this problem, we construct a benchmark that stress-tests three families of mainstream LLM usage: fifteen agent reasoning frameworks, five reinforcement-learning strategies (PPO, GRPO, GSPO, DAPO, and ODP), and retrieval-augmented generation with dense, sparse, and hybrid retrieval. We contribute a methodology-centric dataset of 4,946 peer-reviewed machine-learning papers annotated through a six-stage progressive scheme with Long Chain-of-Thought formatting, and fine-tune 14 representative LLMs as generators. We further test the benchmark on biomedical research problems, a knowledge-intensive non-ML domain, and find that score inflation is amplified precisely where factual grounding is weakest. Across all settings we characterize five manifestations of score inflation, including self-evaluation bias and score-quality decoupling. Grounded in these findings, we propose a staged protocol-generation framework combining persistent memory files, stepwise generation, and dual-judge plus human-expert cross-validation. This framework reduces the mean deviation between LLM-judge and human-expert scores to 10.3 points, less than half the 21 to 26 point deviation of every other approach, yielding protocols that approach expert judgment.

This publication has 19 references indexed in Scilit: