Junyuan Tan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Junyuan Tan has a total of 3 co-authors. Here are the top co-authors by number of shared publications:
- Kewen Zhu
Stevens Institute of Technology, USA
1 shared publication - Ze Sheng
Texas A&M University, USA
1 shared publication - Zixi Liu
University of California, Los Angeles, USA
1 shared publication
Latest Publications
- Open Access
- 21 July 2026
- Published by MDPI AG in Preprints.org
Scale Collapse, Model Disagreement, and False Precision: A Multi-Model Audit of LLM Behavioral Interview Scoring
Large Language Models (LLMs) are increasingly deployed as hiring evaluators, yet no systematic reliability audit exists for this high-stakes domain. We benchmark 9 frontier models from 3 families (GPT-5.4, Claude, Gemini) across 10 behavioral questions, 3 companies, and 2 seniority levels (141 cells, 10–50 samples each, 4,600 total ratings), revealing five reliability failures: (1) Scale collapse — a 5-point scale degenerates into 2.03 effective categories (normalized entropy = 0.44); (2) Test-retest unreliability — ICC(2,1) ranges from 0.16 to 0.85 across models; (3) Inter-model disagreement — Krippendorff's α = 0.52, below the 0.67 tentative-conclusion threshold, with same-brand variants (Spearman ρ = 0.25) disagreeing more than cross-vendor pairs (ρ = 0.87); (4) Question-type sensitivity — identical answers receive ratings differing by 2 ordinal grades across question types; (5) Expectation gap — community-endorsed answers score below "Hire" across all models, with flagship models significantly harsher than lightweight (Wilcoxon z = −2.80, p = .005). We propose a minimum reliability checklist and discuss implications for AI hiring policy.
- 21 July 2026
- Published by MDPI AG in Preprints.org
Open AccessScale Collapse, Model Disagreement, and False Precision: A Multi-Model Audit of LLM Behavioral Interview Scoring
Large Language Models (LLMs) are increasingly deployed as hiring evaluators, yet no systematic reliability audit exists for this high-stakes domain. We benchmark 9 frontier models from 3 families (GPT-5.4, Claude, Gemini) across 10 behavioral questions, 3 companies, and 2 seniority levels (141 cells, 10–50 samples each, 4,600 total ratings), revealing five reliability failures: (1) Scale collapse — a 5-point scale degenerates into 2.03 effective categories (normalized entropy = 0.44); (2) Test-retest unreliability — ICC(2,1) ranges from 0.16 to 0.85 across models; (3) Inter-model disagreement — Krippendorff's α = 0.52, below the 0.67 tentative-conclusion threshold, with same-brand variants (Spearman ρ = 0.25) disagreeing more than cross-vendor pairs (ρ = 0.87); (4) Question-type sensitivity — identical answers receive ratings differing by 2 ordinal grades across question types; (5) Expectation gap — community-endorsed answers score below "Hire" across all models, with flagship models significantly harsher than lightweight (Wilcoxon z = −2.80, p = .005). We propose a minimum reliability checklist and discuss implications for AI hiring policy.