Jiaxin Gao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jiaxin Gao has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Chen Chen
Nanyang Technological University
1 shared publication - Kwok-Yan Lam
Nanyang technological university, Singapore, Singapore
1 shared publication - Qian Wang
Wuhan University, Wuhan, China
1 shared publication - Xueluan Gong
Nanyang Technological University, Singapore
1 shared publication - Yanwen Jia
School of Cyber Science and Engineering, Wuhan University, Wuhan, China
1 shared publication
Latest Publications
- 27 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Network
- Vol. PP (08908044) , 1-8
Evaluating and Mitigating LLM-as-a-Judge Bias in Communication Systems
Large Language Models (LLMs) are increasingly being used to autonomously evaluate the quality of content in communication systems, e.g., to assess responses in telecom customer support chatbots. However, the impartiality of these AI “judges” is not guaranteed, and any biases in their evaluation criteria could skew outcomes and undermine user trust. In this paper, we systematically investigate judgment biases across 6 LLM-as-a-judge models spanning both prompt-based and fine-tuned judges under the pointwise scoring setting, encompassing 11 types of biases that cover both implicit and explicit forms. We observed that state-of-the-art LLM judges demonstrate robustness to biased inputs, generally assigning them lower scores than the corresponding clean samples. We further found that fine-tuning an LLM on high-scoring yet biased responses can significantly degrade its performance, highlighting the risk of training on biased data. We also discovered that the judged scores correlate with task difficulty: a challenging dataset like GPQA yields lower average scores, whereas an open-ended reasoning dataset (e.g., JudgeLM-val) sees higher average scores. Finally, we proposed four potential mitigation strategies to ensure fair and reliable AI judging in practical communication scenarios.
- 27 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Network
- Vol. PP (08908044) , 1-8
Evaluating and Mitigating LLM-as-a-Judge Bias in Communication Systems
Large Language Models (LLMs) are increasingly being used to autonomously evaluate the quality of content in communication systems, e.g., to assess responses in telecom customer support chatbots. However, the impartiality of these AI “judges” is not guaranteed, and any biases in their evaluation criteria could skew outcomes and undermine user trust. In this paper, we systematically investigate judgment biases across 6 LLM-as-a-judge models spanning both prompt-based and fine-tuned judges under the pointwise scoring setting, encompassing 11 types of biases that cover both implicit and explicit forms. We observed that state-of-the-art LLM judges demonstrate robustness to biased inputs, generally assigning them lower scores than the corresponding clean samples. We further found that fine-tuning an LLM on high-scoring yet biased responses can significantly degrade its performance, highlighting the risk of training on biased data. We also discovered that the judged scores correlate with task difficulty: a challenging dataset like GPQA yields lower average scores, whereas an open-ended reasoning dataset (e.g., JudgeLM-val) sees higher average scores. Finally, we proposed four potential mitigation strategies to ensure fair and reliable AI judging in practical communication scenarios.