LLM-as-a-judge is bad, based on AI attempting the exam qualifying for the member of the Polish National Board of Appeal
- 9 March 2026
- journal article
- research article
- Published by Springer Nature in Artificial Intelligence and Law
- p. 1-51
Abstract
No abstract availableKeywords
All Related Versions
This publication has 21 references indexed in Scilit:
- A Reasoning-Focused Legal Retrieval Benchmark2025
- Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks2025
- LeMAJ (Legal LLM-as-a-Judge): Bridging Legal Reasoning and LLM Evaluation2025
- The digital transformation of jurisprudence: an evaluation of ChatGPT-4’s applicability to solve cases in business lawArtificial Intelligence and Law, 2024
- Re-evaluating GPT-4’s bar exam performanceArtificial Intelligence and Law, 2024
- The effect of source disclosure on evaluation of AI-generated messagesComputers in Human Behavior: Artificial Humans, 2024
- LawBench: Benchmarking Legal Knowledge of Large Language Models2024
- Humans or LLMs as the Judge? A Study on Judgement Bias2024
- Does ChatGPT Pass the Brazilian Bar Exam?2023
- G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment2023