Benchmarking AI: Toward Inclusive Evaluation of Language Models
- 30 July 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Computer
- Vol. 58 (8) , 153-159
Abstract
Simply measuring a language model’s performance after deployment is not enough. Rigorous and inclusive benchmarking isn’t just a checkbox along the way—it’s the foundation upon which trustworthy, effective, and human-centered artificial intelligence must be built.Keywords
This publication has 13 references indexed in Scilit:
- On opportunities and challenges of large multimodal foundation models in educationnpj Science of Learning, 2025
- The AI Alignment ParadoxCommunications of the ACM, 2025
- The EU AI Act and the Wager on Trustworthy AICommunications of the ACM, 2024
- A large-scale audit of dataset licensing and attribution in AINature Machine Intelligence, 2024
- AI deception: A survey of examples, risks, and potential solutionsPatterns, 2024
- Disinfecting AI: Mitigating Generative AI’s Top RisksComputer, 2024
- A Survey on Evaluation of Large Language ModelsACM Transactions on Intelligent Systems and Technology, 2024
- Directing AI: Charting a Roadmap of AI Opportunities and RisksComputer, 2024
- interviewee, n.2023
- Reliance on metrics is a fundamental challenge for AIPatterns, 2022