My Computer Is an Honor Student — But How Intelligent Is It? Standardized Tests as a Measure of AI
Authors:
- 1 March 2016
- journal article
- Published by Wiley in AI Magazine
- Vol. 37 (1) , 5-12
Abstract
Given the well‐known limitations of the Turing test, there is a need for objective tests to both focus attention on, and measure progress toward, the goals of AI. In this paper we argue that machine performance on standardized tests should be a key component of any new measure of AI, because attaining a high level of performance requires solving significant AI problems involving language understanding and world modeling — critical skills for any machine that lays claim to intelligence. In addition, standardized tests have all the basic requirements of a practical test: they are accessible, easily comprehensible, clearly measurable, and offer a graduated progression from simple tasks to those requiring deep understanding of the world. Here we propose this task as a challenge problem for the community, summarize our state‐of‐the‐art results on math and science tests, and provide supporting data sets (www.allenai.org).Keywords
This publication has 17 references indexed in Scilit:
- Commonsense reasoning and commonsense knowledge in artificial intelligenceCommunications of the ACM, 2015
- Learning to Solve Arithmetic Word Problems with Verb Categorization2014
- Learning to Automatically Solve Algebra Word Problems2014
- A study of the knowledge base requirements for passing an elementary science test2013
- Computing Lexical ContrastComputational Linguistics, 2013
- Psychometric artificial intelligenceJournal of Experimental & Theoretical Artificial Intelligence, 2011
- Similarity of Semantic RelationsComputational Linguistics, 2006
- A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge.Psychological Review, 1997
- To dream the possible dreamCommunications of the ACM, 1996
- I.—COMPUTING MACHINERY AND INTELLIGENCEMind, 1950