Joint Task Difficulties Estimation and Testees Ranking for Intelligence Evaluation
- 11 March 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 6 (2) , 221-226
Abstract
In this paper, we study the testing tasks evaluation and testees ranking problem, in which tasks have different difficulty levels, and testees have different capabilities. We assume that a testee may have a probability to pass a certain task so as to allow certain uncertainty. The goal of this problem is to simultaneously determine the relative difficulty level of each testing task and the relative capability of every testee, purely based on the test outcome. We design two models to solve this problem. The first one assumes that the test outcome follows a certain Bernoulli distribution; while the second one assumes that the test outcome follows a certain Bernoulli distribution with the beta distribution-type a priori knowledge. Then, we form the original problem into likelihood estimation problems and solve them by using coordinate descent algorithms. We show that the beta distribution-type a priori knowledge is needed, when we only carry out a limited number of tests due to time and financial budgets. All these findings are useful to intelligence tests. Finally, we discuss how to extend this statistical learning model for more general cases as well as in a specific case in the field of Computational Social Systems like artificial social cognition evaluation.Keywords
Funding Information
- National Natural Science Foundation of China (61790565)
- Beijing Municipal Science and Technology Commission, Adminitrative Commission of Zhongguancun Science Park (D171100000317002)
- Beijing Municipal Commission of Transport Program (ZC179074Z)
This publication has 17 references indexed in Scilit:
- Mastering the game of Go without human knowledgeNature, 2017
- Semantic Social Network Analysis by Cross-Domain Tensor FactorizationIEEE Transactions on Computational Social Systems, 2017
- Measuring an artificial intelligence system’s performance on a Verbal IQ test for young childrenJournal of Experimental & Theoretical Artificial Intelligence, 2016
- Social Network Analysis With Data FusionIEEE Transactions on Computational Social Systems, 2016
- Intelligence Testing for Autonomous Vehicles: A New ApproachIEEE Transactions on Intelligent Vehicles, 2016
- Social Discovery: Exploring the Correlation Among Three-Dimensional Social RelationshipsIEEE Transactions on Computational Social Systems, 2015
- An Analytical Comparison of Social Network MeasuresIEEE Transactions on Computational Social Systems, 2014
- Task-Specific Performance Evaluation of UGVs: Case Studies at the IVFCIEEE Transactions on Intelligent Transportation Systems, 2014
- Probabilistic Principal Component AnalysisJournal of the Royal Statistical Society Series B: Statistical Methodology, 1999
- Modelling social action for AI agentsArtificial Intelligence, 1998