Automated Item Generation with Recurrent Neural Networks
- 12 March 2018
- journal article
- research article
- Published by Cambridge University Press (CUP) in Psychometrika
- Vol. 83 (4) , 847-857
Abstract
Utilizing technology for automated item generation is not a new idea. However, test items used in commercial testing programs or in research are still predominantly written by humans, in most cases by content experts or professional item writers. Human experts are a limited resource and testing agencies incur high costs in the process of continuous renewal of item banks to sustain testing programs. Using algorithms instead holds the promise of providing unlimited resources for this crucial part of assessment development. The approach presented here deviates in several ways from previous attempts to solve this problem. In the past, automatic item generation relied either on generating clones of narrowly defined item types such as those found in language free intelligence tests (e.g., Raven’s progressive matrices) or on an extensive analysis of task components and derivation of schemata to produce items with pre-specified variability that are hoped to have predictable levels of difficulty. It is somewhat unlikely that researchers utilizing these previous approaches would look at the proposed approach with favor; however, recent applications of machine learning show success in solving tasks that seemed impossible for machines not too long ago. The proposed approach uses deep learning to implement probabilistic language models, not unlike what Google brain and Amazon Alexa use for language processing and generation.Keywords
This publication has 17 references indexed in Scilit:
- Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and GermanJournal of Research in Personality, 2007
- The international personality item pool and the future of public-domain personality measuresJournal of Research in Personality, 2006
- Long Short-Term MemoryNeural Computation, 1997
- Prediction Functions for Categorical Panel DataThe Annals of Statistics, 1995
- Conditional Log-Linear Models for Analyzing Categorical Panel DataJournal of the American Statistical Association, 1994
- Approximation capabilities of multilayer feedforward networksNeural Networks, 1991
- Artificial neural networks, back propagation, and the Kelley-Bryson gradient procedureJournal of Guidance, Control, and Dynamics, 1990
- Approximation by superpositions of a sigmoidal functionMathematics of Control, Signals, and Systems, 1989
- Elicitation of Personal Probabilities and ExpectationsJournal of the American Statistical Association, 1971
- The perceptron: A probabilistic model for information storage and organization in the brain.Psychological Review, 1958