Difficulty-Controllable Cloze Question Distractor Generation
Preprint
Abstract
Multiple-choice cloze questions are commonly used to assess linguistic proficiency and comprehension. However, generating high-quality distractors remains challenging, as existing methods often lack adaptability and control over difficulty levels, and the absence of difficulty-annotated datasets further hinders progress. To address these issues, we propose a novel framework for generating distractors with controllable difficulty by leveraging both data augmentation and a multitask learning strategy. First, to create a high-quality, difficulty-annotated dataset, we introduce a two-way distractor generation process in order to produce diverse and plausible distractors. These candidates are subsequently refined through filtering and then categorized by difficulty using an ensemble QA system. Second, this newly created dataset is leveraged to train a difficulty-controllable generation model via multitask learning. The framework includes carefully designed auxiliary tasks that enhance the model's semantic understanding of distractors and its ability to estimate their difficulty. Experimental results demonstrate that our method generates high-quality distractors across difficulty levels and substantially outperforms GPT-4o in aligning distractor difficulty with human perception.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Item analysis: the impact of distractor efficiency on the difficulty index and discrimination power of multiple-choice itemsBMC Medical Education, 2024
- Text-based Question Difficulty Prediction: A Systematic Review of Automatic ApproachesInternational Journal of Artificial Intelligence in Education, 2023
- Difficulty-Controllable Neural Question Generation for Reading Comprehension using Item Response Theory2023
- CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model2022
- Eye Gaze and Self-attention: How Humans and Transformers Attend Words in Sentences2022
- Knowledge-Driven Distractor Generation for Cloze-Style Multiple Choice QuestionsProceedings of the AAAI Conference on Artificial Intelligence, 2021
- An exploration of eLearning adoption in the educational ecosystemEducation and Information Technologies, 2020
- Controlling item difficulty for automatic vocabulary question generationResearch and Practice in Technology Enhanced Learning, 2017
- Probase2012
- “Cloze Procedure”: A New Tool for Measuring ReadabilityJournalism Quarterly, 1953