Abstract
Background: Context personalization, the practice of adapting learning materials to students’ personal interests, has been shown to increase student learning and engagement. Within computer science education, research has found that LLMs can generate high-quality contextualized introductory programming exercises. Objective: In this paper, we evaluate the capability of LLMs to generate technically correct and thematically integrated contextualized algorithm design problems. Methods: In a series of three iterative studies, we use LLMs to generate contextualized algorithm design problems from a given base problem and theme, evaluating over 500 generated problems for technical and thematic alignment. Results: We find that LLM-generated algorithm design problems exhibit significantly more issues than prior work has found for introductory programming problems. We identify issues specific to the algorithm design context and then mitigate these issues with prompt engineering techniques and model choice. With these adjustments, we produce LLM-generated contextualized algorithm design problems that are technically strong, deeply themed, and largely realistic, though realism drops with more culturally and locally specific themes. Implications: We demonstrate a viable workflow for generating contextualized algorithm design problems using LLMs, including prompt design, model selection, and identification of specific issues to review for.
Funding Information
  • National Science Foundation (CCF-2313998)
  • National Science Foundation (CCF-2434362)

This publication has 58 references indexed in Scilit: