Leveraging LLM and Multiscale Knowledge States to Improve Knowledge Tracing in Programming Tasks
Open Access
- 12 April 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 7378-7389
Abstract
Knowledge Tracing (KT) is a core task in intelligent tutoring systems, designed to model the dynamic evolution of students' knowledge states by predicting their performance on specific problems. However, existing KT models, primarily developed for traditional subject domains, exhibit poor generalization to programming tasks due to several distinct challenges. First, student responses in programming tasks are characterized by high uncertainty. Second, programming tasks typically provide continuous scores based on the proportion of passed test cases, in contrast to the binary correctness assumed by traditional KT models, with these test-case-based scores often being noisy. Third, most KT models rely exclusively on the understanding of knowledge concepts for performance prediction, whereas success in programming tasks is heavily contingent upon logical reasoning and problem-specific comprehension. To address these issues, we propose an LLM-driven Interaction Enrichment Framework (MIE) to mitigate high uncertainty and problematic labeling, and introduce the Multi-Level Programming Knowledge Tracing (MLPKT) model to capture students' knowledge states across multiple dimensions. MLPKT conducts multi-layer analysis of student submissions to identify the root causes of errors and assign semantically meaningful fine-grained labels. Additionally, we propose a three-level, three-phase KT architecture that captures knowledge dynamics across three dimensions—problems, concepts, and logical skills—through the phases of learning, forgetting/reinforcement, and application. Extensive experiments on three datasets demonstrate that MIE+MLPKT consistently outperforms 18 baseline methods. Our code is available at: https://anonymous.4open.science/r/MIE-MLPKT-D654.Keywords
Funding Information
- National Natural Science Foundation of China (62272093)
- National Natural Science Foundation of China (62137001)
This publication has 22 references indexed in Scilit:
- csKT: Addressing cold-start problem in knowledge tracing via kernel bias and cone attentionExpert Systems with Applications, 2024
- Programming knowledge tracing based on heterogeneous graph representationKnowledge-Based Systems, 2024
- code2vec: learning distributed representations of codeProceedings of the ACM on Programming Languages, 2019
- Addressing two problems in deep knowledge tracing via prediction-consistent regularization2018
- Deep Knowledge Tracing for Free-Form Student Code Progression2018
- A general path-based representation for predicting program propertiesACM SIGPLAN Notices, 2018
- Deep Knowledge Tracing On Programming Exercises2017
- Individualized Bayesian Knowledge Tracing Models2013
- KT-IDEM: Introducing Item Difficulty to the Knowledge Tracing Model2011
- More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing2008