Xiuxiu Tang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Xiuxiu Tang has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Ben Van Dusen
School of Education Iowa State University Ames Iowa USA
1 shared publication - Hua-Hua Chang
Department of Educational Studies, Purdue University, West Lafayette, 47907, Indiana, USA
1 shared publication Hua‐Hua Chang
Purdue University
1 shared publication- Jason Morphew
Purdue University
1 shared publication - Jayson M. Nissen
Montana State University
1 shared publication John T. Behrens
1 shared publication
Latest Publications
- Open Access
- 23 April 2026
- Published by Center for Open Science in PsyArXiv
Can Large Language Models Emulate Human Performance on Educational Assessments?
Large language models (LLMs) are increasingly used to generate synthetic data for research and evaluation, including simulating student responses to educational assessments. However, their validity for emulating authentic human performance remains unclear. This study evaluates whether LLMs can-generate assessment data that preserve the score distribution, subgroup relationships, item functioning, latent-variable model properties, and overall distributional similarity required for valid measurement. Using fourth-grade mathematics data from TIMSS 2011, we compare synthetic responses generated by GPT-4o and GPT-5 under multiple prompting conditions against empirical student responses and theory-driven simulated data from a cognitive diagnostic model (CDM). Across both LLM architectures, LLM-generated data exhibit systematic score inflation and variance compression, with distortions more pronounced for GPT-5. Item easiness is overestimated, item discriminations are attenuated, and alignment with empirical item properties is weak. Adding information on cognitive mastery profiles and personal background modestly increases variability but does not recover human-like psychometric structure. Although the CDM shows adequate fit to some LLM-generated datasets, item parameter recovery remains poor. In contrast, CDM-based simulations closely reproduce human score distributions, item properties, and latent structure. These results suggest that LLM-generated synthetic assessment data deviate systematically from human data, which limits their usefulness for inferences requiring structural fidelity. Caution is therefore warranted when using LLM-generated synthetic data in measurement contexts where variance, item functioning, and latent structure are central.
- 23 April 2026
- Published by Center for Open Science in PsyArXiv
Open AccessCan Large Language Models Emulate Human Performance on Educational Assessments?
Large language models (LLMs) are increasingly used to generate synthetic data for research and evaluation, including simulating student responses to educational assessments. However, their validity for emulating authentic human performance remains unclear. This study evaluates whether LLMs can-generate assessment data that preserve the score distribution, subgroup relationships, item functioning, latent-variable model properties, and overall distributional similarity required for valid measurement. Using fourth-grade mathematics data from TIMSS 2011, we compare synthetic responses generated by GPT-4o and GPT-5 under multiple prompting conditions against empirical student responses and theory-driven simulated data from a cognitive diagnostic model (CDM). Across both LLM architectures, LLM-generated data exhibit systematic score inflation and variance compression, with distortions more pronounced for GPT-5. Item easiness is overestimated, item discriminations are attenuated, and alignment with empirical item properties is weak. Adding information on cognitive mastery profiles and personal background modestly increases variability but does not recover human-like psychometric structure. Although the CDM shows adequate fit to some LLM-generated datasets, item parameter recovery remains poor. In contrast, CDM-based simulations closely reproduce human score distributions, item properties, and latent structure. These results suggest that LLM-generated synthetic assessment data deviate systematically from human data, which limits their usefulness for inferences requiring structural fidelity. Caution is therefore warranted when using LLM-generated synthetic data in measurement contexts where variance, item functioning, and latent structure are central.
- Open Access
- 23 January 2026
- Published by Cambridge University Press (CUP) in Psychometrika
- Vol. 91 (2) , 395-423
A Likelihood-Based Profile Shrinkage Algorithm for Efficient Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT)
Various item selection algorithms have been proposed for cognitive diagnostic computerized adaptive testing (CD-CAT), with the goal of efficiently diagnosing examinees’ strengths and weaknesses. However, these algorithms often come with significant computational costs, which can hinder their practical implementation. A likelihood-based profile shrinkage (LBPS) algorithm is proposed to simplify the item selection process and reduce the computational cost in CD-CAT. Our simulation results indicate that incorporating LBPS into existing item selection methods yields substantial computational efficiency gains, with greater reductions in computation time as the number of attributes and test length increase. Additionally, LBPS maintains estimation accuracy at both the attribute and pattern levels. These findings suggest that LBPS is a scalable and effective solution for the item selection of CD-CAT in complex scenarios.
- 23 January 2026
- Published by Cambridge University Press (CUP) in Psychometrika
- Vol. 91 (2) , 395-423
Open AccessA Likelihood-Based Profile Shrinkage Algorithm for Efficient Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT)
Various item selection algorithms have been proposed for cognitive diagnostic computerized adaptive testing (CD-CAT), with the goal of efficiently diagnosing examinees’ strengths and weaknesses. However, these algorithms often come with significant computational costs, which can hinder their practical implementation. A likelihood-based profile shrinkage (LBPS) algorithm is proposed to simplify the item selection process and reduce the computational cost in CD-CAT. Our simulation results indicate that incorporating LBPS into existing item selection methods yields substantial computational efficiency gains, with greater reductions in computation time as the number of attributes and test length increase. Additionally, LBPS maintains estimation accuracy at both the attribute and pattern levels. These findings suggest that LBPS is a scalable and effective solution for the item selection of CD-CAT in complex scenarios.
- Open Access
- 12 September 2024
- Published by American Association of Physics Teachers (AAPT)
- p. 243-249
Mechanics Cognitive Diagnostic: Mathematics skills tested in introductory physics courses
Physics instructors and education researchers use research-based assessments (RBAs) to evaluate students' preparation for physics courses. This preparation can cover a wide range of constructs, including mathematics and physics content. Using separate mathematics and physics RBAs consumes course time. We are developing a mechanics cognitive diagnostic (MCD) as an online test using both computerized adaptive testing and cognitive diagnostic models. This design allows the MCD to assess mathematics and physics content knowledge within a single assessment. Our work used an evidence-centered design framework to inform the extent to which our models of skills students develop in physics courses fit the data from three mathematics RBAs. Our dataset came from the LASSO platform and includes 3,491 responses from the Calculus Concept Assessment, Calculus Concept Inventory, and Pre-calculus Concept Assessment. Our model included five skills: apply vectors, conceptual relationships, algebra, visualizations, and calculus. The "deterministic inputs, noisy 'and' gate" (DINA) analyses demonstrated a good fit for the five skills. The classification accuracies for the skills were satisfactory. Including items of the three mathematics RBAs in the MCD's item bank will provide a flexible assessment of these skills across mathematics and physics content areas that can adapt to instructors' needs.
- 12 September 2024
- Published by American Association of Physics Teachers (AAPT)
- p. 243-249
Open AccessMechanics Cognitive Diagnostic: Mathematics skills tested in introductory physics courses
Physics instructors and education researchers use research-based assessments (RBAs) to evaluate students' preparation for physics courses. This preparation can cover a wide range of constructs, including mathematics and physics content. Using separate mathematics and physics RBAs consumes course time. We are developing a mechanics cognitive diagnostic (MCD) as an online test using both computerized adaptive testing and cognitive diagnostic models. This design allows the MCD to assess mathematics and physics content knowledge within a single assessment. Our work used an evidence-centered design framework to inform the extent to which our models of skills students develop in physics courses fit the data from three mathematics RBAs. Our dataset came from the LASSO platform and includes 3,491 responses from the Calculus Concept Assessment, Calculus Concept Inventory, and Pre-calculus Concept Assessment. Our model included five skills: apply vectors, conceptual relationships, algebra, visualizations, and calculus. The "deterministic inputs, noisy 'and' gate" (DINA) analyses demonstrated a good fit for the five skills. The classification accuracies for the skills were satisfactory. Including items of the three mathematics RBAs in the MCD's item bank will provide a flexible assessment of these skills across mathematics and physics content areas that can adapt to instructors' needs.
- 6 June 2024
- Published by Wiley in Journal of Educational Measurement
- Vol. 62 (3) , 468-495
Utilizing Response Time for Item Selection in On‐the‐Fly Multistage Adaptive Testing for PISA Assessment
Multistage adaptive testing (MST) has been recently adopted for international large‐scale assessments such as Programme for International Student Assessment (PISA). MST offers improved measurement efficiency over traditional nonadaptive tests and improved practical convenience over single‐item‐adaptive computerized adaptive testing (CAT). As a third alternative adaptive test design to MST and CAT, Zheng and Chang proposed the “on‐the‐fly multistage adaptive testing” (OMST), which combines the benefits of MST and CAT and offsets their limitations. In this study, we adopted the OMST design while also incorporating response time (RT) in item selection. Via simulations emulating the PISA 2018 reading test, including using the real item attributes and replicating PISA 2018 reading test's MST design, we compared the performance of our OMST designs against the simulated MST design in (1) measurement accuracy of test takers’ ability, (2) test time efficiency and consistency, and (3) expected gains in precision by design. We also investigated the performance of OMST in item bank usage and constraints management. Results show great potential for the proposed RT‐incorporated OMST designs to be used for PISA and potentially other international large‐scale assessments.
- 6 June 2024
- Published by Wiley in Journal of Educational Measurement
- Vol. 62 (3) , 468-495
Utilizing Response Time for Item Selection in On‐the‐Fly Multistage Adaptive Testing for PISA Assessment
Multistage adaptive testing (MST) has been recently adopted for international large‐scale assessments such as Programme for International Student Assessment (PISA). MST offers improved measurement efficiency over traditional nonadaptive tests and improved practical convenience over single‐item‐adaptive computerized adaptive testing (CAT). As a third alternative adaptive test design to MST and CAT, Zheng and Chang proposed the “on‐the‐fly multistage adaptive testing” (OMST), which combines the benefits of MST and CAT and offsets their limitations. In this study, we adopted the OMST design while also incorporating response time (RT) in item selection. Via simulations emulating the PISA 2018 reading test, including using the real item attributes and replicating PISA 2018 reading test's MST design, we compared the performance of our OMST designs against the simulated MST design in (1) measurement accuracy of test takers’ ability, (2) test time efficiency and consistency, and (3) expected gains in precision by design. We also investigated the performance of OMST in item bank usage and constraints management. Results show great potential for the proposed RT‐incorporated OMST designs to be used for PISA and potentially other international large‐scale assessments.