Yi-Hsin Chen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Yi-Hsin Chen has a total of 29 co-authors. Here are the top co-authors by number of shared publications:
Yan Wang
Department of Psychology, University of Massachusetts, Lowell, MA, United States
2 shared publications- Alexander C. Kah
Department of Psychology, University of South Florida, Tampa, FL, USA
1 shared publication - Allan Feldman
University of South Florida
1 shared publication Chunhua Cao
Department of Educational Studies in Psychology, The University of Alabama, Tuscaloosa, AL, United States
1 shared publicationClaus H. Carstensen
Psychology and Methods of Educational Research, Otto-Friedrich-Universität Bamberg, Germany
1 shared publication- David Rosengrant
Kennesaw State University, Kennesaw, GA
1 shared publication
Latest Publications
- Open Access
- 6 April 2026
- Published by Taylor & Francis in International Journal of Testing
- Vol. 26 (3) , 1-31
Using parametric logistic item response theory model trees to detect item parameter heterogeneity in a high-stakes reading comprehension test
- 6 April 2026
- Published by Taylor & Francis in International Journal of Testing
- Vol. 26 (3) , 1-31
Open AccessUsing parametric logistic item response theory model trees to detect item parameter heterogeneity in a high-stakes reading comprehension test
- Open Access
- 28 February 2025
- Published by Frontiers Media SA in Frontiers in Education
- Vol. 10, 1506674
Accuracy of attribute estimation in the crossed random effects linear logistic test model: impact of Q-matrix misspecification
A simulation study is designed to explore the accuracy of attribute parameter estimation in the crossed random effects linear logistic test model (CRELLTM) with the impact of Q-matrix misspecification on attribute parameter estimation using the SAS® GLIMMIX procedure with a scaling constraint on item parameter. In addition, the impact of the interactions of Q-matrix misspecification with other manipulated factors, such as population distribution, sample size, and Q-matrix density, on parameter estimation is also investigated. The results indicated that misspecification type and percent have a considerable impact on the bias and root mean squared error of attribute estimates, especially under the conditions of high percent misspecification and over-misspecification. However, attribute correlation between the estimated and true parameters is not affected by misspecification type and percent. Other manipulated variables have no impact or interaction effects with Q-matrix misspecifications on attribute estimates. Since the Q-matrix is an indispensable element in applying the crossed random effects linear logistic test model, specifying an appropriate Q-matrix is a crucial task and must be completed with generous assistance from content and subject experts.
- 28 February 2025
- Published by Frontiers Media SA in Frontiers in Education
- Vol. 10, 1506674
Open AccessAccuracy of attribute estimation in the crossed random effects linear logistic test model: impact of Q-matrix misspecification
A simulation study is designed to explore the accuracy of attribute parameter estimation in the crossed random effects linear logistic test model (CRELLTM) with the impact of Q-matrix misspecification on attribute parameter estimation using the SAS® GLIMMIX procedure with a scaling constraint on item parameter. In addition, the impact of the interactions of Q-matrix misspecification with other manipulated factors, such as population distribution, sample size, and Q-matrix density, on parameter estimation is also investigated. The results indicated that misspecification type and percent have a considerable impact on the bias and root mean squared error of attribute estimates, especially under the conditions of high percent misspecification and over-misspecification. However, attribute correlation between the estimated and true parameters is not affected by misspecification type and percent. Other manipulated variables have no impact or interaction effects with Q-matrix misspecifications on attribute estimates. Since the Q-matrix is an indispensable element in applying the crossed random effects linear logistic test model, specifying an appropriate Q-matrix is a crucial task and must be completed with generous assistance from content and subject experts.
- 31 January 2025
- Published by SAGE Publications in Teaching of Psychology
- Vol. 52 (3) , 305-313
Content and Item Response Theory Analysis of ChatGPT-4-Generated Multiple-Choice Items
Background: Multiple-choice item (MCI) assessments are burdensome for instructors to develop. Artificial intelligence (AI, e.g., ChatGPT) can streamline the process without sacrificing quality. The quality of AI-generated MCIs and human experts is comparable. However, whether the quality of AI-generated MCIs is equally good across various domain- and task-specific prompts remains to be determined. Therefore, we ask whether AI can generate high-quality MCIs to assess learning outcomes from a psychology textbook chapter reading. Objective: In an exploratory study, we enlist Item Response Theory analysis and expert reviewers to assess MCIs generated by ChatGPT-4 from a psychology textbook chapter. Method: We submitted a prompt and textbook chapter to ChatGPT-4 requesting 20 MCIs. One hundred ninety undergraduate participants read the chapter before responding to the MCIs. Expert reviewers assessed the MCIs for learning outcome alignment and quality. Results: ChatGPT-4-generated MCIs were low in difficulty and high in discrimination. Expert reviewers found that nearly all items were logically sound, aligned with learning objectives, and met prevailing standards of MCI quality. Conclusion: When carefully prompted, ChatGPT-4 can rapidly generate high-quality MCIs to test comprehension of a psychology textbook chapter. However, due to the uniformly low difficulty of the items, we recommend enlisting ChatGPT-4 to write MCIs for formative, but not summative, assessments.
- 31 January 2025
- Published by SAGE Publications in Teaching of Psychology
- Vol. 52 (3) , 305-313
Content and Item Response Theory Analysis of ChatGPT-4-Generated Multiple-Choice Items
Background: Multiple-choice item (MCI) assessments are burdensome for instructors to develop. Artificial intelligence (AI, e.g., ChatGPT) can streamline the process without sacrificing quality. The quality of AI-generated MCIs and human experts is comparable. However, whether the quality of AI-generated MCIs is equally good across various domain- and task-specific prompts remains to be determined. Therefore, we ask whether AI can generate high-quality MCIs to assess learning outcomes from a psychology textbook chapter reading. Objective: In an exploratory study, we enlist Item Response Theory analysis and expert reviewers to assess MCIs generated by ChatGPT-4 from a psychology textbook chapter. Method: We submitted a prompt and textbook chapter to ChatGPT-4 requesting 20 MCIs. One hundred ninety undergraduate participants read the chapter before responding to the MCIs. Expert reviewers assessed the MCIs for learning outcome alignment and quality. Results: ChatGPT-4-generated MCIs were low in difficulty and high in discrimination. Expert reviewers found that nearly all items were logically sound, aligned with learning objectives, and met prevailing standards of MCI quality. Conclusion: When carefully prompted, ChatGPT-4 can rapidly generate high-quality MCIs to test comprehension of a psychology textbook chapter. However, due to the uniformly low difficulty of the items, we recommend enlisting ChatGPT-4 to write MCIs for formative, but not summative, assessments.
- Open Access
- 29 November 2024
- Published by SAGE Publications in Journal of Psychoeducational Assessment
- Vol. 43 (3) , 310-327
Development and Initial Validation of the Computer-Based Orthographic Processing Assessment Short Form: An Application of Cognitive Diagnostic Modeling
A short form of the 60-item computer-based orthographic processing assessment (long-form COPA or COPA-LF) was developed. The COPA-LF consists of five skills, including rapid perception, access, differentiation, correction, and arrangement. Thirty items from the COPA-LF were selected for the short-form COPA (COPA-SF) based on cognitive diagnostic modeling (CDM) analyses. Both parametric and nonparametric CDM and correlational analyses were conducted with the responses of 74 U.S. second-grade students on the COPA-SF. Results supported five skills of orthographic processing represented on the COPA-SF. Concurrent and convergent validity evidence was found for the COPA-SF. Overall, the findings suggest that the COPA-SF is a time-efficient, reliable, multidimensional measure of orthographic processing.
- 29 November 2024
- Published by SAGE Publications in Journal of Psychoeducational Assessment
- Vol. 43 (3) , 310-327
Open AccessDevelopment and Initial Validation of the Computer-Based Orthographic Processing Assessment Short Form: An Application of Cognitive Diagnostic Modeling
A short form of the 60-item computer-based orthographic processing assessment (long-form COPA or COPA-LF) was developed. The COPA-LF consists of five skills, including rapid perception, access, differentiation, correction, and arrangement. Thirty items from the COPA-LF were selected for the short-form COPA (COPA-SF) based on cognitive diagnostic modeling (CDM) analyses. Both parametric and nonparametric CDM and correlational analyses were conducted with the responses of 74 U.S. second-grade students on the COPA-SF. Results supported five skills of orthographic processing represented on the COPA-SF. Concurrent and convergent validity evidence was found for the COPA-SF. Overall, the findings suggest that the COPA-SF is a time-efficient, reliable, multidimensional measure of orthographic processing.
- 1 October 2024
- Published by Taylor & Francis in Applied Measurement in Education
- Vol. 37 (4) , 356-376
Cross-Cultural Validation of the Mathematics Construct and Attribute Profiles: A Differential Item Functioning Approach
- 1 October 2024
- Published by Taylor & Francis in Applied Measurement in Education
- Vol. 37 (4) , 356-376
Cross-Cultural Validation of the Mathematics Construct and Attribute Profiles: A Differential Item Functioning Approach
- 12 May 2022
- Published by SAGE Publications in Journal of Psychoeducational Assessment
- Vol. 40 (6) , 790-806
Validation of Mathematics Construct and Diagnostic Profiles Using a Differential Item Functioning Approach
The quality of diagnostic profiles and probability assignment depends on the validity of the proposed attributes and Q-matrix. The rule-space method (RSM), one of diagnostic classification models, provides the quality indices of diagnostic profiles, such as the classification rate and the squared Mahalanobis distance. The study aims to further validate the attributes and Q-matrix that represent construct validity of mathematics tests as well as diagnostic profiles by applying differential item functioning (DIF) analysis along with a different grouping technique. The grouping technique is based on the Q-matrix and diagnostic profiles (i.e., attribute mastery patterns and probabilities) from the RSM. For the validity evidence, it is expected to exhibit DIF for all test items in this study. The results indicated that approximately 90% of items were identified as DIF items that all favored the mastery groups. This suggests that the proposed attributes and Q-matrix with RSM work well to map Taiwanese students into appropriate attribute mastery patterns and to assign appropriate attribute mastery probabilities to them. This study also provides evidence that a DIF approach with the new grouping technique is a viable method for the validation research. More discussion and suggestions are presented in the paper.
- 12 May 2022
- Published by SAGE Publications in Journal of Psychoeducational Assessment
- Vol. 40 (6) , 790-806
Validation of Mathematics Construct and Diagnostic Profiles Using a Differential Item Functioning Approach
The quality of diagnostic profiles and probability assignment depends on the validity of the proposed attributes and Q-matrix. The rule-space method (RSM), one of diagnostic classification models, provides the quality indices of diagnostic profiles, such as the classification rate and the squared Mahalanobis distance. The study aims to further validate the attributes and Q-matrix that represent construct validity of mathematics tests as well as diagnostic profiles by applying differential item functioning (DIF) analysis along with a different grouping technique. The grouping technique is based on the Q-matrix and diagnostic profiles (i.e., attribute mastery patterns and probabilities) from the RSM. For the validity evidence, it is expected to exhibit DIF for all test items in this study. The results indicated that approximately 90% of items were identified as DIF items that all favored the mastery groups. This suggests that the proposed attributes and Q-matrix with RSM work well to map Taiwanese students into appropriate attribute mastery patterns and to assign appropriate attribute mastery probabilities to them. This study also provides evidence that a DIF approach with the new grouping technique is a viable method for the validation research. More discussion and suggestions are presented in the paper.
- Open Access
- 1 October 2021
- Published by SAGE Publications in Sage Open
- Vol. 11 (4)
Learning Strategies as Moderators Between Motivation and Mathematics Performance in East Asian Students: Latent Class Analysis
This study applied a three-step latent class analysis (LCA) approach to explore latent classes of learning strategy use and their moderation effects on the relationships between motivation and mathematics performance. The data of 15-year-old students from five East Asian educational systems related to Chinese culture in the Programme of International Student Assessment (PISA) in 2012 were analyzed. The findings indicated that Shanghai, Singapore, Taiwan, and Macau showed three latent classes of learning strategies, whereas Hong Kong had two latent classes. Most students in the five educational systems reported to use the control strategy, some students reported the use of combined learning strategies, and few students reported the use of memorization except for students in Shanghai. Furthermore, we found the moderation effects of learning strategy use on mathematics performance depended on the types of motivation and educational systems. This study provides insights into the advantages of a three-step LCA approach in educational research.
- 1 October 2021
- Published by SAGE Publications in Sage Open
- Vol. 11 (4)
Open AccessLearning Strategies as Moderators Between Motivation and Mathematics Performance in East Asian Students: Latent Class Analysis
This study applied a three-step latent class analysis (LCA) approach to explore latent classes of learning strategy use and their moderation effects on the relationships between motivation and mathematics performance. The data of 15-year-old students from five East Asian educational systems related to Chinese culture in the Programme of International Student Assessment (PISA) in 2012 were analyzed. The findings indicated that Shanghai, Singapore, Taiwan, and Macau showed three latent classes of learning strategies, whereas Hong Kong had two latent classes. Most students in the five educational systems reported to use the control strategy, some students reported the use of combined learning strategies, and few students reported the use of memorization except for students in Shanghai. Furthermore, we found the moderation effects of learning strategy use on mathematics performance depended on the types of motivation and educational systems. This study provides insights into the advantages of a three-step LCA approach in educational research.
- Open Access
- 1 October 2021
- Published by American Association of Physics Teachers (AAPT)
- p. 371-376
Preliminary results on a video-based force concept inventory
- 1 October 2021
- Published by American Association of Physics Teachers (AAPT)
- p. 371-376
Open AccessPreliminary results on a video-based force concept inventory
- 27 July 2020
- Published by Taylor & Francis in The Journal of Experimental Education
- Vol. 90 (2) , 505-520
Brief Research Report: A Comparison of Robust Tests for Homogeneity of Variance in Factorial ANOVA
A simulation study was conducted to examine the efficacy of nine frequently-used HOV tests, including Levene’s tests with squared residuals and with absolute residuals, Brown and Forsythe (BF) test, Bootstrap BF test, O’Brien test, Z-variance test, Box-Scheffé (BS) test, Bartlett test, and Pseudo jackknife test under comprehensive simulation conditions. The results indicate that O’Brien, Levene’s with squared deviations, BS, BF, and Bootstrap BF tests had adequate Type I error control, especially for O’Brien test and Levene’s test with squared deviations. Under conditions of adequately controlling for Type I error, BF and Bootstrap BF tests had consistently higher power estimates across simulation conditions. Practical recommendations of guidelines for HOV test selection in factorial ANOVA designs are presented in the paper.
- 27 July 2020
- Published by Taylor & Francis in The Journal of Experimental Education
- Vol. 90 (2) , 505-520
Brief Research Report: A Comparison of Robust Tests for Homogeneity of Variance in Factorial ANOVA
A simulation study was conducted to examine the efficacy of nine frequently-used HOV tests, including Levene’s tests with squared residuals and with absolute residuals, Brown and Forsythe (BF) test, Bootstrap BF test, O’Brien test, Z-variance test, Box-Scheffé (BS) test, Bartlett test, and Pseudo jackknife test under comprehensive simulation conditions. The results indicate that O’Brien, Levene’s with squared deviations, BS, BF, and Bootstrap BF tests had adequate Type I error control, especially for O’Brien test and Levene’s test with squared deviations. Under conditions of adequately controlling for Type I error, BF and Bootstrap BF tests had consistently higher power estimates across simulation conditions. Practical recommendations of guidelines for HOV test selection in factorial ANOVA designs are presented in the paper.