Nanyu Luo
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Nanyu Luo has a total of 22 co-authors. Here are the top co-authors by number of shared publications:
- Feng Ji
Department of Applied Psychology and Human Development, University of Toronto, Toronto, Canada
6 shared publications - Jinbo He
Department of Biosciences and Bioinformatics, School of Science, Xi’an Jiaotong-Liverpool University, Suzhou, China
3 shared publications - Biying Zhou
Department of Applied Psychology and Human Development, University of Toronto, Toronto, Canada
2 shared publications - Jia Qi
Department of Applied Psychology and Human Development, University of Toronto, Toronto, Canada
2 shared publications - Xiaoya Zhang
Department of Family, Youth and Community Sciences, Institute of Food and Agricultural Sciences, University of Florida, Gainesville, USA
2 shared publications - Yuchen Zhang
Department of Applied Psychology and Human Development, University of Toronto, Toronto, Canada
2 shared publications
Latest Publications
- Open Access
- 4 June 2026
- Published by Center for Open Science in PsyArXiv
Large Language Models (LLMs) for Evidence Synthesis: An Exploratory Evaluation and A New Approach for Automated Data Extraction and Validation
Data extraction and coding are among one of the most labor-intensive and error-prone stages of evidence synthesis (ES), yet they directly affect the validity of meta-analytic conclusions. Automating this step could alleviate these constraints, but must first demonstrate accurate and reliable extraction. This study investigates whether generative large language models (LLMs) can meet this standard by improving the accuracy, robustness, and traceability of automated extraction for ES in the social and behavioral sciences. We propose the Divide, Conquer, then Recheck (DCR) pipeline, which segments articles into section-level units, applies multimodal multi-pass extraction with JSON-constrained prompting, and resolves cross-section discrepancies through evidence-based aggregation. In a preliminary exploration (Study 1), standard chatbot interfaces demonstrated feasible but uneven extraction, with GPT-4o achieving the highest accuracy (0.80). Using the same model, the DCR pipeline (Study 2) raised accuracy to 0.95 and F1 to 0.97, with gains driven by increased precision. Cross-model evaluation (Study 3) of Gemini 2.5 Flash, GPT-5-mini, and Claude Haiku 4.5 confirmed strong performance across models (F1 above 0.93), with Gemini 2.5 Flash performing best (F1 = 0.99) at low temperature settings (temperature = 0). External validation (Study 4) on three independent meta-analysis datasets maintained F1 above 0.96. An user-friendly interactive comparison dashboard enables researchers to audit extraction reasoning and trace values to source sections. Dashboard review also revealed that some apparent errors reflected limitations in the human-coded reference rather than pipeline failures. These findings suggest that the DCR pipeline improves not only extraction accuracy but also transparency and verifiability, supporting more rigorous ES workflows.
- 4 June 2026
- Published by Center for Open Science in PsyArXiv
Open AccessLarge Language Models (LLMs) for Evidence Synthesis: An Exploratory Evaluation and A New Approach for Automated Data Extraction and Validation
Data extraction and coding are among one of the most labor-intensive and error-prone stages of evidence synthesis (ES), yet they directly affect the validity of meta-analytic conclusions. Automating this step could alleviate these constraints, but must first demonstrate accurate and reliable extraction. This study investigates whether generative large language models (LLMs) can meet this standard by improving the accuracy, robustness, and traceability of automated extraction for ES in the social and behavioral sciences. We propose the Divide, Conquer, then Recheck (DCR) pipeline, which segments articles into section-level units, applies multimodal multi-pass extraction with JSON-constrained prompting, and resolves cross-section discrepancies through evidence-based aggregation. In a preliminary exploration (Study 1), standard chatbot interfaces demonstrated feasible but uneven extraction, with GPT-4o achieving the highest accuracy (0.80). Using the same model, the DCR pipeline (Study 2) raised accuracy to 0.95 and F1 to 0.97, with gains driven by increased precision. Cross-model evaluation (Study 3) of Gemini 2.5 Flash, GPT-5-mini, and Claude Haiku 4.5 confirmed strong performance across models (F1 above 0.93), with Gemini 2.5 Flash performing best (F1 = 0.99) at low temperature settings (temperature = 0). External validation (Study 4) on three independent meta-analysis datasets maintained F1 above 0.96. An user-friendly interactive comparison dashboard enables researchers to audit extraction reasoning and trace values to source sections. Dashboard review also revealed that some apparent errors reflected limitations in the human-coded reference rather than pipeline failures. These findings suggest that the DCR pipeline improves not only extraction accuracy but also transparency and verifiability, supporting more rigorous ES workflows.
- 22 April 2026
- Published by Springer Nature in Behavior Research Methods
- Vol. 58 (5) , 134
BKT: A Bayesian knowledge tracing package for the R environment
Bayesian Knowledge Tracing (BKT) is a popular model in cognitive science and educational technology used to estimate learners’ evolving knowledge sta
- 22 April 2026
- Published by Springer Nature in Behavior Research Methods
- Vol. 58 (5) , 134
BKT: A Bayesian knowledge tracing package for the R environment
Bayesian Knowledge Tracing (BKT) is a popular model in cognitive science and educational technology used to estimate learners’ evolving knowledge sta
- Open Access
- 12 April 2026
- Published by American Educational Research Association (AERA) in Journal of Educational and Behavioral Statistics
Fitting Bayesian Item Response Theory Models Using Deep Learning Computational Frameworks
PyTorch and TensorFlow are two widely adopted modern deep learning frameworks that provide comprehensive computational libraries for developing and fitting complex models. Motivated by the technical barriers in recent item response theory (IRT) work and the lack of practice-oriented tutorials, we demonstrate how modern deep learning platforms can be used for Bayesian IRT parameter estimation by providing a didactic yet in-depth introduction to PyTorch and TensorFlow in a psychometric context, framing IRT models as graphical models, and offering step-by-step guidance that bridges probabilistic machine learning and psychometrics. In this study, we illustrate how to leverage these platforms to estimate widely used psychometric models in educational testing, psychological measurement, and behavioral assessment, namely dichotomous and polytomous IRT models and their multidimensional extensions. We compare Hamiltonian Monte Carlo and variational inference estimators for these models in a unified computational environment. Simulation studies show that both approaches yield parameter estimates with low mean squared error and bias in low-dimensional settings, while also indicating that VI might underestimate aspects of posterior uncertainty in higher-dimensional scenarios. Nonetheless, for practitioners who prioritize computational efficiency and scalability, especially when Graphics Processing Unit (GPU) acceleration is available, VI remains a compelling option. Three empirical case studies further demonstrate how PyTorch- and TensorFlow-based implementations compare with established IRT software in applied settings. We conclude by discussing the broader potential of integrating contemporary deep learning tools and perspectives into psychometric research.
- 12 April 2026
- Published by American Educational Research Association (AERA) in Journal of Educational and Behavioral Statistics
Open AccessFitting Bayesian Item Response Theory Models Using Deep Learning Computational Frameworks
PyTorch and TensorFlow are two widely adopted modern deep learning frameworks that provide comprehensive computational libraries for developing and fitting complex models. Motivated by the technical barriers in recent item response theory (IRT) work and the lack of practice-oriented tutorials, we demonstrate how modern deep learning platforms can be used for Bayesian IRT parameter estimation by providing a didactic yet in-depth introduction to PyTorch and TensorFlow in a psychometric context, framing IRT models as graphical models, and offering step-by-step guidance that bridges probabilistic machine learning and psychometrics. In this study, we illustrate how to leverage these platforms to estimate widely used psychometric models in educational testing, psychological measurement, and behavioral assessment, namely dichotomous and polytomous IRT models and their multidimensional extensions. We compare Hamiltonian Monte Carlo and variational inference estimators for these models in a unified computational environment. Simulation studies show that both approaches yield parameter estimates with low mean squared error and bias in low-dimensional settings, while also indicating that VI might underestimate aspects of posterior uncertainty in higher-dimensional scenarios. Nonetheless, for practitioners who prioritize computational efficiency and scalability, especially when Graphics Processing Unit (GPU) acceleration is available, VI remains a compelling option. Three empirical case studies further demonstrate how PyTorch- and TensorFlow-based implementations compare with established IRT software in applied settings. We conclude by discussing the broader potential of integrating contemporary deep learning tools and perspectives into psychometric research.
- Open Access
- 23 February 2026
- Published by Center for Open Science in PsyArXiv
Predicting probable eating disorders in Chinese adolescents using longitudinal data: A comparison between traditional machine learning and modern deep learning approaches
Eating disorders (EDs) are serious psychiatric conditions, and earlier prediction of ED risk may reduce untreated illness and complications. We compared traditional machine learning (ML) models, including logistic regression, decision trees, support vector machines, and Extreme Gradient Boosting (XGBoost), with two deep learning (DL) models (i.e., xDeepFM, FGCNN+xDeepFM) for predicting probable EDs among Chinese adolescents. Using a four-wave (18-month) longitudinal dataset, we predicted probable ED status at follow-up (T4) from risk factors assessed at T1–T3, with models trained and evaluated separately for females and males. Performance was assessed with accuracy, AUC, and F1-score. XGBoost achieved the best overall performance across sexes (accuracy = .9099, AUC = .9472, F1 = .6314) and outperformed the evaluated DL models. The hybrid DL model (FGCNN+xDeepFM) showed the second-highest performance. Feature-importance analyses indicated that prior eating disturbance was the strongest predictor for both sexes. Among females, the next highest-ranked predictors were body shame, peer appearance pressure, and weight bias internalization; among males, they were weight bias internalization, psychological distress, and family appearance pressure. Thus, traditional machine learning, particularly XGBoost, may support early detection of probable ED risks, while DL approaches were competitive but not consistently superior. Sex-specific profiles may inform tailored prevention.
- 23 February 2026
- Published by Center for Open Science in PsyArXiv
Open AccessPredicting probable eating disorders in Chinese adolescents using longitudinal data: A comparison between traditional machine learning and modern deep learning approaches
Eating disorders (EDs) are serious psychiatric conditions, and earlier prediction of ED risk may reduce untreated illness and complications. We compared traditional machine learning (ML) models, including logistic regression, decision trees, support vector machines, and Extreme Gradient Boosting (XGBoost), with two deep learning (DL) models (i.e., xDeepFM, FGCNN+xDeepFM) for predicting probable EDs among Chinese adolescents. Using a four-wave (18-month) longitudinal dataset, we predicted probable ED status at follow-up (T4) from risk factors assessed at T1–T3, with models trained and evaluated separately for females and males. Performance was assessed with accuracy, AUC, and F1-score. XGBoost achieved the best overall performance across sexes (accuracy = .9099, AUC = .9472, F1 = .6314) and outperformed the evaluated DL models. The hybrid DL model (FGCNN+xDeepFM) showed the second-highest performance. Feature-importance analyses indicated that prior eating disturbance was the strongest predictor for both sexes. Among females, the next highest-ranked predictors were body shame, peer appearance pressure, and weight bias internalization; among males, they were weight bias internalization, psychological distress, and family appearance pressure. Thus, traditional machine learning, particularly XGBoost, may support early detection of probable ED risks, while DL approaches were competitive but not consistently superior. Sex-specific profiles may inform tailored prevention.
- Open Access
- 19 September 2025
- Published by The Open Journal in Journal of Open Source Software
- Vol. 10 (113) , 7869
FedIRT: An R package and shiny app for estimating federated item response theory models
Educational and psychological testing data are often sensitive, making centralized analysis challenging due to privacy concerns. To address this, we developed FedIRT, an R package that estimates Item Response Theory (IRT) models-including 1PL, 2PL, and graded response models, using federated learning. This approach enables multiple institutions (e.g., schools, districts) to collaboratively estimate model parameters without sharing raw data. FedIRT ensures privacy preservation while maintaining estimation accuracy comparable to mainstream IRT packages. Additionally, the package includes a user-friendly Shiny app, making federated IRT analysis accessible to researchers and practitioners without requiring advanced programming skills.
- 19 September 2025
- Published by The Open Journal in Journal of Open Source Software
- Vol. 10 (113) , 7869
Open AccessFedIRT: An R package and shiny app for estimating federated item response theory models
Educational and psychological testing data are often sensitive, making centralized analysis challenging due to privacy concerns. To address this, we developed FedIRT, an R package that estimates Item Response Theory (IRT) models-including 1PL, 2PL, and graded response models, using federated learning. This approach enables multiple institutions (e.g., schools, districts) to collaboratively estimate model parameters without sharing raw data. FedIRT ensures privacy preservation while maintaining estimation accuracy comparable to mainstream IRT packages. Additionally, the package includes a user-friendly Shiny app, making federated IRT analysis accessible to researchers and practitioners without requiring advanced programming skills.
- Open Access
- 15 April 2025
- Published by Center for Open Science in OSF Preprints
Fitting item response theory models using deep learning computational frameworks
PyTorch and TensorFlow are two widely adopted, modern deep learning frameworksthat offer comprehensive computation libraries for deep learning models. We illustratehow to utilize these deep learning computational platforms and infrastructure to estimate aclass of popular psychometric models, dichotomous and polytomous Item Response Theory(IRT) models, along with their multidimensional extensions. Through simulation studies,the estimation performance on the simulated datasets demonstrates low mean square errorand bias for model parameters. We discuss the potential of integrating modern deeplearning tools and views into psychometric research.
- 15 April 2025
- Published by Center for Open Science in OSF Preprints
Open AccessFitting item response theory models using deep learning computational frameworks
PyTorch and TensorFlow are two widely adopted, modern deep learning frameworksthat offer comprehensive computation libraries for deep learning models. We illustratehow to utilize these deep learning computational platforms and infrastructure to estimate aclass of popular psychometric models, dichotomous and polytomous Item Response Theory(IRT) models, along with their multidimensional extensions. Through simulation studies,the estimation performance on the simulated datasets demonstrates low mean square errorand bias for model parameters. We discuss the potential of integrating modern deeplearning tools and views into psychometric research.