Ren Liu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Ren Liu has a total of 51 co-authors. Here are the top co-authors by number of shared publications:
- Zhehan Jiang
Institute of Medical Education, Health Science Center, Peking University, Beijing, China
7 shared publications Haiyan Liu
University of California
6 shared publicationsAnne Corinne Huggins-Manley
University of Florida, Gainesville, FL, USA
4 shared publications- Dexin Shi
University of South Carolina
4 shared publications Dexin Shi
University of South Carolina, Columbia, SC, USA
3 shared publications- Jorge Luis Bazan
Faculty of Mathematics Pontificia Universidad Católica de Chile Santiago Chile
3 shared publications
Latest Publications
- Open Access
- 15 July 2025
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 50 (1-2) , 21-32
A Study of Latent State-Trait Theory Framework in Piecewise Growth Models
Latent state-trait (LST) theory provides a psychometric framework that facilitates the measurement of long-term trait change and short-term state variability in longitudinal data. While LST theory has guided the development and extension of linear latent growth models within its theoretical framework, the integration of piecewise growth models (PGMs) into the LST theory framework remains uninvestigated. PGMs are well suited for modeling nonlinear developmental processes comprised of distinct stages, which frequently arise in psychological and educational research. Their ability to capture phase-specific changes makes them a useful tool for applied and methodological researchers. This paper introduces a novel measurement approach that integrates PGMs into the framework of LST theory by presenting single-indicator piecewise growth models (SI-PGMs) and multiple-indicator piecewise growth models (MI-PGMs). We detail the model specifications for both SI-PGMs and MI-PGMs. For SI-PGMs, we define the reliability coefficient; for MI-PGMs, we define the consistency coefficient, occasion specificity coefficient, and reliability coefficient. We then conduct simulations to evaluate the models’ performance in accurately recovering growth parameters and capturing true reliability. The simulation results indicated that SI-PGMs and MI-PGMs successfully recovered growth parameters and performed comparably in the absence of situational influences. However, MI-PGMs outperformed SI-PGMs when situational influences were present. We conclude by outlining directions for future research and providing M plus syntax to support the dissemination of the models.
- 15 July 2025
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 50 (1-2) , 21-32
Open AccessA Study of Latent State-Trait Theory Framework in Piecewise Growth Models
Latent state-trait (LST) theory provides a psychometric framework that facilitates the measurement of long-term trait change and short-term state variability in longitudinal data. While LST theory has guided the development and extension of linear latent growth models within its theoretical framework, the integration of piecewise growth models (PGMs) into the LST theory framework remains uninvestigated. PGMs are well suited for modeling nonlinear developmental processes comprised of distinct stages, which frequently arise in psychological and educational research. Their ability to capture phase-specific changes makes them a useful tool for applied and methodological researchers. This paper introduces a novel measurement approach that integrates PGMs into the framework of LST theory by presenting single-indicator piecewise growth models (SI-PGMs) and multiple-indicator piecewise growth models (MI-PGMs). We detail the model specifications for both SI-PGMs and MI-PGMs. For SI-PGMs, we define the reliability coefficient; for MI-PGMs, we define the consistency coefficient, occasion specificity coefficient, and reliability coefficient. We then conduct simulations to evaluate the models’ performance in accurately recovering growth parameters and capturing true reliability. The simulation results indicated that SI-PGMs and MI-PGMs successfully recovered growth parameters and performed comparably in the absence of situational influences. However, MI-PGMs outperformed SI-PGMs when situational influences were present. We conclude by outlining directions for future research and providing M plus syntax to support the dissemination of the models.
- 1 March 2025
- Published by Institute of Mathematical Statistics in Brazilian Journal of Probability and Statistics
- Vol. 39 (1) , 19-38
Multidimensional graded response models with hierarchical structure and Q-matrix
Multidimensional item response theory (MIRT) models estimate multiple latent traits from individual item response data and have been applied to problems in several knowledge areas. This article explores some statistical aspects about the compensatory multidimensional graded response model with two hierarchical structures and Q-matrix. Specifically, the efficiency of the estimation method using No-U-Turn Sampler (NUTS) algorithm and the performance of the three model comparison criteria regarding the adequacy of the models with the same dataset were verified. An application was conducted with recent data from responses to a questionnaire on sustainability perception applied to residents in Brazil verified the model that best fit the data, thus justifying the choice of the appropriate model for the research. The results were used for the selection of a model for the application. The explored models provided detailed information about individuals, which may be useful for the design of public policies aimed at the development of specific educational actions for the economic, environmental and social fields. Codes are available for the proposed methodology to be explored in other applications.
- 1 March 2025
- Published by Institute of Mathematical Statistics in Brazilian Journal of Probability and Statistics
- Vol. 39 (1) , 19-38
Multidimensional graded response models with hierarchical structure and Q-matrix
Multidimensional item response theory (MIRT) models estimate multiple latent traits from individual item response data and have been applied to problems in several knowledge areas. This article explores some statistical aspects about the compensatory multidimensional graded response model with two hierarchical structures and Q-matrix. Specifically, the efficiency of the estimation method using No-U-Turn Sampler (NUTS) algorithm and the performance of the three model comparison criteria regarding the adequacy of the models with the same dataset were verified. An application was conducted with recent data from responses to a questionnaire on sustainability perception applied to residents in Brazil verified the model that best fit the data, thus justifying the choice of the appropriate model for the research. The results were used for the selection of a model for the application. The explored models provided detailed information about individuals, which may be useful for the design of public policies aimed at the development of specific educational actions for the economic, environmental and social fields. Codes are available for the proposed methodology to be explored in other applications.
- Open Access
- 18 February 2025
- Published by Taylor & Francis in Medical Education Online
- Vol. 30 (1) , 2467477
Optimizing cost-effectiveness in remote objective structured clinical examinations through targeted double scoring methodologies
The remote Objective Structured Clinical Examination (OSCE) is a cornerstone of medical education, enabling structured and objective assessment of clinical skills, communication, and patient-centered care. However, its widespread adoption has introduced challenges related to cost-effectiveness and efficient use of rater resources. Traditional double scoring (DS) ensures reliability but is labor-intensive and costly, especially in large-scale assessments. To address these challenges, this study introduces Targeted Double Scoring (TDS), a novel methodology that selectively applies DS to specific score ranges, particularly those near the pass/fail threshold. The study was conducted using data from a pilot remote OSCE administered to 550 clinical medicine undergraduates in China. The OSCE consisted of three stations: Clinical Reasoning (CR), Physical Examination (PE), and Fundamental Skills (FS). Each station was scored remotely by two raters, with a cut-off score of 60 out of 100. The TDS methodology was modeled based on the OSCE's DS design and fitted with scoring data. A decision-theoretic approach identified optimal Critical Score Ranges (CSRs) for targeted double scoring, balancing reliability and cost-effectiveness. The findings show that TDS significantly reduces rater workload and costs while maintaining high reliability and fairness. For instance, TDS achieved up to 70% cost savings compared to traditional DS under certain configurations. The study also highlights the flexibility of TDS, which can be tailored to different OSCE designs and scoring rubrics. These results have broad implications for medical education, especially in resource-constrained settings where optimizing assessment efficiency is critical. This study provides a practical solution to the cost-related challenges of remote OSCEs and offers a framework for adopting TDS in assessments. By focusing raters on critical score ranges, TDS maintains rigorous and fair evaluations without overburdening faculty or exceeding budgets. Future research should explore TDS scalability and its integration with emerging technologies like artificial intelligence to enhance efficiency and reliability.
- 18 February 2025
- Published by Taylor & Francis in Medical Education Online
- Vol. 30 (1) , 2467477
Open AccessOptimizing cost-effectiveness in remote objective structured clinical examinations through targeted double scoring methodologies
The remote Objective Structured Clinical Examination (OSCE) is a cornerstone of medical education, enabling structured and objective assessment of clinical skills, communication, and patient-centered care. However, its widespread adoption has introduced challenges related to cost-effectiveness and efficient use of rater resources. Traditional double scoring (DS) ensures reliability but is labor-intensive and costly, especially in large-scale assessments. To address these challenges, this study introduces Targeted Double Scoring (TDS), a novel methodology that selectively applies DS to specific score ranges, particularly those near the pass/fail threshold. The study was conducted using data from a pilot remote OSCE administered to 550 clinical medicine undergraduates in China. The OSCE consisted of three stations: Clinical Reasoning (CR), Physical Examination (PE), and Fundamental Skills (FS). Each station was scored remotely by two raters, with a cut-off score of 60 out of 100. The TDS methodology was modeled based on the OSCE's DS design and fitted with scoring data. A decision-theoretic approach identified optimal Critical Score Ranges (CSRs) for targeted double scoring, balancing reliability and cost-effectiveness. The findings show that TDS significantly reduces rater workload and costs while maintaining high reliability and fairness. For instance, TDS achieved up to 70% cost savings compared to traditional DS under certain configurations. The study also highlights the flexibility of TDS, which can be tailored to different OSCE designs and scoring rubrics. These results have broad implications for medical education, especially in resource-constrained settings where optimizing assessment efficiency is critical. This study provides a practical solution to the cost-related challenges of remote OSCEs and offers a framework for adopting TDS in assessments. By focusing raters on critical score ranges, TDS maintains rigorous and fair evaluations without overburdening faculty or exceeding budgets. Future research should explore TDS scalability and its integration with emerging technologies like artificial intelligence to enhance efficiency and reliability.
- Open Access
- 31 October 2024
- Published by Springer Nature in International Journal of Behavioral Medicine
- Vol. 31 (6) , 815-818
Advancing Behavioral Medicine: The International Pursuit of Science for 30 Years
- 31 October 2024
- Published by Springer Nature in International Journal of Behavioral Medicine
- Vol. 31 (6) , 815-818
Open AccessAdvancing Behavioral Medicine: The International Pursuit of Science for 30 Years
- Open Access
- 23 July 2024
- Published by Taylor & Francis in Structural Equation Modeling: A Multidisciplinary Journal
- Vol. 32 (3) , 550-564
Latent Variable Modeling of Social Networks with Directional Relations: An Exploration of Profile Similarity of Latent Factors
- 23 July 2024
- Published by Taylor & Francis in Structural Equation Modeling: A Multidisciplinary Journal
- Vol. 32 (3) , 550-564
Open AccessLatent Variable Modeling of Social Networks with Directional Relations: An Exploration of Profile Similarity of Latent Factors
- 29 October 2023
- Published by Wiley in Journal of Addictions & Offender Counseling
- Vol. 45 (1) , 3-19
Attitudes toward individuals with opioid use disorders and criminal justice involvement: An experimental vignette study
Providing moderate levels of treatment to individuals with a criminal justice history is associated with lowered rates of recidivism; yet, treatment access remains low. One possible factor contributing to the lack of treatment options is public stigma. An experimental study was conducted that tested for differences in stigmatizing attitudes across conditions: (a) major depressive disorder (MDD), (b) MDD and opioid use disorder (OUD), and (c) MDD, OUD, and nonviolent, drug‐related criminal histories. Findings indicate that individuals who meet the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, criteria for OUD with MDD, regardless of criminal histories, are stigmatized at higher levels than those with MDD alone.
- 29 October 2023
- Published by Wiley in Journal of Addictions & Offender Counseling
- Vol. 45 (1) , 3-19
Attitudes toward individuals with opioid use disorders and criminal justice involvement: An experimental vignette study
Providing moderate levels of treatment to individuals with a criminal justice history is associated with lowered rates of recidivism; yet, treatment access remains low. One possible factor contributing to the lack of treatment options is public stigma. An experimental study was conducted that tested for differences in stigmatizing attitudes across conditions: (a) major depressive disorder (MDD), (b) MDD and opioid use disorder (OUD), and (c) MDD, OUD, and nonviolent, drug‐related criminal histories. Findings indicate that individuals who meet the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, criteria for OUD with MDD, regardless of criminal histories, are stigmatized at higher levels than those with MDD alone.
- 27 March 2023
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 84 (1) , 171-189
Evaluating Close Fit in Ordinal Factor Analysis Models With Multiply Imputed Data
Multiple imputation (MI) is one of the recommended techniques for handling missing data in ordinal factor analysis models. However, methods for computing MI-based fit indices under ordinal factor analysis models have yet to be developed. In this short note, we introduced the methods of using the standardized root mean squared residual (SRMR) and the root mean square error of approximation (RMSEA) to assess the fit of ordinal factor analysis models with multiply imputed data. Specifically, we described the procedure for computing the MI-based sample estimates and constructing the confidence intervals. Simulation results showed that the proposed methods could yield sufficiently accurate point and interval estimates for both SRMR and RMSEA, especially in conditions with larger sample sizes, less missing data, more response categories, and higher degrees of misfit. Based on the findings, implications and recommendations were discussed.
- 27 March 2023
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 84 (1) , 171-189
Evaluating Close Fit in Ordinal Factor Analysis Models With Multiply Imputed Data
Multiple imputation (MI) is one of the recommended techniques for handling missing data in ordinal factor analysis models. However, methods for computing MI-based fit indices under ordinal factor analysis models have yet to be developed. In this short note, we introduced the methods of using the standardized root mean squared residual (SRMR) and the root mean square error of approximation (RMSEA) to assess the fit of ordinal factor analysis models with multiply imputed data. Specifically, we described the procedure for computing the MI-based sample estimates and constructing the confidence intervals. Simulation results showed that the proposed methods could yield sufficiently accurate point and interval estimates for both SRMR and RMSEA, especially in conditions with larger sample sizes, less missing data, more response categories, and higher degrees of misfit. Based on the findings, implications and recommendations were discussed.
- Open Access
- 6 September 2022
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 47 (1) , 64-75
Applying Negative Binomial Distribution in Diagnostic Classification Models for Analyzing Count Data
Diagnostic classification models (DCMs) have been used to classify examinees into groups based on their possession status of a set of latent traits. In addition to traditional item-based scoring approaches, examinees may be scored based on their completion of a series of small and similar tasks. Those scores are usually considered as count variables. To model count scores, this study proposes a new class of DCMs that uses the negative binomial distribution at its core. We explained the proposed model framework and demonstrated its use through an operational example. Simulation studies were conducted to evaluate the performance of the proposed model and compare it with the Poisson-based DCM.
- 6 September 2022
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 47 (1) , 64-75
Open AccessApplying Negative Binomial Distribution in Diagnostic Classification Models for Analyzing Count Data
Diagnostic classification models (DCMs) have been used to classify examinees into groups based on their possession status of a set of latent traits. In addition to traditional item-based scoring approaches, examinees may be scored based on their completion of a series of small and similar tasks. Those scores are usually considered as count variables. To model count scores, this study proposes a new class of DCMs that uses the negative binomial distribution at its core. We explained the proposed model framework and demonstrated its use through an operational example. Simulation studies were conducted to evaluate the performance of the proposed model and compare it with the Poisson-based DCM.
- 4 September 2022
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 83 (5) , 984-1006
The NEAT Equating Via Chaining Random Forests in the Context of Small Sample Sizes: A Machine-Learning Method
The part of responses that is absent in the nonequivalent groups with anchor test (NEAT) design can be managed to a planned missing scenario. In the context of small sample sizes, we present a machine learning (ML)-based imputation technique called chaining random forests (CRF) to perform equating tasks within the NEAT design. Specifically, seven CRF-based imputation equating methods are proposed based on different data augmentation methods. The equating performance of the proposed methods is examined through a simulation study. Five factors are considered: (a) test length (20, 30, 40, 50), (b) sample size per test form (50 versus 100), (c) ratio of common/anchor items (0.2 versus 0.3), and (d) equivalent versus nonequivalent groups taking the two forms (no mean difference versus a mean difference of 0.5), and (e) three different types of anchors (random, easy, and hard), resulting in 96 conditions. In addition, five traditional equating methods, (1) Tucker method; (2) Levine observed score method; (3) equipercentile equating method; (4) circle-arc method; and (5) concurrent calibration based on Rasch model, were also considered, plus seven CRF-based imputation equating methods for a total of 12 methods in this study. The findings suggest that benefiting from the advantages of ML techniques, CRF-based methods that incorporate the equating result of the Tucker method, such as IMP_total_Tucker, IMP_pair_Tucker, and IMP_Tucker_cirlce methods, can yield more robust and trustable estimates for the “missingness” in an equating task and therefore result in more accurate equated scores than other counterparts in short-length tests with small samples.
- 4 September 2022
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 83 (5) , 984-1006
The NEAT Equating Via Chaining Random Forests in the Context of Small Sample Sizes: A Machine-Learning Method
The part of responses that is absent in the nonequivalent groups with anchor test (NEAT) design can be managed to a planned missing scenario. In the context of small sample sizes, we present a machine learning (ML)-based imputation technique called chaining random forests (CRF) to perform equating tasks within the NEAT design. Specifically, seven CRF-based imputation equating methods are proposed based on different data augmentation methods. The equating performance of the proposed methods is examined through a simulation study. Five factors are considered: (a) test length (20, 30, 40, 50), (b) sample size per test form (50 versus 100), (c) ratio of common/anchor items (0.2 versus 0.3), and (d) equivalent versus nonequivalent groups taking the two forms (no mean difference versus a mean difference of 0.5), and (e) three different types of anchors (random, easy, and hard), resulting in 96 conditions. In addition, five traditional equating methods, (1) Tucker method; (2) Levine observed score method; (3) equipercentile equating method; (4) circle-arc method; and (5) concurrent calibration based on Rasch model, were also considered, plus seven CRF-based imputation equating methods for a total of 12 methods in this study. The findings suggest that benefiting from the advantages of ML techniques, CRF-based methods that incorporate the equating result of the Tucker method, such as IMP_total_Tucker, IMP_pair_Tucker, and IMP_Tucker_cirlce methods, can yield more robust and trustable estimates for the “missingness” in an equating task and therefore result in more accurate equated scores than other counterparts in short-length tests with small samples.
- 1 August 2022
- Published by American Psychological Association (APA) in Training and Education in Professional Psychology
- Vol. 16 (3) , 253-262
Examining the effects of the supervisory relationship and therapeutic alliance on client outcomes in novice therapists.
- 1 August 2022
- Published by American Psychological Association (APA) in Training and Education in Professional Psychology
- Vol. 16 (3) , 253-262
Examining the effects of the supervisory relationship and therapeutic alliance on client outcomes in novice therapists.