Michelle Y. Chen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Michelle Y. Chen has a total of 27 co-authors. Here are the top co-authors by number of shared publications:
- Yan Liu
Carleton University
2 shared publications - Alejandra Fonseca-Cuevas
University of Saskatchewan
1 shared publication Amery D. Wu
The University of British Columbia, Vancouver, Canada
1 shared publication- Ashley Pullman
Education Policy Research Initiative, Graduate School of Public and International Affairs, Faculty of Social Sciences, University of Ottawa, Ottawa, ON, Canada
1 shared publication Audrey Béliveau
Department of Statistics and Actuarial Science, University of Waterloo
1 shared publication- Bruno D. Zumbo
The University of British Columbia
1 shared publication
Latest Publications
- Open Access
Identifying Longitudinal CD4:CD8 Ratio Trajectories Indicative of Chronic Renal Disease Risk among People Living with HIV: An Application of Growth Mixture Models
The incidence of chronic kidney disease (CKD) is increasing among people living with HIV (PLWH). Routine monitoring of indicators such as CD4:CD8 ratio might improve the early detection of CKD. Our objective was to identify clinically relevant CD4:CD8 ratio trajectories indicative of CKD risk. Participants were ≥ 18 years old, initiated antiretroviral therapy between 2000 and 2016, and were followed for ≥6 months until 31 March 2017 or last contact date. Outcome was incidence of CKD. Growth mixture models (GMMs) and decay models were used to compare CD4:CD8 ratio trajectories. Following GMM, 4547 (93.5%) participants were classified in Class 1 with 5.4% developing CKD, and 316 (6.5%) participants were classified in Class 2 with 20.9% developing CKD. The final model suggested that participants in Class 2 had 8.72 times the incidence rate of developing CKD than those in Class 1. Exponential decay models indicated a significant CD4:CD8 ratio decline among Class 2 participants who developed CKD. Among those who developed CKD in Class 2, starting at 5.5 years of follow-up, the slope of their ratio trajectory curve changed significantly, and the rate of decline increased dramatically. Routine monitored CD4:CD8 ratios can be an effective strategy to identify early CKD risk among PLWH.
Open AccessIdentifying Longitudinal CD4:CD8 Ratio Trajectories Indicative of Chronic Renal Disease Risk among People Living with HIV: An Application of Growth Mixture Models
The incidence of chronic kidney disease (CKD) is increasing among people living with HIV (PLWH). Routine monitoring of indicators such as CD4:CD8 ratio might improve the early detection of CKD. Our objective was to identify clinically relevant CD4:CD8 ratio trajectories indicative of CKD risk. Participants were ≥ 18 years old, initiated antiretroviral therapy between 2000 and 2016, and were followed for ≥6 months until 31 March 2017 or last contact date. Outcome was incidence of CKD. Growth mixture models (GMMs) and decay models were used to compare CD4:CD8 ratio trajectories. Following GMM, 4547 (93.5%) participants were classified in Class 1 with 5.4% developing CKD, and 316 (6.5%) participants were classified in Class 2 with 20.9% developing CKD. The final model suggested that participants in Class 2 had 8.72 times the incidence rate of developing CKD than those in Class 1. Exponential decay models indicated a significant CD4:CD8 ratio decline among Class 2 participants who developed CKD. Among those who developed CKD in Class 2, starting at 5.5 years of follow-up, the slope of their ratio trajectory curve changed significantly, and the rate of decline increased dramatically. Routine monitored CD4:CD8 ratios can be an effective strategy to identify early CKD risk among PLWH.
- 18 October 2022
- Published by Taylor & Francis in Multivariate Behavioral Research
- Vol. 58 (4) , 706-722
A Gentle Introduction to Bayesian Network Meta-Analysis Using an Automated R Package
Network meta-analysis is an extension of standard meta-analysis. It allows researchers to build a network of evidence to compare multiple interventions that may have not been compared directly in existing publications. With a Bayesian approach, network meta-analysis can be used to obtain a posterior probability distribution of all the relative treatment effects, which allows for the estimation of relative treatment effects to quantify the uncertainty of parameter estimates, and to rank all the treatments in the network. Ranking treatments using both direct and indirect evidence can provide guidance to policy makers and clinicians for making decisions. The purpose of this paper is to introduce fundamental concepts of Bayesian network meta-analysis (BNMA) to researchers in psychology and social sciences. We discuss several essential concepts of BNMA, including the assumptions of homogeneity and consistency, the fixed and random effects models, prior specification, and model fit evaluation strategies, while pointing out some issues and areas where researchers should use caution in the application of BNMA. Additionally, using an automated R package, we provide a step-by-step demonstration on how to conduct and report the findings of BNMA with a real dataset of psychological interventions extracted from PubMed.
- 18 October 2022
- Published by Taylor & Francis in Multivariate Behavioral Research
- Vol. 58 (4) , 706-722
A Gentle Introduction to Bayesian Network Meta-Analysis Using an Automated R Package
Network meta-analysis is an extension of standard meta-analysis. It allows researchers to build a network of evidence to compare multiple interventions that may have not been compared directly in existing publications. With a Bayesian approach, network meta-analysis can be used to obtain a posterior probability distribution of all the relative treatment effects, which allows for the estimation of relative treatment effects to quantify the uncertainty of parameter estimates, and to rank all the treatments in the network. Ranking treatments using both direct and indirect evidence can provide guidance to policy makers and clinicians for making decisions. The purpose of this paper is to introduce fundamental concepts of Bayesian network meta-analysis (BNMA) to researchers in psychology and social sciences. We discuss several essential concepts of BNMA, including the assumptions of homogeneity and consistency, the fixed and random effects models, prior specification, and model fit evaluation strategies, while pointing out some issues and areas where researchers should use caution in the application of BNMA. Additionally, using an automated R package, we provide a step-by-step demonstration on how to conduct and report the findings of BNMA with a real dataset of psychological interventions extracted from PubMed.
- 26 October 2020
- Published by Taylor & Francis in International Journal of Lifelong Education
- Vol. 39 (5-6) , 576-590
Varieties of employment: a comparison of skill-based activities at work among youth and young adults in Canada
According to practice engagement theory within the field of adult education, everyday workplace activities offer an opportunity for skill development among youth and young adults; yet, prior research highlights the prevalence of routine and low-skill work within entry-level jobs. Utilising data from the Programme for the International Assessment of Adult Competencies (PIAAC), the objective of the following study is to demonstrate how workplace literacy, numeracy, and problem-solving activities differ among people aged 16 to 34 in Canada. Latent class analysis provides a typology of literacy, numeracy, and problem-solving activities at work and generates insight into how socio-demographic and employment factors are associated with each group. The findings highlight that employment cannot be uniformly characterised as low-skill. Rather, there are unique employment groups with differing literacy, numeracy, and problem-solving activity characteristics. Nevertheless, socio-demographic factors do matter for gaining access to employment that involves daily skill-based activities. While age, parental education, immigration background, and education level are significantly associated with the latent classes, economic and employment characteristics reduce the magnitude of these associations. Together, the results suggest that access to skill-based activities at work are more or less accessible for certain groups and connect to other aspects of early-career job quality and occupational segregation.
- 26 October 2020
- Published by Taylor & Francis in International Journal of Lifelong Education
- Vol. 39 (5-6) , 576-590
Varieties of employment: a comparison of skill-based activities at work among youth and young adults in Canada
According to practice engagement theory within the field of adult education, everyday workplace activities offer an opportunity for skill development among youth and young adults; yet, prior research highlights the prevalence of routine and low-skill work within entry-level jobs. Utilising data from the Programme for the International Assessment of Adult Competencies (PIAAC), the objective of the following study is to demonstrate how workplace literacy, numeracy, and problem-solving activities differ among people aged 16 to 34 in Canada. Latent class analysis provides a typology of literacy, numeracy, and problem-solving activities at work and generates insight into how socio-demographic and employment factors are associated with each group. The findings highlight that employment cannot be uniformly characterised as low-skill. Rather, there are unique employment groups with differing literacy, numeracy, and problem-solving activity characteristics. Nevertheless, socio-demographic factors do matter for gaining access to employment that involves daily skill-based activities. While age, parental education, immigration background, and education level are significantly associated with the latent classes, economic and employment characteristics reduce the magnitude of these associations. Together, the results suggest that access to skill-based activities at work are more or less accessible for certain groups and connect to other aspects of early-career job quality and occupational segregation.
- Open Access
- 16 October 2020
- Published by University of New Brunswick Libraries - UNB in Canadian Journal of Applied Linguistics / Revue Canadienne de Linguistique Appliquée
- Vol. 23 (2) , 1-19
Investigating the Alignment Between the CELPIP-General Reading Test and the Canadian Language Benchmarks: A Content Validation Study
Seeking evidence to support content validity is essential to test validation. This is especially the case in contexts where test scores are interpreted in relation to external proficiency standards and where new test content is constantly being produced to meet test administration and security demands. In this paper, we describe a modified scale- anchoring approach to assessing the alignment between the Canadian English Language Proficiency Index Program (CELPIP) test and the Canadian Language Benchmarks (CLB), the proficiency framework to which the test scores are linked. We discuss how proficiency frameworks such as the CLB can be used to support the content validation of large-scale standardized tests through an evaluation of the alignment between the test content and the performance standards. By sharing both the positive implications and challenges of working with the CLB in high-stakes language test validation, we hope to help raise the profile of this national language framework among scholars and practitioners.
- 16 October 2020
- Published by University of New Brunswick Libraries - UNB in Canadian Journal of Applied Linguistics / Revue Canadienne de Linguistique Appliquée
- Vol. 23 (2) , 1-19
Open AccessInvestigating the Alignment Between the CELPIP-General Reading Test and the Canadian Language Benchmarks: A Content Validation Study
Seeking evidence to support content validity is essential to test validation. This is especially the case in contexts where test scores are interpreted in relation to external proficiency standards and where new test content is constantly being produced to meet test administration and security demands. In this paper, we describe a modified scale- anchoring approach to assessing the alignment between the Canadian English Language Proficiency Index Program (CELPIP) test and the Canadian Language Benchmarks (CLB), the proficiency framework to which the test scores are linked. We discuss how proficiency frameworks such as the CLB can be used to support the content validation of large-scale standardized tests through an evaluation of the alignment between the test content and the performance standards. By sharing both the positive implications and challenges of working with the CLB in high-stakes language test validation, we hope to help raise the profile of this national language framework among scholars and practitioners.
- 28 June 2020
- Published by SAGE Publications in Language Testing
- Vol. 37 (4) , 523-549
Understanding writing quality change: A longitudinal study of repeaters of a high-stakes standardized English proficiency test
This study examined the writing score and writing feature changes of 562 repeat test takers who took the Canadian English Language Proficiency Index Program–General (CELPIP–General) test at least three times, with a short (30–40 day) interval between the first and second attempts and a longer (90–180 day) interval between the first and third attempts. Analysis was conducted to uncover whether changes occurred at different testing durations (short vs. long) and whether the observed changes varied across repeater’s initial proficiency groups (low, mid, high). The writing scores measured by CELPIP bands showed great stability over the 6-month period, but the trends of development differed by proficiency group. Low proficiency test takers were more likely to have faster observable score gains, compared to the medium proficiency group, whereas high proficiency repeaters may not maintain their score levels at later attempts. Writing quality was analyzed using natural language processing (NLP) tools. Results suggested that for all proficiency groups, lexical features were more likely to improve over the 6-month period, with some measures showing improvement at 1 month; features in cohesion and syntactic sophistication, however, did not change significantly.
- 28 June 2020
- Published by SAGE Publications in Language Testing
- Vol. 37 (4) , 523-549
Understanding writing quality change: A longitudinal study of repeaters of a high-stakes standardized English proficiency test
This study examined the writing score and writing feature changes of 562 repeat test takers who took the Canadian English Language Proficiency Index Program–General (CELPIP–General) test at least three times, with a short (30–40 day) interval between the first and second attempts and a longer (90–180 day) interval between the first and third attempts. Analysis was conducted to uncover whether changes occurred at different testing durations (short vs. long) and whether the observed changes varied across repeater’s initial proficiency groups (low, mid, high). The writing scores measured by CELPIP bands showed great stability over the 6-month period, but the trends of development differed by proficiency group. Low proficiency test takers were more likely to have faster observable score gains, compared to the medium proficiency group, whereas high proficiency repeaters may not maintain their score levels at later attempts. Writing quality was analyzed using natural language processing (NLP) tools. Results suggested that for all proficiency groups, lexical features were more likely to improve over the 6-month period, with some measures showing improvement at 1 month; features in cohesion and syntactic sophistication, however, did not change significantly.
- Open Access
- 3 June 2020
- Published by Frontiers Media SA in Frontiers in Psychology
- Vol. 11, 1088
Using Corpus Analyses to Help Address the DIF Interpretation: Gender Differences in Standardized Writing Assessment
Addressing differential item functioning (DIF) provides validity evidence to support the interpretation of test scores across groups. Conventional DIF methods flag DIF items statistically, but often fail to consolidate a substantive interpretation. The lack of interpretability of DIF results is particularly pronounced in writing assessment where the matching of test takers’ proficiency levels often relies on external variables and the reported DIF effect is frequently small in magnitude. Using responses to a prompt that showed small gender DIF favoring female test takers, we demonstrate a corpus-based approach that helps address DIF interpretation. To provide linguistic insights into the possible sources of the small DIF effect, this study compared a gender-balanced corpus of 826 writing samples matched by test takers’ performance on the reading and listening components of the test. Four groups of linguistic features that correspond to the rating dimensions, and thus partially represent the writing construct were analyzed. They include (1) sentiment and social cognition, (2) cohesion, (3) syntactic features, and (4) lexical features. After initial screening, 123 linguistic features, all of which were correlated with the writing scores, were retained for gender comparison. Among these selected features, female test takers’ writing samples scored higher on six of them with small effect sizes in the categories of cohesion and syntactic features. Three of the six features were positively correlated with higher writing scores, while the other three were negative. These results are largely consistent with previous findings of gender differences in language use. Additionally, the small differences in the language features of the writing samples (in terms of the small number of features that differ between genders and the small effect size of the observed differences) are consistent with the previous DIF results, both suggesting that the effect of gender differences on the writing scores is likely to be very small. In sum, the corpus-based findings provide linguistic insights into the gender-related language differences and their potential consequences in a testing context. These findings are meaningful for furthering our understanding of the small gender DIF effect identified through statistical analysis, which lends support to the validity of writing scores.
- 3 June 2020
- Published by Frontiers Media SA in Frontiers in Psychology
- Vol. 11, 1088
Open AccessUsing Corpus Analyses to Help Address the DIF Interpretation: Gender Differences in Standardized Writing Assessment
Addressing differential item functioning (DIF) provides validity evidence to support the interpretation of test scores across groups. Conventional DIF methods flag DIF items statistically, but often fail to consolidate a substantive interpretation. The lack of interpretability of DIF results is particularly pronounced in writing assessment where the matching of test takers’ proficiency levels often relies on external variables and the reported DIF effect is frequently small in magnitude. Using responses to a prompt that showed small gender DIF favoring female test takers, we demonstrate a corpus-based approach that helps address DIF interpretation. To provide linguistic insights into the possible sources of the small DIF effect, this study compared a gender-balanced corpus of 826 writing samples matched by test takers’ performance on the reading and listening components of the test. Four groups of linguistic features that correspond to the rating dimensions, and thus partially represent the writing construct were analyzed. They include (1) sentiment and social cognition, (2) cohesion, (3) syntactic features, and (4) lexical features. After initial screening, 123 linguistic features, all of which were correlated with the writing scores, were retained for gender comparison. Among these selected features, female test takers’ writing samples scored higher on six of them with small effect sizes in the categories of cohesion and syntactic features. Three of the six features were positively correlated with higher writing scores, while the other three were negative. These results are largely consistent with previous findings of gender differences in language use. Additionally, the small differences in the language features of the writing samples (in terms of the small number of features that differ between genders and the small effect size of the observed differences) are consistent with the previous DIF results, both suggesting that the effect of gender differences on the writing scores is likely to be very small. In sum, the corpus-based findings provide linguistic insights into the gender-related language differences and their potential consequences in a testing context. These findings are meaningful for furthering our understanding of the small gender DIF effect identified through statistical analysis, which lends support to the validity of writing scores.
- 4 October 2019
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 80 (3) , 476-498
A Propensity Score Method for Investigating Differential Item Functioning in Performance Assessment
This study introduces a novel differential item functioning (DIF) method based on propensity score matching that tackles two challenges in analyzing performance assessment data, that is, continuous task scores and lack of a reliable internal variable as a proxy for ability or aptitude. The proposed DIF method consists of two main stages. First, propensity score matching is used to eliminate preexisting group differences before the test, ideally creating equivalent groups as in a randomized experimental study. Then, linear mixed effects models are adopted to perform DIF analysis based on the matched data set. We demonstrate this propensity DIF method using a high-stakes functional English language proficiency test. DIF due to education was investigated in the writing component, which consists of two continuously scored performance-based tasks. Although the proposed method is demonstrated in the context of language testing, it can be applied to other types of performance assessments.
- 4 October 2019
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 80 (3) , 476-498
A Propensity Score Method for Investigating Differential Item Functioning in Performance Assessment
This study introduces a novel differential item functioning (DIF) method based on propensity score matching that tackles two challenges in analyzing performance assessment data, that is, continuous task scores and lack of a reliable internal variable as a proxy for ability or aptitude. The proposed DIF method consists of two main stages. First, propensity score matching is used to eliminate preexisting group differences before the test, ideally creating equivalent groups as in a randomized experimental study. Then, linear mixed effects models are adopted to perform DIF analysis based on the matched data set. We demonstrate this propensity DIF method using a high-stakes functional English language proficiency test. DIF due to education was investigated in the writing component, which consists of two continuously scored performance-based tasks. Although the proposed method is demonstrated in the context of language testing, it can be applied to other types of performance assessments.
- 5 July 2019
- Published by SAGE Publications in Journal of Psychoeducational Assessment
- Vol. 38 (3) , 389-401
Linking Test-Taking Process to Performance Through Mixed-Effects Regression Models: A Response Process–Based Validation Study
Answering the call for response process–based validation, this study shows how researchers can evaluate validity evidence of test scores based on the mental “processes” test-takers use, rather than based on correlations with other “outcome” measures. The proposed methods for process-based validation studies are demonstrated using a sample of 189 adults who took two listening comprehension tasks. Immediately after completing each task, the test-takers filled out a 10-item survey to reflect on the mental processes involved in reaching their answers. These 10 process variables attempted to capture five desired and five undesired response processes in answering multiple-choice listening comprehension questions. We investigated the relationships between these process variables and the binary outcome of item score (correct vs. incorrect) using mixed-effects logistic regression models, and showed how the results could provide validity evidence (or lack thereof). By doing so, we offer an alternative approach to study response process and test performance and encourage more process-based validation studies.
- 5 July 2019
- Published by SAGE Publications in Journal of Psychoeducational Assessment
- Vol. 38 (3) , 389-401
Linking Test-Taking Process to Performance Through Mixed-Effects Regression Models: A Response Process–Based Validation Study
Answering the call for response process–based validation, this study shows how researchers can evaluate validity evidence of test scores based on the mental “processes” test-takers use, rather than based on correlations with other “outcome” measures. The proposed methods for process-based validation studies are demonstrated using a sample of 189 adults who took two listening comprehension tasks. Immediately after completing each task, the test-takers filled out a 10-item survey to reflect on the mental processes involved in reaching their answers. These 10 process variables attempted to capture five desired and five undesired response processes in answering multiple-choice listening comprehension questions. We investigated the relationships between these process variables and the binary outcome of item score (correct vs. incorrect) using mixed-effects logistic regression models, and showed how the results could provide validity evidence (or lack thereof). By doing so, we offer an alternative approach to study response process and test performance and encourage more process-based validation studies.
- 8 February 2019
- Published by Springer Nature
- p. 137-153
Benign: Volvulus and Diverticulitis