Seung Ma Choi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Seung Ma Choi has a total of 55 co-authors. Here are the top co-authors by number of shared publications:
- Sangdon Lim
The University of Texas at Austin
9 shared publications - Sooyong Lee
Research Institute for Aerospace Engineering and Technology, Korea Aerospace University, Goyang-si 10540, Gyeonggi-do, Republic of Korea
4 shared publications - Wim J. van der Linden
University of Twente
3 shared publications - Aaron J. Boulton
Center for Health Assessment Research and Translation, University of Delaware, Newark, USA
2 shared publications - Callie E. Tyner
Center for Health Assessment Research and Translation, University of Delaware, Newark, USA
2 shared publications - David Ring
The University of Texas at Austin
2 shared publications
Latest Publications
- Open Access
- 4 July 2026
- Published by Springer Nature in Quality of Life Research
- Vol. 35 (8)
A methodological overview of new measure development for the Limb Injury Measurement Battery for Quality of Life (LIMB-QOL)
Purpose: To describe the data analytic strategy used to develop new quality-of-life measures for the Limb Injury Measurement Battery for Quality of Life (LIMB-QOL). Methods: Several item pools were created and administered to a large sample of individuals with a history of major extremity injury or limb loss (n = 603). Item analyses adhered to modern psychometric standards (e.g., PROMIS®, COSMIN) and aimed to create several item response theory-based (IRT) item banks based on the graded response model. Items were removed iteratively based on pre-defined criteria and IRT model assumptions were met for the final item pools (monotonicity, unidimensionality, local item independence); differential item functioning, test–retest reliability, and convergent validity were then evaluated. Computer adaptive test and short-form versions of final item banks were created and examined using data simulation. Results: Item analyses led to the development of 8 new item banks and two fixed-length scales. These 10 new LIMB-QOL measures demonstrated initial evidence of reliability (α range = 0.94–0.98, test–retest ICC range: 0.68–0.91) and convergent validity for use in individuals with a history of major extremity injury or limb loss, and abbreviated formats of the full item banks exhibited comparable performance. Conclusion: The new LIMB-QOL measures demonstrated strong psychometric properties and can be used to collect patient-reported assessments of quality of life following major extremity injury and limb loss. The analytic strategy described herein exemplifies how the PROMIS methodology can be utilized to design IRT-based patient-reported outcome measures to fill measurement gaps for specific clinical populations.
- 4 July 2026
- Published by Springer Nature in Quality of Life Research
- Vol. 35 (8)
Open AccessA methodological overview of new measure development for the Limb Injury Measurement Battery for Quality of Life (LIMB-QOL)
Purpose: To describe the data analytic strategy used to develop new quality-of-life measures for the Limb Injury Measurement Battery for Quality of Life (LIMB-QOL). Methods: Several item pools were created and administered to a large sample of individuals with a history of major extremity injury or limb loss (n = 603). Item analyses adhered to modern psychometric standards (e.g., PROMIS®, COSMIN) and aimed to create several item response theory-based (IRT) item banks based on the graded response model. Items were removed iteratively based on pre-defined criteria and IRT model assumptions were met for the final item pools (monotonicity, unidimensionality, local item independence); differential item functioning, test–retest reliability, and convergent validity were then evaluated. Computer adaptive test and short-form versions of final item banks were created and examined using data simulation. Results: Item analyses led to the development of 8 new item banks and two fixed-length scales. These 10 new LIMB-QOL measures demonstrated initial evidence of reliability (α range = 0.94–0.98, test–retest ICC range: 0.68–0.91) and convergent validity for use in individuals with a history of major extremity injury or limb loss, and abbreviated formats of the full item banks exhibited comparable performance. Conclusion: The new LIMB-QOL measures demonstrated strong psychometric properties and can be used to collect patient-reported assessments of quality of life following major extremity injury and limb loss. The analytic strategy described herein exemplifies how the PROMIS methodology can be utilized to design IRT-based patient-reported outcome measures to fill measurement gaps for specific clinical populations.
- Open Access
- 4 July 2026
- Published by Springer Nature in Quality of Life Research
- Vol. 35 (8) , 224
Overview of the limb injury measurement battery for quality of life (LIMB-QOL)
Purpose: The Limb Injury Measurement Battery for Quality of Life (LIMB-QOL) was developed to comprehensively assess patient reported outcomes (PRO) for research and clinical use with individuals who have sustained major extremity trauma and limb loss. Methods: A mixed-methods approach was used that included qualitative focus groups with civilians, service members, and clinicians as well as quantitative PRO data collection with individuals with a history of major traumatic limb injury or limb-loss due to such trauma or sudden-onset illness. Individuals with dysvascular and other chronic disease causes were excluded. Qualitative data were collected via focus groups ( n = 56 individuals with major extremity trauma; n = 34 clinicians) and cognitive debriefing interviews ( n = 41 individuals with major extremity trauma). Quantitative data were obtained through structured phone interviews with a large sample representing the target limb trauma and limb loss population ( n = 603). Newly developed item banks were calibrated using graded response model item response theory analysis. Results: LIMB-QOL includes 25 measurement scales including 10 new, targeted item banks developed with and for individuals with major limb trauma and 15 existing PRO item banks measuring salient physical, emotional, and social aspects of health-related QOL. Computer adaptive tests and short forms were programmed into the Assessment Center application programming interface and are now available through REDCap and other platforms. Conclusion: LIMB-QOL provides a comprehensive and standardized system of PRO assessments for individuals with amputation or limb preservation due to sudden-onset limb injury. Research and clinical applications are discussed.
- 4 July 2026
- Published by Springer Nature in Quality of Life Research
- Vol. 35 (8) , 224
Open AccessOverview of the limb injury measurement battery for quality of life (LIMB-QOL)
Purpose: The Limb Injury Measurement Battery for Quality of Life (LIMB-QOL) was developed to comprehensively assess patient reported outcomes (PRO) for research and clinical use with individuals who have sustained major extremity trauma and limb loss. Methods: A mixed-methods approach was used that included qualitative focus groups with civilians, service members, and clinicians as well as quantitative PRO data collection with individuals with a history of major traumatic limb injury or limb-loss due to such trauma or sudden-onset illness. Individuals with dysvascular and other chronic disease causes were excluded. Qualitative data were collected via focus groups ( n = 56 individuals with major extremity trauma; n = 34 clinicians) and cognitive debriefing interviews ( n = 41 individuals with major extremity trauma). Quantitative data were obtained through structured phone interviews with a large sample representing the target limb trauma and limb loss population ( n = 603). Newly developed item banks were calibrated using graded response model item response theory analysis. Results: LIMB-QOL includes 25 measurement scales including 10 new, targeted item banks developed with and for individuals with major limb trauma and 15 existing PRO item banks measuring salient physical, emotional, and social aspects of health-related QOL. Computer adaptive tests and short forms were programmed into the Assessment Center application programming interface and are now available through REDCap and other platforms. Conclusion: LIMB-QOL provides a comprehensive and standardized system of PRO assessments for individuals with amputation or limb preservation due to sudden-onset limb injury. Research and clinical applications are discussed.
- Open Access
- 24 November 2025
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 50 (4-5) , 173-193
Anchor Detection Strategy in Moderated Non-Linear Factor Analysis for Differential Item Functioning (DIF)
Ensuring measurement invariance is crucial for fair psychological and educational assessments, particularly in detecting Differential Item Functioning (DIF). Moderated Non-linear Factor Analysis (MNLFA) provides a flexible framework for detecting DIF by modeling item parameters as functions of observed covariates. However, a significant challenge in MNLFA-based DIF detection is anchor item selection, as improperly chosen anchors can bias results. This study proposes a refined constrained-baseline anchor detection approach utilizing information criteria (IC) for model selection. The proposed three-step procedure sequentially identifies potential DIF items through the Bayesian Information Criterion (BIC) and Weighted Information Criterion (WIC), followed by DIF-free anchor items using the Akaike Information Criterion (AIC). The method’s effectiveness in controlling Type I error rates while maintaining statistical power is evaluated through simulation studies and empirical data analysis. Comparisons with regularization approaches demonstrate the proposed method’s accuracy and computational efficiency.
- 24 November 2025
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 50 (4-5) , 173-193
Open AccessAnchor Detection Strategy in Moderated Non-Linear Factor Analysis for Differential Item Functioning (DIF)
Ensuring measurement invariance is crucial for fair psychological and educational assessments, particularly in detecting Differential Item Functioning (DIF). Moderated Non-linear Factor Analysis (MNLFA) provides a flexible framework for detecting DIF by modeling item parameters as functions of observed covariates. However, a significant challenge in MNLFA-based DIF detection is anchor item selection, as improperly chosen anchors can bias results. This study proposes a refined constrained-baseline anchor detection approach utilizing information criteria (IC) for model selection. The proposed three-step procedure sequentially identifies potential DIF items through the Bayesian Information Criterion (BIC) and Weighted Information Criterion (WIC), followed by DIF-free anchor items using the Akaike Information Criterion (AIC). The method’s effectiveness in controlling Type I error rates while maintaining statistical power is evaluated through simulation studies and empirical data analysis. Comparisons with regularization approaches demonstrate the proposed method’s accuracy and computational efficiency.
- 19 March 2025
- Published by SPIE-Intl Soc Optical Eng in Journal of Medical Imaging
- Vol. 12 (05) , 1-31
Breast cancer survivors’ perceptual map of breast reconstruction appearance outcomes
PurposeIt is often hard for patients to articulate their expectations about breast reconstruction appearance outcomes to their providers. Our overarching goal is to develop a tool to help patients visually express what they expect to look like after reconstruction. We aim to comprehensively understand how breast cancer survivors perceive diverse breast appearance states by mapping them onto a low-dimensional Euclidean space, which simplifies the complex information about perceptual similarity relationships into a more interpretable form.ApproachWe recruited breast cancer survivors and conducted observer experiments to assess the visual similarities among clinical photographs depicting a range of appearances of the torso relevant to breast reconstruction. Then, we developed a perceptual map to illuminate how breast cancer survivors perceive and distinguish among these appearance states.ResultsWe sampled 100 photographs as stimuli and recruited 34 breast cancer survivors locally. The resulting perceptual map, constructed in two dimensions, offers valuable insights into factors influencing breast cancer survivors’ perceptions of breast reconstruction outcomes. Our findings highlight specific aspects, such as the number of nipples, symmetry, ptosis, scars, and breast shape, that emerge as particularly noteworthy for breast cancer survivors.ConclusionsAnalysis of the perceptual map identified factors associated with breast cancer survivors’ perceptions of breast appearance states that should be emphasized in the appearance consultation process. The perceptual map could be used to assist patients in visually expressing what they expect to look like. Our study lays the groundwork for evaluating interventions intended to help patients form realistic expectations.
- 19 March 2025
- Published by SPIE-Intl Soc Optical Eng in Journal of Medical Imaging
- Vol. 12 (05) , 1-31
Breast cancer survivors’ perceptual map of breast reconstruction appearance outcomes
PurposeIt is often hard for patients to articulate their expectations about breast reconstruction appearance outcomes to their providers. Our overarching goal is to develop a tool to help patients visually express what they expect to look like after reconstruction. We aim to comprehensively understand how breast cancer survivors perceive diverse breast appearance states by mapping them onto a low-dimensional Euclidean space, which simplifies the complex information about perceptual similarity relationships into a more interpretable form.ApproachWe recruited breast cancer survivors and conducted observer experiments to assess the visual similarities among clinical photographs depicting a range of appearances of the torso relevant to breast reconstruction. Then, we developed a perceptual map to illuminate how breast cancer survivors perceive and distinguish among these appearance states.ResultsWe sampled 100 photographs as stimuli and recruited 34 breast cancer survivors locally. The resulting perceptual map, constructed in two dimensions, offers valuable insights into factors influencing breast cancer survivors’ perceptions of breast reconstruction outcomes. Our findings highlight specific aspects, such as the number of nipples, symmetry, ptosis, scars, and breast shape, that emerge as particularly noteworthy for breast cancer survivors.ConclusionsAnalysis of the perceptual map identified factors associated with breast cancer survivors’ perceptions of breast appearance states that should be emphasized in the appearance consultation process. The perceptual map could be used to assist patients in visually expressing what they expect to look like. Our study lays the groundwork for evaluating interventions intended to help patients form realistic expectations.
- Open Access
- 14 February 2025
- Published by Springer Nature in Archives of Women's Mental Health
- Vol. 28 (5) , 1-10
Computer adaptive testing strategies for the Edinburgh Postnatal Depression Scale (EPDS)
Purpose: Perinatal mood and anxiety disorders (PMADs) include depressive and anxiety disorders during pregnancy or postpartum and can have significant consequences for the parent, child, and family. When severe, these conditions can lead to suicide. Despite numerous policy efforts to improve screening, education, and referral structures, disparities in PMAD diagnosis and treatment still exists, particularly among racial and ethnic minorities. Computer Adaptive Testing (CAT) has been shown to improve the efficiency of screening by significantly reducing test length. This study evaluates whether applying CAT to the Edinburgh Postnatal Depression Scale (EPDS) maintains diagnostic accuracy while ensuring these methods do not exacerbate racial disparities in PMAD screening outcomes. Methods: Using real data simulation, we assessed three CAT-based short-form versions of the EPDS, derived from one-, two-, and three-factor item response theory models. We evaluated their diagnostic precision and examined potential racial disparities in false negative rates compared to the full-length EPDS. Results: We demonstrate that estimated scores from three short versions of the EPDS administered through CAT—assuming one, two, and three-factor item response theory models—are more highly correlated with the full-length EPDS measure traditionally used to make clinical decisions (r’s between 0.96 and 0.97) than the major depressive disorder subtest (CAT-MDD) from CAT-Mental Health (CAT-MH®) (r =.82), as previously reported. Importantly, the false negative rates of the CAT-implied diagnoses did not significantly vary between racial groups, indicating no evidence of racial bias in diagnostic accuracy. Conclusion: The CAT-based versions of the EPDS offers a promising solution for improving the efficiency of PMAD screening without sacrificing diagnostic precision or exacerbating racial groups. By reducing evaluation time, these tools could facilitate more widespread and equitable screening, enabling earlier diagnosis and treatment of PMADs across diverse populations.
- 14 February 2025
- Published by Springer Nature in Archives of Women's Mental Health
- Vol. 28 (5) , 1-10
Open AccessComputer adaptive testing strategies for the Edinburgh Postnatal Depression Scale (EPDS)
Purpose: Perinatal mood and anxiety disorders (PMADs) include depressive and anxiety disorders during pregnancy or postpartum and can have significant consequences for the parent, child, and family. When severe, these conditions can lead to suicide. Despite numerous policy efforts to improve screening, education, and referral structures, disparities in PMAD diagnosis and treatment still exists, particularly among racial and ethnic minorities. Computer Adaptive Testing (CAT) has been shown to improve the efficiency of screening by significantly reducing test length. This study evaluates whether applying CAT to the Edinburgh Postnatal Depression Scale (EPDS) maintains diagnostic accuracy while ensuring these methods do not exacerbate racial disparities in PMAD screening outcomes. Methods: Using real data simulation, we assessed three CAT-based short-form versions of the EPDS, derived from one-, two-, and three-factor item response theory models. We evaluated their diagnostic precision and examined potential racial disparities in false negative rates compared to the full-length EPDS. Results: We demonstrate that estimated scores from three short versions of the EPDS administered through CAT—assuming one, two, and three-factor item response theory models—are more highly correlated with the full-length EPDS measure traditionally used to make clinical decisions (r’s between 0.96 and 0.97) than the major depressive disorder subtest (CAT-MDD) from CAT-Mental Health (CAT-MH®) (r =.82), as previously reported. Importantly, the false negative rates of the CAT-implied diagnoses did not significantly vary between racial groups, indicating no evidence of racial bias in diagnostic accuracy. Conclusion: The CAT-based versions of the EPDS offers a promising solution for improving the efficiency of PMAD screening without sacrificing diagnostic precision or exacerbating racial groups. By reducing evaluation time, these tools could facilitate more widespread and equitable screening, enabling earlier diagnosis and treatment of PMADs across diverse populations.
- 5 February 2025
- Published by Wolters Kluwer Health in Clinical Orthopaedics and Related Research
- Vol. 483 (4) , 647-663
A New Measure of Quantified Social Health Is Associated With Levels of Discomfort, Capability, and Mental and General Health Among Patients Seeking Musculoskeletal Specialty Care
Background: A better understanding of the correlation between social health and mindsets, comfort, and capability could aid the design of individualized care models. However, currently available social health checklists are relatively lengthy, burdensome, and designed for descriptive screening purposes rather than quantitative assessment for clinical research, patient monitoring, or quality improvement. Alternatives such as area deprivation index are prone to overgeneralization, lack depth in regard to personal circumstances, and evolve rapidly with gentrification. To fill this void, we aimed to identify the underlying themes of social health and develop a new, personalized and quantitative social health measure. Questions/purposes: (1) What underlying themes of social health (factors) among a subset of items derived from available legacy checklists and questionnaires can be identified and quantified using a brief social health measure? (2) How much of the variation in levels of discomfort, capability, general health, feelings of distress, and unhelpful thoughts regarding symptoms is accounted for by quantified social health? Methods: In this two-stage, cross-sectional study among people seeking musculoskeletal specialty care in an urban area in the United States, all English and Spanish literate adults (ages 18 to 89 years) were invited to participate in two separate cohorts to help develop a provisional new measure of quantified social health. In a first stage (December 2021 to August 2022) , 291 patients rated a subset of items derived from commonly used social health checklists and questionnaires (Tool for Health and Resilience in Vulnerable Environments [THRIVE]; Protocol for Responding to and Assessing Patient Assets, Risks and Experiences [PRAPARE]; and Accountable Health Communities Health-Related Social Needs Screening Tool [HRSN]), of whom 95% (275 of 291; 57% women; mean ± SD age 49 ± 16 years; 51% White, 33% Hispanic; 21% Spanish speaking; 38% completed high school or less) completed all items required to perform factor analysis and were included. Given that so few patients decline participation (estimated at < 5%), we did not track them. We then randomly parsed participants into (1) a learning cohort (69% [189 of 275]) used to identify underlying themes of social health and develop a new measure of quantified social health using exploratory and confirmatory factor analysis (CFA), and (2) a validation cohort (31% [86 of 275]) used to test and internally validate the findings on data not used in its development. During the validation process, we found inconsistencies in the correlations of quantified social health with levels of discomfort and capability between the learning and validation cohort that could not be resolved or explained despite various sensitivity analyses. We therefore identified an additional cohort of 356 eligible patients (February 2023 to June 2023) to complete a new extended subset of items directed at financial security and social support (5 items from the initial stage and 11 new items derived from the Interpersonal Support Evaluation List, Financial Well-Being Scale, Multidimensional Scale of Perceived Social Support, Medical Outcomes Study Social Support Survey, and 6-item Social Support Questionnaire, and “I have to work multiple jobs in order to finance my life” was self-created), of whom 95% (338 of 356; 53% women; mean ± SD age 48 ± 16 years; 38% White, 48% Hispanic; 31% Spanish speaking; 47% completed high school or less) completed all items required to perform factor analysis and were included. We repeated factor analysis to identify the underlying themes of social health and then applied item response theory–based graded response modeling to identify the items that were best able to measure differences in social health (high item discrimination) with the lowest possible floor and ceiling effects (proportion of participants with lowest or highest possible score, respectively; a range of different item difficulties). We also assessed the CFA factor loadings (correlation of an individual item with the identified factor) and modification indices (parameters that suggest whether specific changes to the model would improve model fit appreciably). We then iteratively removed items based on low factor loadings (< 0.4, generally regarded as threshold for items to be considered stable) and high modification indices until model fit in CFA was acceptable (root mean square of error approximation [RMSEA] < 0.05). We then assessed local dependencies among the remaining items (strong relationships between items unrelated to the underlying factor) using Yen Q3 and aimed to combine only items with local dependencies of < 0.25. Because we exhausted our set of items, we were not able to address all local dependencies. Among the remaining items, we then repeated CFA to assess model fit (RMSEA) and used Cronbach alpha to assess internal consistency (the extent to which different subsets of the included items would provide the same measurement outcomes). We performed a differential item functioning analysis to assess whether certain items are rated discordantly based on differences in self-reported age, gender, race, or level of education, which can introduce bias. Last, we assessed the correlations of the new quantified social health measure with various self-reported sociodemographic characteristics (external validity) as well as level of discomfort, capability, general health, and mental health (clinical relevance) using bivariate and multivariable linear regression analyses. Results: We identified two factors representing financial security (11 items) and social support (5 items). After removing problematic items based on our prespecified protocol, we selected 5 items to address financial security (including “I am concerned that the money I have or will save won’t last”) and 4 items to address social support (including “There is a special person who is around when I am in need”). The selected items of the new quantified social health measure (Social Health Scale [SHS]) displayed good model fit in CFA (RMSEA 0.046, confirming adequate factor structure) and good internal consistency (Cronbach α = 0.80 to 0.84), although there were some remaining local dependencies that could not be resolved by removing items because we exhausted our set of items. We found that more disadvantaged quantitative social health was moderately associated with various sociodemographic characteristics (self-reported Black race [regression coefficient (RC) 2.6 (95% confidence interval [CI] 0.29 to 4.9)], divorced [RC 2.5 (95% CI 0.23 to 4.8)], unemployed [RC 1.7 (95% CI 0.023 to 3.4)], uninsured [RC 3.5 (95% CI 0.33 to 6.7)], and earning less than USD 75,000 per year [RC 2.7 (95% CI 0.020 to 5.4) to 6.8 (95% CI 4.3 to 9.3)]), slightly with higher levels of discomfort (RC 0.055 [95% CI 0.16 to 0.093]), slightly with lower levels of capability (RC -0.19 [95% CI -0.34 to -0.035]), slightly with worse general health (RC 0.13 [95% CI 0.069 to 0.18]), moderately with higher levels of unhelpful thoughts (RC 0.17 [95% CI 0.13 to 0.22]), and moderately with greater feelings of distress (RC 0.23 [95% CI 0.19 to 0.28]). Conclusion: A quantitative measure of social health with domains of financial security and social support had acceptable psychometric properties and seems clinically relevant given the associations with levels of discomfort, capability, and general health. It is important to mention that people with disadvantaged social health should not be further disadvantaged by using a quantitative measure of social health to screen or cherry pick in contexts of incentivized or mandated reporting, which could worsen inequities in access and care. Rather, one should consider disadvantaged social health and its associated stressors as one of several previously less considered and potentially modifiable aspects of comprehensive musculoskeletal health. Clinical Relevance: A personalized, quantitative measure of social health would be useful to better capture and understand the role of social health in comprehensive musculoskeletal specialty care. The SHS can be used to measure the distinct contribution of social health to various aspects of musculoskeletal health to inform development of personalized, whole-person care pathways. Clinicians may also use the SHS to identify and monitor patients with disadvantaged social circumstances. This line of inquiry may benefit from additional research including a larger number of items focused on a broader range of social health to further develop the SHS.
- 5 February 2025
- Published by Wolters Kluwer Health in Clinical Orthopaedics and Related Research
- Vol. 483 (4) , 647-663
A New Measure of Quantified Social Health Is Associated With Levels of Discomfort, Capability, and Mental and General Health Among Patients Seeking Musculoskeletal Specialty Care
Background: A better understanding of the correlation between social health and mindsets, comfort, and capability could aid the design of individualized care models. However, currently available social health checklists are relatively lengthy, burdensome, and designed for descriptive screening purposes rather than quantitative assessment for clinical research, patient monitoring, or quality improvement. Alternatives such as area deprivation index are prone to overgeneralization, lack depth in regard to personal circumstances, and evolve rapidly with gentrification. To fill this void, we aimed to identify the underlying themes of social health and develop a new, personalized and quantitative social health measure. Questions/purposes: (1) What underlying themes of social health (factors) among a subset of items derived from available legacy checklists and questionnaires can be identified and quantified using a brief social health measure? (2) How much of the variation in levels of discomfort, capability, general health, feelings of distress, and unhelpful thoughts regarding symptoms is accounted for by quantified social health? Methods: In this two-stage, cross-sectional study among people seeking musculoskeletal specialty care in an urban area in the United States, all English and Spanish literate adults (ages 18 to 89 years) were invited to participate in two separate cohorts to help develop a provisional new measure of quantified social health. In a first stage (December 2021 to August 2022) , 291 patients rated a subset of items derived from commonly used social health checklists and questionnaires (Tool for Health and Resilience in Vulnerable Environments [THRIVE]; Protocol for Responding to and Assessing Patient Assets, Risks and Experiences [PRAPARE]; and Accountable Health Communities Health-Related Social Needs Screening Tool [HRSN]), of whom 95% (275 of 291; 57% women; mean ± SD age 49 ± 16 years; 51% White, 33% Hispanic; 21% Spanish speaking; 38% completed high school or less) completed all items required to perform factor analysis and were included. Given that so few patients decline participation (estimated at < 5%), we did not track them. We then randomly parsed participants into (1) a learning cohort (69% [189 of 275]) used to identify underlying themes of social health and develop a new measure of quantified social health using exploratory and confirmatory factor analysis (CFA), and (2) a validation cohort (31% [86 of 275]) used to test and internally validate the findings on data not used in its development. During the validation process, we found inconsistencies in the correlations of quantified social health with levels of discomfort and capability between the learning and validation cohort that could not be resolved or explained despite various sensitivity analyses. We therefore identified an additional cohort of 356 eligible patients (February 2023 to June 2023) to complete a new extended subset of items directed at financial security and social support (5 items from the initial stage and 11 new items derived from the Interpersonal Support Evaluation List, Financial Well-Being Scale, Multidimensional Scale of Perceived Social Support, Medical Outcomes Study Social Support Survey, and 6-item Social Support Questionnaire, and “I have to work multiple jobs in order to finance my life” was self-created), of whom 95% (338 of 356; 53% women; mean ± SD age 48 ± 16 years; 38% White, 48% Hispanic; 31% Spanish speaking; 47% completed high school or less) completed all items required to perform factor analysis and were included. We repeated factor analysis to identify the underlying themes of social health and then applied item response theory–based graded response modeling to identify the items that were best able to measure differences in social health (high item discrimination) with the lowest possible floor and ceiling effects (proportion of participants with lowest or highest possible score, respectively; a range of different item difficulties). We also assessed the CFA factor loadings (correlation of an individual item with the identified factor) and modification indices (parameters that suggest whether specific changes to the model would improve model fit appreciably). We then iteratively removed items based on low factor loadings (< 0.4, generally regarded as threshold for items to be considered stable) and high modification indices until model fit in CFA was acceptable (root mean square of error approximation [RMSEA] < 0.05). We then assessed local dependencies among the remaining items (strong relationships between items unrelated to the underlying factor) using Yen Q3 and aimed to combine only items with local dependencies of < 0.25. Because we exhausted our set of items, we were not able to address all local dependencies. Among the remaining items, we then repeated CFA to assess model fit (RMSEA) and used Cronbach alpha to assess internal consistency (the extent to which different subsets of the included items would provide the same measurement outcomes). We performed a differential item functioning analysis to assess whether certain items are rated discordantly based on differences in self-reported age, gender, race, or level of education, which can introduce bias. Last, we assessed the correlations of the new quantified social health measure with various self-reported sociodemographic characteristics (external validity) as well as level of discomfort, capability, general health, and mental health (clinical relevance) using bivariate and multivariable linear regression analyses. Results: We identified two factors representing financial security (11 items) and social support (5 items). After removing problematic items based on our prespecified protocol, we selected 5 items to address financial security (including “I am concerned that the money I have or will save won’t last”) and 4 items to address social support (including “There is a special person who is around when I am in need”). The selected items of the new quantified social health measure (Social Health Scale [SHS]) displayed good model fit in CFA (RMSEA 0.046, confirming adequate factor structure) and good internal consistency (Cronbach α = 0.80 to 0.84), although there were some remaining local dependencies that could not be resolved by removing items because we exhausted our set of items. We found that more disadvantaged quantitative social health was moderately associated with various sociodemographic characteristics (self-reported Black race [regression coefficient (RC) 2.6 (95% confidence interval [CI] 0.29 to 4.9)], divorced [RC 2.5 (95% CI 0.23 to 4.8)], unemployed [RC 1.7 (95% CI 0.023 to 3.4)], uninsured [RC 3.5 (95% CI 0.33 to 6.7)], and earning less than USD 75,000 per year [RC 2.7 (95% CI 0.020 to 5.4) to 6.8 (95% CI 4.3 to 9.3)]), slightly with higher levels of discomfort (RC 0.055 [95% CI 0.16 to 0.093]), slightly with lower levels of capability (RC -0.19 [95% CI -0.34 to -0.035]), slightly with worse general health (RC 0.13 [95% CI 0.069 to 0.18]), moderately with higher levels of unhelpful thoughts (RC 0.17 [95% CI 0.13 to 0.22]), and moderately with greater feelings of distress (RC 0.23 [95% CI 0.19 to 0.28]). Conclusion: A quantitative measure of social health with domains of financial security and social support had acceptable psychometric properties and seems clinically relevant given the associations with levels of discomfort, capability, and general health. It is important to mention that people with disadvantaged social health should not be further disadvantaged by using a quantitative measure of social health to screen or cherry pick in contexts of incentivized or mandated reporting, which could worsen inequities in access and care. Rather, one should consider disadvantaged social health and its associated stressors as one of several previously less considered and potentially modifiable aspects of comprehensive musculoskeletal health. Clinical Relevance: A personalized, quantitative measure of social health would be useful to better capture and understand the role of social health in comprehensive musculoskeletal specialty care. The SHS can be used to measure the distinct contribution of social health to various aspects of musculoskeletal health to inform development of personalized, whole-person care pathways. Clinicians may also use the SHS to identify and monitor patients with disadvantaged social circumstances. This line of inquiry may benefit from additional research including a larger number of items focused on a broader range of social health to further develop the SHS.
- 25 October 2024
- Published by Wolters Kluwer Health in Clinical Orthopaedics and Related Research
- Vol. 483 (4) , 693-703
Is It Possible to Develop a Patient-reported Experience Measure With Lower Ceiling Effect?
Background: Patient-reported experience measures (PREMs), such as the Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE) or the Wake Forest Trust in Physician Scale (WTPS), have notable intercorrelation and ceiling effects (the proportion of observations with the highest possible score). Information is lost when high ceiling effects occur as there almost certainly is at least some variation among the patients with the highest score that the measurement tool was unable to measure. Efforts to identify and quantify factors associated with diminished patient experience can benefit from a PREM with more variability and a smaller proportion of highest possible scores (that is, a more limited ceiling effect) than occurs with currently available PREMs. Questions/purposes: In the first stage of a two-stage process, using a cohort of patients seeking musculoskeletal specialty care, we asked: (1) What groupings of items that address a similar aspect of patient experience are present among binary items directed at patient experience and derived from commonly used PREMs? (2) Can a small number of representative items provide a measure with potential for less of a ceiling effect (high item difficulty parameters)? In a second, independent cohort enrolled to assess whether the identified items perform consistently among different cohorts, we asked: (3) Does the new PREM perform differently in terms of item groupings (factor structure), and would different subsets of the included items provide the same measurement results (internal consistency) when items are measured using a 5-point rating scale instead of a binary scale? (4) What are the differences in survey properties (for example, ceiling effects) and correlation between the new PREM and commonly used PREMs? Methods: In two cross-sectional studies among patients seeking musculoskeletal specialty care conducted in 2022 and 2023, all English-speaking and English-reading adults (ages 18 to 89 years) without cognitive deficiency were invited to participate in two consecutive, separate cohorts to help develop (the initial, learning cohort) and internally validate (the second, validation cohort) a provisional new PREM. We identified 218 eligible patients for the initial learning cohort, of whom all completed all measures. Participants had a mean ± SD age of 55 ± 16 years, 60% (130) were women, 45% (99) had private insurance, and most sought care for lower extremity (56% [121]) and nontraumatic conditions (63% [137]). We measured 25 items derived from other commonly used PREMs that address aspects of patient experience in which patients reported whether they agreed or disagreed (binary) with certain statements about their clinician. We performed an exploratory factor analysis and confirmatory factor analysis (CFA) to identify groups of items that measure the same underlying construct related to patient experience. We then applied a two-parameter logistic model based on item response theory to identify the most discriminating items with the most variability (item difficulty) with the aim of reducing the ceiling effect. We also conducted a differential item functioning analysis to assess whether specific items are rated discordantly by specific subgroups of patients, which can introduce bias. We then enrolled 154 eligible patients, of whom 99% (153) completed all required measures, into a validation cohort with similar demographic characteristics. We changed the binary items to 5-point Likert scales to increase the potential for variation in an attempt to further reduce ceiling effects and repeated the CFA. We also measured internal consistency (using Cronbach alpha) and the correlation of the new PREM with other commonly used PREMs using bivariate analyses. Results: We identified three groupings of items in the learning cohort representing “trust in clinician” (13 items), “relationship with clinician” (7 items), and “participation in shared decision-making” (4 items). The “trust in clinician” factor performed best of all three factors and therefore was selected for subsequent analyses. We selected the best-performing items in terms of item difficulty to generate a 7-item short form. We found excellent CFA model fit (the 13-item and 7-item versions both had a root mean square error of approximation [RMSEA] of < 0.001), excellent internal consistency (Cronbach α was 0.94 for the 13-item version and 0.91 for the 7-item version), good item response theory parameters (item difficulty ranging between -0.37 and 0.16 for the 7-item version, with higher values indicating lower ceiling effect), no local dependencies, and no differential item functioning among any of the items. The other two factors were excluded from measure development due to low item response theory parameters (item difficulty ranging between -1.3 and -0.69, indicating higher ceiling effect), multiple local dependencies, and exhausting the number of items without being able to address these issues. The validation cohort confirmed adequate item selection and performance of both the 13-item and 7-item version of the Trust and Experience with Clinicians Scale (TRECS), with good to excellent CFA model fit (RMSEA 0.058 [TRECS-13]; RMSEA 0.016 [TRECS-7]), excellent internal consistency (Cronbach α = 0.96 [TRECS-13]; Cronbach α = 0.92 [TRECS-7]), no differential item functioning and limited ceiling effects (11% [TRECS-13]; 14% [TRECS-7]), and notable correlation with other PREMs such as the JSPPPE (ρ = 0.77) and WTPS (ρ = 0.74). Conclusion: A relatively brief 7-item measure of patient experience focused on trust can eliminate most of the ceiling effects common to PREMs with good psychometric properties. Future studies may externally validate the TRECS in other populations as well as provide population-based T-score conversion tables based on a larger sample size more representative of the...
- 25 October 2024
- Published by Wolters Kluwer Health in Clinical Orthopaedics and Related Research
- Vol. 483 (4) , 693-703
Is It Possible to Develop a Patient-reported Experience Measure With Lower Ceiling Effect?
Background: Patient-reported experience measures (PREMs), such as the Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE) or the Wake Forest Trust in Physician Scale (WTPS), have notable intercorrelation and ceiling effects (the proportion of observations with the highest possible score). Information is lost when high ceiling effects occur as there almost certainly is at least some variation among the patients with the highest score that the measurement tool was unable to measure. Efforts to identify and quantify factors associated with diminished patient experience can benefit from a PREM with more variability and a smaller proportion of highest possible scores (that is, a more limited ceiling effect) than occurs with currently available PREMs. Questions/purposes: In the first stage of a two-stage process, using a cohort of patients seeking musculoskeletal specialty care, we asked: (1) What groupings of items that address a similar aspect of patient experience are present among binary items directed at patient experience and derived from commonly used PREMs? (2) Can a small number of representative items provide a measure with potential for less of a ceiling effect (high item difficulty parameters)? In a second, independent cohort enrolled to assess whether the identified items perform consistently among different cohorts, we asked: (3) Does the new PREM perform differently in terms of item groupings (factor structure), and would different subsets of the included items provide the same measurement results (internal consistency) when items are measured using a 5-point rating scale instead of a binary scale? (4) What are the differences in survey properties (for example, ceiling effects) and correlation between the new PREM and commonly used PREMs? Methods: In two cross-sectional studies among patients seeking musculoskeletal specialty care conducted in 2022 and 2023, all English-speaking and English-reading adults (ages 18 to 89 years) without cognitive deficiency were invited to participate in two consecutive, separate cohorts to help develop (the initial, learning cohort) and internally validate (the second, validation cohort) a provisional new PREM. We identified 218 eligible patients for the initial learning cohort, of whom all completed all measures. Participants had a mean ± SD age of 55 ± 16 years, 60% (130) were women, 45% (99) had private insurance, and most sought care for lower extremity (56% [121]) and nontraumatic conditions (63% [137]). We measured 25 items derived from other commonly used PREMs that address aspects of patient experience in which patients reported whether they agreed or disagreed (binary) with certain statements about their clinician. We performed an exploratory factor analysis and confirmatory factor analysis (CFA) to identify groups of items that measure the same underlying construct related to patient experience. We then applied a two-parameter logistic model based on item response theory to identify the most discriminating items with the most variability (item difficulty) with the aim of reducing the ceiling effect. We also conducted a differential item functioning analysis to assess whether specific items are rated discordantly by specific subgroups of patients, which can introduce bias. We then enrolled 154 eligible patients, of whom 99% (153) completed all required measures, into a validation cohort with similar demographic characteristics. We changed the binary items to 5-point Likert scales to increase the potential for variation in an attempt to further reduce ceiling effects and repeated the CFA. We also measured internal consistency (using Cronbach alpha) and the correlation of the new PREM with other commonly used PREMs using bivariate analyses. Results: We identified three groupings of items in the learning cohort representing “trust in clinician” (13 items), “relationship with clinician” (7 items), and “participation in shared decision-making” (4 items). The “trust in clinician” factor performed best of all three factors and therefore was selected for subsequent analyses. We selected the best-performing items in terms of item difficulty to generate a 7-item short form. We found excellent CFA model fit (the 13-item and 7-item versions both had a root mean square error of approximation [RMSEA] of < 0.001), excellent internal consistency (Cronbach α was 0.94 for the 13-item version and 0.91 for the 7-item version), good item response theory parameters (item difficulty ranging between -0.37 and 0.16 for the 7-item version, with higher values indicating lower ceiling effect), no local dependencies, and no differential item functioning among any of the items. The other two factors were excluded from measure development due to low item response theory parameters (item difficulty ranging between -1.3 and -0.69, indicating higher ceiling effect), multiple local dependencies, and exhausting the number of items without being able to address these issues. The validation cohort confirmed adequate item selection and performance of both the 13-item and 7-item version of the Trust and Experience with Clinicians Scale (TRECS), with good to excellent CFA model fit (RMSEA 0.058 [TRECS-13]; RMSEA 0.016 [TRECS-7]), excellent internal consistency (Cronbach α = 0.96 [TRECS-13]; Cronbach α = 0.92 [TRECS-7]), no differential item functioning and limited ceiling effects (11% [TRECS-13]; 14% [TRECS-7]), and notable correlation with other PREMs such as the JSPPPE (ρ = 0.77) and WTPS (ρ = 0.74). Conclusion: A relatively brief 7-item measure of patient experience focused on trust can eliminate most of the ceiling effects common to PREMs with good psychometric properties. Future studies may externally validate the TRECS in other populations as well as provide population-based T-score conversion tables based on a larger sample size more representative of the...
- 15 March 2024
- Published by Wiley in Journal of Educational Measurement
- Vol. 61 (2) , 303-324
A Bayesian Moderated Nonlinear Factor Analysis Approach for DIF Detection under Violation of the Equal Variance Assumption
Research has shown that multiple‐indicator multiple‐cause (MIMIC) models can result in inflated Type I error rates in detecting differential item functioning (DIF) when the assumption of equal latent variance is violated. This study explains how the violation of the equal variance assumption adversely impacts the detection of nonuniform DIF and how it can be addressed through moderated nonlinear factor analysis (MNLFA) model via Bayesian estimation approach to overcome limitations from the restrictive assumption. The Bayesian MNLFA approach suggested in this study better control Type I errors by freely estimating latent factor variances across different groups. Our experimentation with simulated data demonstrates that the BMNFA models outperform the existing MIMIC models, in terms of Type I error control as well as parameter recovery. The results suggest that the MNLFA models have the potential to be a superior choice to the existing MIMIC models, especially in situations where the assumption of equal latent variance assumption is not likely to hold.
- 15 March 2024
- Published by Wiley in Journal of Educational Measurement
- Vol. 61 (2) , 303-324
A Bayesian Moderated Nonlinear Factor Analysis Approach for DIF Detection under Violation of the Equal Variance Assumption
Research has shown that multiple‐indicator multiple‐cause (MIMIC) models can result in inflated Type I error rates in detecting differential item functioning (DIF) when the assumption of equal latent variance is violated. This study explains how the violation of the equal variance assumption adversely impacts the detection of nonuniform DIF and how it can be addressed through moderated nonlinear factor analysis (MNLFA) model via Bayesian estimation approach to overcome limitations from the restrictive assumption. The Bayesian MNLFA approach suggested in this study better control Type I errors by freely estimating latent factor variances across different groups. Our experimentation with simulated data demonstrates that the BMNFA models outperform the existing MIMIC models, in terms of Type I error control as well as parameter recovery. The results suggest that the MNLFA models have the potential to be a superior choice to the existing MIMIC models, especially in situations where the assumption of equal latent variance assumption is not likely to hold.
- Open Access
- 1 January 2024
- Published by XMLink in Journal of Korean Medical Science
- Vol. 39 (33) , e239
Subtyping of Performance Trajectory During Medical School, Medical Internship, and the First Year of Residency in Training Physicians: A Longitudinal Cohort Study
Yun JY, et al. J Korean Med Sci. 2024 Jul;39:e239. https://doi.org/10.3346/jkms.2024.39.e239
- 1 January 2024
- Published by XMLink in Journal of Korean Medical Science
- Vol. 39 (33) , e239
Open AccessSubtyping of Performance Trajectory During Medical School, Medical Internship, and the First Year of Residency in Training Physicians: A Longitudinal Cohort Study
Yun JY, et al. J Korean Med Sci. 2024 Jul;39:e239. https://doi.org/10.3346/jkms.2024.39.e239
- Open Access
- 11 December 2023
- Published by Springer Nature in Behaviormetrika
- Vol. 51 (1) , 125-156
Item exposure and utilization control methods for optimal test assembly
Exposure control is an important component of computerized adaptive testing. Exposure control is often used to secure test content by preventing item overexposure, and also to balance item utilization by avoiding item underexposure. Content balancing is another important component of computerized adaptive testing. The shadow-test approach to content balancing has many practical merits, including that it ensures complete satisfaction of all content requirements in every assembled test. Several exposure control methods have been developed over the years. The current study describes modifications to existing exposure control methods for use with shadow-test-based adaptive testing, with the objective of achieving bidirectional exposure control. The performances of the modified methods were compared with respect to item exposure and utilization along with ability estimation accuracy. For the purpose of comparison, a Monte Carlo simulation was performed using realistic item pools and content constraints. Implications for real-world applications are discussed.
- 11 December 2023
- Published by Springer Nature in Behaviormetrika
- Vol. 51 (1) , 125-156
Open AccessItem exposure and utilization control methods for optimal test assembly
Exposure control is an important component of computerized adaptive testing. Exposure control is often used to secure test content by preventing item overexposure, and also to balance item utilization by avoiding item underexposure. Content balancing is another important component of computerized adaptive testing. The shadow-test approach to content balancing has many practical merits, including that it ensures complete satisfaction of all content requirements in every assembled test. Several exposure control methods have been developed over the years. The current study describes modifications to existing exposure control methods for use with shadow-test-based adaptive testing, with the objective of achieving bidirectional exposure control. The performances of the modified methods were compared with respect to item exposure and utilization along with ability estimation accuracy. For the purpose of comparison, a Monte Carlo simulation was performed using realistic item pools and content constraints. Implications for real-world applications are discussed.