Cindy J. Nowinski
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Cindy J. Nowinski has a total of 118 co-authors. Here are the top co-authors by number of shared publications:
- Richard C. Gershon
Northwestern University Feinberg School of Medicine, Chicago, IL, USA
9 shared publications - Elizabeth McManus Dworak
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine
6 shared publications - Stephanie Ruth Young
Department of Medical Social Sciences Northwestern University Feinberg School of Medicine Chicago Illinois USA
6 shared publications - David Cella
Northwestern University, Evanston, USA
5 shared publications - Julia Yoshino Benavente
Center for Applied Health Research on Aging, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
5 shared publications - Aaron James Kaat
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine , Chicago, IL
4 shared publications
Latest Publications
- Open Access
- 24 December 2025
- Published by Wiley in Alzheimer's & Dementia
- Vol. 21 (S3) , e099517
Cross validation of novel secondary executive functioning scores in the Mobile Toolbox for monitoring cognitive change
Background: Executive Function (EF) has multiple components (e.g., attention, inhibitory control, etc.) which can be differentially affected by AD/ADRD. Specific EF deficits can be early indicators of cognitive impairment, providing clues about etiology and likelihood of disease progression. The NIH Mobile Toolbox (MTB) includes adapted, self‐administered measures from the NIH Toolbox® (NIHTB) on a smartphone that are brief, sensitive, and easy‐to‐use for tracking cognitive functioning across the lifespan. The primary scores produced for MTB EF measures (Arrow Matching; Shape‐Color Sorting) are rate‐based scores (# correct/time to complete), which have been shown to be reliable and valid (Novack et al., 2024). In the current study, we expand on these measures by investigating a series of secondary scores (anticipation errors, error rate, and median response time for incorrect and correct responses) towards a more comprehensive EF assessment. Method: Participants were administered the MTB in person on study‐provided iOS smartphones and completed the NIHTB measures on study‐provided iPads (Study 1: n = 92, aged M=49.27, SD=17.65). Participants also completed external measures intended to assess convergent validity (e.g., D‐KEFS, WCST) and divergent validity (e.g., PPVT‐5, NIHTB PV). Another group of participants completed the NIHTB in person and MTB remotely using their own Android/iOS smartphone no more than 14 days after the first session (Study 2: n = 1021, aged M=43.97, SD=21.24). Result: Across studies, there was a significant relationship between all novel scores in both tasks and age ( ρ = [.27,.60], p s < .05); older individuals made more anticipation and total errors and were slower to make correct and incorrect responses. Most scores showed significant, expected correlations against concurrent measures (absolute value ρ = [.13,.77], p s < .05). Most scores showed non‐significant, low correlations with the proposed divergent measures ( ρ = [‐.01,.09], p s > .05), except for anticipation and total error scores, which showed significant but weaker negative relationships with vocabulary measures. All scores showed strong internal reliability via split‐half correlations (Study 2; SCS median = .94, 25‐75th percentile=[.93,.94]; AM median = .98, 25‐75th percentile=[.97,.98]). Conclusion: Expanded scores from the MTB EF measures are valid and may be suitable for clinical and pharmaceutical studies, particularly relating to age related cognitive changes. Additional clinical validation is underway, including populations at risk for or diagnosed with cognitive impairment or AD/ADRD.
- 24 December 2025
- Published by Wiley in Alzheimer's & Dementia
- Vol. 21 (S3) , e099517
Open AccessCross validation of novel secondary executive functioning scores in the Mobile Toolbox for monitoring cognitive change
Background: Executive Function (EF) has multiple components (e.g., attention, inhibitory control, etc.) which can be differentially affected by AD/ADRD. Specific EF deficits can be early indicators of cognitive impairment, providing clues about etiology and likelihood of disease progression. The NIH Mobile Toolbox (MTB) includes adapted, self‐administered measures from the NIH Toolbox® (NIHTB) on a smartphone that are brief, sensitive, and easy‐to‐use for tracking cognitive functioning across the lifespan. The primary scores produced for MTB EF measures (Arrow Matching; Shape‐Color Sorting) are rate‐based scores (# correct/time to complete), which have been shown to be reliable and valid (Novack et al., 2024). In the current study, we expand on these measures by investigating a series of secondary scores (anticipation errors, error rate, and median response time for incorrect and correct responses) towards a more comprehensive EF assessment. Method: Participants were administered the MTB in person on study‐provided iOS smartphones and completed the NIHTB measures on study‐provided iPads (Study 1: n = 92, aged M=49.27, SD=17.65). Participants also completed external measures intended to assess convergent validity (e.g., D‐KEFS, WCST) and divergent validity (e.g., PPVT‐5, NIHTB PV). Another group of participants completed the NIHTB in person and MTB remotely using their own Android/iOS smartphone no more than 14 days after the first session (Study 2: n = 1021, aged M=43.97, SD=21.24). Result: Across studies, there was a significant relationship between all novel scores in both tasks and age ( ρ = [.27,.60], p s < .05); older individuals made more anticipation and total errors and were slower to make correct and incorrect responses. Most scores showed significant, expected correlations against concurrent measures (absolute value ρ = [.13,.77], p s < .05). Most scores showed non‐significant, low correlations with the proposed divergent measures ( ρ = [‐.01,.09], p s > .05), except for anticipation and total error scores, which showed significant but weaker negative relationships with vocabulary measures. All scores showed strong internal reliability via split‐half correlations (Study 2; SCS median = .94, 25‐75th percentile=[.93,.94]; AM median = .98, 25‐75th percentile=[.97,.98]). Conclusion: Expanded scores from the MTB EF measures are valid and may be suitable for clinical and pharmaceutical studies, particularly relating to age related cognitive changes. Additional clinical validation is underway, including populations at risk for or diagnosed with cognitive impairment or AD/ADRD.
- Open Access
- 1 October 2025
- Published by Wiley in Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring
- Vol. 17 (4) , e70219
Clinical validation and machine learning optimization of MyCog: A self‐administered cognitive screener for primary care settings
Background: Primary care offers ideal opportunities for early detection of cognitive impairment, yet clinics face significant barriers to routine screening. MyCog is an electronic health record–integrated tablet application self‐administered during a primary care visit designed to overcome barriers to screening. Methods: We compared MyCog performance between 65 adults age 65+ with diagnosed cognitive impairment and 80 cognitively normal adults. Five modeling approaches achieved consensus to select consistently discriminative variables for the final detection algorithm. Performance was primarily assessed via receiver operating characteristic area under the curve (AUC), sensitivity, specificity, and accuracy. Results: All models demonstrated strong diagnostic performance (AUC 0.817 to 0.873). Memory accuracy and executive function efficiency scores were consistently selected as predictors of impairment across models. The final logistic regression achieved AUC 0.890, with sensitivity 0.723 to 0.831, specificity 0.788 to 0.912, and accuracy 0.807 to 0.828 depending on threshold. Discussion: Findings suggest MyCog accurately detects cognitive impairment via a streamlined self‐administered app that efficiently fits into primary care workflows.
- 1 October 2025
- Published by Wiley in Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring
- Vol. 17 (4) , e70219
Open AccessClinical validation and machine learning optimization of MyCog: A self‐administered cognitive screener for primary care settings
Background: Primary care offers ideal opportunities for early detection of cognitive impairment, yet clinics face significant barriers to routine screening. MyCog is an electronic health record–integrated tablet application self‐administered during a primary care visit designed to overcome barriers to screening. Methods: We compared MyCog performance between 65 adults age 65+ with diagnosed cognitive impairment and 80 cognitively normal adults. Five modeling approaches achieved consensus to select consistently discriminative variables for the final detection algorithm. Performance was primarily assessed via receiver operating characteristic area under the curve (AUC), sensitivity, specificity, and accuracy. Results: All models demonstrated strong diagnostic performance (AUC 0.817 to 0.873). Memory accuracy and executive function efficiency scores were consistently selected as predictors of impairment across models. The final logistic regression achieved AUC 0.890, with sensitivity 0.723 to 0.831, specificity 0.788 to 0.912, and accuracy 0.807 to 0.828 depending on threshold. Discussion: Findings suggest MyCog accurately detects cognitive impairment via a streamlined self‐administered app that efficiently fits into primary care workflows.
- Open Access
MyCog Mobile smartphone-based cognitive screening system: a cross-sectional construct validation in a general population sample at multiple research facilities in the USA
Objectives Annual cognitive screening in older adults is essential for early detection of cognitive impairment, yet primary care settings face time constraints that present barriers to routine screening. MyCog Mobile is a self-administered, smartphone-based cognitive screener that has the potential to help overcome these screening obstacles. We compared MyCog Mobile to ‘gold-standard’ measures to support its reliability and validity and examined performance differences between in-person versus remote assessment. Design Cross-sectional convenience sample study comparing MyCog Mobile measures to established cognitive assessments. Setting Five in-person research facility locations in Georgia, Texas, New Jersey, Florida and Arizona. Participants 200 adults aged 65–87 years (M=72.56, SD=5.11) who spoke English. Primary outcome measures Convergent and divergent validity were examined via Spearman’s rho correlations between MyCog Mobile measures (MyPictures, MyFaces, MySorting and MySequences) and established measures (verbal paired associates, digit span, letter-number sequencing, Color-Word Interference Test and Trail Making Test), as well as a confirmatory factor analysis using these same measures. Internal consistency was assessed via Spearman-Brown split-half correlations. Score distributions were compared between in-person and remote administration. Results Internal consistency for all MyCog Mobile measures was above acceptable thresholds, ranging from ρ=0.71 to 0.89. Strong correlations were found between MyCog Mobile subtests and their tablet counterparts (MyPictures-Picture Sequence Memory: ρ=0.52; MySorting-dimensional change card sorting: ρ=0.55). MyPictures and MyFaces demonstrated large correlations with convergent memory measures (verbal paired associates immediate ρ=0.51 and 0.56, respectively) and small correlations with discriminant measures. Confirmatory factor analysis supported a two-factor model with subtests loading onto episodic memory and executive functioning factors as expected (χ²(25)=39.136; Comparative Fit Index (CFI) = 0.965; Tucker–Lewis Index (TLI) = 0.950; Root Mean Square Error of Approximation (RMSEA) = 0.053; Standardized Root Mean Square Residual (SRMR) = 0.040). Remote participants scored significantly higher on the MyPictures task overall (t(249)=2.98, p=0.004), while in-person participants scored higher on the MyFaces First Letter subtest and MySequences. Conclusions MyCog Mobile offers a reliable and valid assessment of executive functioning and episodic memory in older adults. Performance may differ by setting, but further research is needed. These findings support future clinical validation studies to determine the diagnostic accuracy of MyCog Mobile to detect cognitive impairment.
Open AccessMyCog Mobile smartphone-based cognitive screening system: a cross-sectional construct validation in a general population sample at multiple research facilities in the USA
Objectives Annual cognitive screening in older adults is essential for early detection of cognitive impairment, yet primary care settings face time constraints that present barriers to routine screening. MyCog Mobile is a self-administered, smartphone-based cognitive screener that has the potential to help overcome these screening obstacles. We compared MyCog Mobile to ‘gold-standard’ measures to support its reliability and validity and examined performance differences between in-person versus remote assessment. Design Cross-sectional convenience sample study comparing MyCog Mobile measures to established cognitive assessments. Setting Five in-person research facility locations in Georgia, Texas, New Jersey, Florida and Arizona. Participants 200 adults aged 65–87 years (M=72.56, SD=5.11) who spoke English. Primary outcome measures Convergent and divergent validity were examined via Spearman’s rho correlations between MyCog Mobile measures (MyPictures, MyFaces, MySorting and MySequences) and established measures (verbal paired associates, digit span, letter-number sequencing, Color-Word Interference Test and Trail Making Test), as well as a confirmatory factor analysis using these same measures. Internal consistency was assessed via Spearman-Brown split-half correlations. Score distributions were compared between in-person and remote administration. Results Internal consistency for all MyCog Mobile measures was above acceptable thresholds, ranging from ρ=0.71 to 0.89. Strong correlations were found between MyCog Mobile subtests and their tablet counterparts (MyPictures-Picture Sequence Memory: ρ=0.52; MySorting-dimensional change card sorting: ρ=0.55). MyPictures and MyFaces demonstrated large correlations with convergent memory measures (verbal paired associates immediate ρ=0.51 and 0.56, respectively) and small correlations with discriminant measures. Confirmatory factor analysis supported a two-factor model with subtests loading onto episodic memory and executive functioning factors as expected (χ²(25)=39.136; Comparative Fit Index (CFI) = 0.965; Tucker–Lewis Index (TLI) = 0.950; Root Mean Square Error of Approximation (RMSEA) = 0.053; Standardized Root Mean Square Residual (SRMR) = 0.040). Remote participants scored significantly higher on the MyPictures task overall (t(249)=2.98, p=0.004), while in-person participants scored higher on the MyFaces First Letter subtest and MySequences. Conclusions MyCog Mobile offers a reliable and valid assessment of executive functioning and episodic memory in older adults. Performance may differ by setting, but further research is needed. These findings support future clinical validation studies to determine the diagnostic accuracy of MyCog Mobile to detect cognitive impairment.
- 28 July 2025
- Published by Elsevier in Infant Behavior and Development
- Vol. 80, 102110
Development and validation of the NIH Baby Toolbox Social-Emotional Functioning measures
- 28 July 2025
- Published by Elsevier in Infant Behavior and Development
- Vol. 80, 102110
Development and validation of the NIH Baby Toolbox Social-Emotional Functioning measures
- Open Access
Clinical Validation and Machine Learning Optimization of MyCog: A Self-Administered Cognitive Screener for Primary Care Settings
Importance Early detection of cognitive impairment in older adults is crucial for identifying reversible causes, managing symptoms, and enabling future planning; yet primary care settings face significant barriers to implementing effective cognitive screening. Objective To clinically validate an innovated self-administered, tablet-based cognitive screening tool (“MyCog”, adapted from the NIH Toolbox Cognitive Battery) and to develop an optimized scoring algorithm for detecting cognitive impairment in older adults during primary care visits. Design Cross-sectional clinical validation study conducted between summer 2021 and fall 2024, comparing cognitive performance between impaired and non-impaired participants using multiple machine learning approaches to identify optimal predictors of impairment. Setting Academic medical centers and community-based general population recruitment. Participants 248 adults aged 65 and older, including 70 participants with clinically diagnosed cognitive impairment (MCI or dementia), 81 participants confirmed to be cognitively intact from clinical settings, and 97 participants from the general population. Participants were predominantly White (88%) with varied educational backgrounds and a mean age of 76.3 years (SD = 6.9). Main Outcomes and Measures Sensitivity, specificity, and area under the receiver operating characteristic curve (ROC AUC) were the primary metrics used for evaluating model fit in classifying cognitive impairment of known groups from the MyCog Picture Sequence Memory and Dimensional Change Card Sort tasks. Results Across nine machine learning model classes, Picture Sequence Memory exact match score and Dimensional Change Card Sort rate-correct score were identified as the most robust predictors of cognitive impairment. The final optimized model achieved a sensitivity of 82.5% (bootstrapped 95% CI [0.733, 0.913]) and a specificity of 76.8% (bootstrapped 95% CI [0.709, 0.828]) with an ROC AUC of 0.892, comparable to traditional, long form cognitive screening measures while requiring less than 7 minutes to self-administer. Conclusions and Relevance MyCog demonstrates strong diagnostic accuracy for detecting cognitive impairment in older adults with a parsimonious model focusing on episodic memory and executive function. As a brief, self-administered screening tool that can be integrated into electronic health records, MyCog addresses key barriers to cognitive screening in primary care settings, which may help increase screening and early detection rates.
Open AccessClinical Validation and Machine Learning Optimization of MyCog: A Self-Administered Cognitive Screener for Primary Care Settings
Importance Early detection of cognitive impairment in older adults is crucial for identifying reversible causes, managing symptoms, and enabling future planning; yet primary care settings face significant barriers to implementing effective cognitive screening. Objective To clinically validate an innovated self-administered, tablet-based cognitive screening tool (“MyCog”, adapted from the NIH Toolbox Cognitive Battery) and to develop an optimized scoring algorithm for detecting cognitive impairment in older adults during primary care visits. Design Cross-sectional clinical validation study conducted between summer 2021 and fall 2024, comparing cognitive performance between impaired and non-impaired participants using multiple machine learning approaches to identify optimal predictors of impairment. Setting Academic medical centers and community-based general population recruitment. Participants 248 adults aged 65 and older, including 70 participants with clinically diagnosed cognitive impairment (MCI or dementia), 81 participants confirmed to be cognitively intact from clinical settings, and 97 participants from the general population. Participants were predominantly White (88%) with varied educational backgrounds and a mean age of 76.3 years (SD = 6.9). Main Outcomes and Measures Sensitivity, specificity, and area under the receiver operating characteristic curve (ROC AUC) were the primary metrics used for evaluating model fit in classifying cognitive impairment of known groups from the MyCog Picture Sequence Memory and Dimensional Change Card Sort tasks. Results Across nine machine learning model classes, Picture Sequence Memory exact match score and Dimensional Change Card Sort rate-correct score were identified as the most robust predictors of cognitive impairment. The final optimized model achieved a sensitivity of 82.5% (bootstrapped 95% CI [0.733, 0.913]) and a specificity of 76.8% (bootstrapped 95% CI [0.709, 0.828]) with an ROC AUC of 0.892, comparable to traditional, long form cognitive screening measures while requiring less than 7 minutes to self-administer. Conclusions and Relevance MyCog demonstrates strong diagnostic accuracy for detecting cognitive impairment in older adults with a parsimonious model focusing on episodic memory and executive function. As a brief, self-administered screening tool that can be integrated into electronic health records, MyCog addresses key barriers to cognitive screening in primary care settings, which may help increase screening and early detection rates.
- Open Access
- 21 March 2025
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 9 (1) , e60215
Customizing Computerized Adaptive Test Stopping Rules for Clinical Settings Using the Negative Affect Subdomain of the NIH Toolbox Emotion Battery: Simulation Study
Background: Patient-reported outcome measures are crucial for informed medical decisions and evaluating treatments. However, they can be burdensome for patients and sometimes lack the reliability clinicians need for clear clinical interpretations. Objective: We aimed to assess the extent to which applying alternative stopping rules can increase reliability for clinical use while minimizing the burden of computerized adaptive tests (CATs). Methods: CAT simulations were conducted on 3 adult item banks in the NIH Toolbox for Assessment of Neurological and Behavioral Function Emotion Battery; the item banks were in the Negative Affect subdomain (ie, Anger Affect, Fear Affect, and Sadness) and contained at least 8 items. In the originally applied NIH Toolbox CAT stopping rules, the CAT was stopped if the score SE reached <0.3 before 12 items were administered. We first contrasted this with a SE-change rule in a planned simulation analysis. We then contrasted the original rules with fixed-length CATs (4‐12 items), a reduction of the maximum number of items to 8, and other modifications in post hoc analyses. Burden was measured by the number of items administered per simulation, precision by the percentage of assessments yielding reliability cutoffs (0.85, 0.90, and 0.95), and accurate score recovery by the root mean squared error between the generating θ and the CAT-estimated “expected a posteriori”–based θ. Results: In general, relative to the original rules, the alternative stopping rules slightly decreased burden while also increasing the proportion of assessments achieving high reliability for the adult banks; however, the SE-change rule and fixed-length CATs with 8 or fewer items also notably increased assessments yielding reliability <0.85. Among the alternative rules explored, the reduced maximum stopping rule best balanced precision and parsimony, presenting another option beyond the original rules. Conclusions: Our findings demonstrate the challenges in attempting to reduce test burden while also achieving score precision for clinical use. Stopping rules should be modified in accordance with the context of the study population and the purpose of the study.
- 21 March 2025
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 9 (1) , e60215
Open AccessCustomizing Computerized Adaptive Test Stopping Rules for Clinical Settings Using the Negative Affect Subdomain of the NIH Toolbox Emotion Battery: Simulation Study
Background: Patient-reported outcome measures are crucial for informed medical decisions and evaluating treatments. However, they can be burdensome for patients and sometimes lack the reliability clinicians need for clear clinical interpretations. Objective: We aimed to assess the extent to which applying alternative stopping rules can increase reliability for clinical use while minimizing the burden of computerized adaptive tests (CATs). Methods: CAT simulations were conducted on 3 adult item banks in the NIH Toolbox for Assessment of Neurological and Behavioral Function Emotion Battery; the item banks were in the Negative Affect subdomain (ie, Anger Affect, Fear Affect, and Sadness) and contained at least 8 items. In the originally applied NIH Toolbox CAT stopping rules, the CAT was stopped if the score SE reached <0.3 before 12 items were administered. We first contrasted this with a SE-change rule in a planned simulation analysis. We then contrasted the original rules with fixed-length CATs (4‐12 items), a reduction of the maximum number of items to 8, and other modifications in post hoc analyses. Burden was measured by the number of items administered per simulation, precision by the percentage of assessments yielding reliability cutoffs (0.85, 0.90, and 0.95), and accurate score recovery by the root mean squared error between the generating θ and the CAT-estimated “expected a posteriori”–based θ. Results: In general, relative to the original rules, the alternative stopping rules slightly decreased burden while also increasing the proportion of assessments achieving high reliability for the adult banks; however, the SE-change rule and fixed-length CATs with 8 or fewer items also notably increased assessments yielding reliability <0.85. Among the alternative rules explored, the reduced maximum stopping rule best balanced precision and parsimony, presenting another option beyond the original rules. Conclusions: Our findings demonstrate the challenges in attempting to reduce test burden while also achieving score precision for clinical use. Stopping rules should be modified in accordance with the context of the study population and the purpose of the study.
- Open Access
- 1 December 2024
- Published by Wiley in Alzheimer's & Dementia
- Vol. 20 (S10)
Digital Cognitive Screening in Primary Care: Feasibility, Acceptability, and Usability of the MyCog Measurement System
Background: Annual cognitive screening in older adults is essential for early detection of cognitive impairment, yet less than half of cases are detected in primary care. We introduce two innovative app‐based screeners that help overcome barriers to routine cognitive screening. MyCog is a tablet app that is self‐administered in person during the rooming process for a primary care visit (Figure 1). MyCog Mobile is a smartphone app that is self‐administered remotely prior to a primary care visit (Figure 2), which can save clinicians even more time during an in‐person visit. We present evidence from two studies: 1) an evaluation of the adoption and usability of MyCog in primary care; and 2) a pilot study of the reliability and usability of MyCog Mobile.Method: In Study 1, the implementation of MyCog in 10 primary care clinics was assessed as part of a pragmatic trial evaluating cognitive screening for adults aged 65+ during Annual Wellness Visits (AWVs). Using EHR data, we assessed adoption via the portion of patients who completed MyCog. In Study 2, 51 adults 65+ self‐administered MyCog Mobile remotely on their own smartphones twice about two weeks apart and rated its usability on the Simplified System Usability Scale (S‐SUS). We examined the reliability, correlations with education, and usability ratings.Result: In Study 1, of the 3921 eligible patients, 1263 completed MyCog (32.2%), with individual clinic compliance ranging from 11% to 68% of patients. Feedback revealed several addressable clinician and patient barriers, including patient frustration on memory tests, unclear test progress, and need for a Spanish version, and minor design flaws. In Study 2, all participants successfully completed each measure, and reliability metrics were broadly acceptable (Table 1). The MyPictures and education correlation was lower than expected. Ratings on the S‐SUS indicated good usability of MyCog Mobile (M = 73.78, SD = 20.11).Conclusion: MyCog demonstrated good adoption by primary care clinics in a pragmatic trial, which may be improved through design and implementation changes. Some limitations of MyCog may be addressed with MyCog Mobile, which demonstrated preliminary feasibility evidence. Practice implications and next steps are discussed.
- 1 December 2024
- Published by Wiley in Alzheimer's & Dementia
- Vol. 20 (S10)
Open AccessDigital Cognitive Screening in Primary Care: Feasibility, Acceptability, and Usability of the MyCog Measurement System
Background: Annual cognitive screening in older adults is essential for early detection of cognitive impairment, yet less than half of cases are detected in primary care. We introduce two innovative app‐based screeners that help overcome barriers to routine cognitive screening. MyCog is a tablet app that is self‐administered in person during the rooming process for a primary care visit (Figure 1). MyCog Mobile is a smartphone app that is self‐administered remotely prior to a primary care visit (Figure 2), which can save clinicians even more time during an in‐person visit. We present evidence from two studies: 1) an evaluation of the adoption and usability of MyCog in primary care; and 2) a pilot study of the reliability and usability of MyCog Mobile.Method: In Study 1, the implementation of MyCog in 10 primary care clinics was assessed as part of a pragmatic trial evaluating cognitive screening for adults aged 65+ during Annual Wellness Visits (AWVs). Using EHR data, we assessed adoption via the portion of patients who completed MyCog. In Study 2, 51 adults 65+ self‐administered MyCog Mobile remotely on their own smartphones twice about two weeks apart and rated its usability on the Simplified System Usability Scale (S‐SUS). We examined the reliability, correlations with education, and usability ratings.Result: In Study 1, of the 3921 eligible patients, 1263 completed MyCog (32.2%), with individual clinic compliance ranging from 11% to 68% of patients. Feedback revealed several addressable clinician and patient barriers, including patient frustration on memory tests, unclear test progress, and need for a Spanish version, and minor design flaws. In Study 2, all participants successfully completed each measure, and reliability metrics were broadly acceptable (Table 1). The MyPictures and education correlation was lower than expected. Ratings on the S‐SUS indicated good usability of MyCog Mobile (M = 73.78, SD = 20.11).Conclusion: MyCog demonstrated good adoption by primary care clinics in a pragmatic trial, which may be improved through design and implementation changes. Some limitations of MyCog may be addressed with MyCog Mobile, which demonstrated preliminary feasibility evidence. Practice implications and next steps are discussed.
- Open Access
- 1 December 2024
- Published by Wiley in Alzheimer's & Dementia
- Vol. 20 (S5)
Digital Cognitive Screening in Primary Care: Feasibility, Acceptability, and Usability of the MyCog Measurement System
Background: Annual cognitive screening in older adults is essential for early detection of cognitive impairment, yet less than half of cases are detected in primary care. We introduce two innovative app‐based screeners that help overcome barriers to routine cognitive screening. MyCog is a tablet app that is self‐administered in person during the rooming process for a primary care visit (Figure 1). MyCog Mobile is a smartphone app that is self‐administered remotely prior to a primary care visit (Figure 2), which can save clinicians even more time during an in‐person visit. We present evidence from two studies: 1) an evaluation of the adoption and usability of MyCog in primary care; and 2) a pilot study of the reliability and usability of MyCog Mobile.Method: In Study 1, the implementation of MyCog in 10 primary care clinics was assessed as part of a pragmatic trial evaluating cognitive screening for adults aged 65+ during Annual Wellness Visits (AWVs). Using EHR data, we assessed adoption via the portion of patients who completed MyCog. In Study 2, 51 adults 65+ self‐administered MyCog Mobile remotely on their own smartphones twice about two weeks apart and rated its usability on the Simplified System Usability Scale (S‐SUS). We examined the reliability, correlations with education, and usability ratings.Result: In Study 1, of the 3921 eligible patients, 1263 completed MyCog (32.2%), with individual clinic compliance ranging from 11% to 68% of patients. Feedback revealed several addressable clinician and patient barriers, including patient frustration on memory tests, unclear test progress, and need for a Spanish version, and minor design flaws. In Study 2, all participants successfully completed each measure, and reliability metrics were broadly acceptable (Table 1). The MyPictures and education correlation was lower than expected. Ratings on the S‐SUS indicated good usability of MyCog Mobile (M = 73.78, SD = 20.11).Conclusion: MyCog demonstrated good adoption by primary care clinics in a pragmatic trial, which may be improved through design and implementation changes. Some limitations of MyCog may be addressed with MyCog Mobile, which demonstrated preliminary feasibility evidence. Practice implications and next steps are discussed.
- 1 December 2024
- Published by Wiley in Alzheimer's & Dementia
- Vol. 20 (S5)
Open AccessDigital Cognitive Screening in Primary Care: Feasibility, Acceptability, and Usability of the MyCog Measurement System
Background: Annual cognitive screening in older adults is essential for early detection of cognitive impairment, yet less than half of cases are detected in primary care. We introduce two innovative app‐based screeners that help overcome barriers to routine cognitive screening. MyCog is a tablet app that is self‐administered in person during the rooming process for a primary care visit (Figure 1). MyCog Mobile is a smartphone app that is self‐administered remotely prior to a primary care visit (Figure 2), which can save clinicians even more time during an in‐person visit. We present evidence from two studies: 1) an evaluation of the adoption and usability of MyCog in primary care; and 2) a pilot study of the reliability and usability of MyCog Mobile.Method: In Study 1, the implementation of MyCog in 10 primary care clinics was assessed as part of a pragmatic trial evaluating cognitive screening for adults aged 65+ during Annual Wellness Visits (AWVs). Using EHR data, we assessed adoption via the portion of patients who completed MyCog. In Study 2, 51 adults 65+ self‐administered MyCog Mobile remotely on their own smartphones twice about two weeks apart and rated its usability on the Simplified System Usability Scale (S‐SUS). We examined the reliability, correlations with education, and usability ratings.Result: In Study 1, of the 3921 eligible patients, 1263 completed MyCog (32.2%), with individual clinic compliance ranging from 11% to 68% of patients. Feedback revealed several addressable clinician and patient barriers, including patient frustration on memory tests, unclear test progress, and need for a Spanish version, and minor design flaws. In Study 2, all participants successfully completed each measure, and reliability metrics were broadly acceptable (Table 1). The MyPictures and education correlation was lower than expected. Ratings on the S‐SUS indicated good usability of MyCog Mobile (M = 73.78, SD = 20.11).Conclusion: MyCog demonstrated good adoption by primary care clinics in a pragmatic trial, which may be improved through design and implementation changes. Some limitations of MyCog may be addressed with MyCog Mobile, which demonstrated preliminary feasibility evidence. Practice implications and next steps are discussed.
- Open Access
- 7 February 2024
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 8 (1) , e54299
Remote Self-Administration of Cognitive Screeners for Older Adults Prior to a Primary Care Visit: Pilot Cross-Sectional Study of the Reliability and Usability of the MyCog Mobile Screening App
Background: Routine cognitive screening is essential in the early detection of dementia, but time constraints in primary care settings often limit clinicians’ ability to conduct screenings. MyCog Mobile is a newly developed cognitive screening system that patients can self-administer on their smartphones before a primary care visit, which can help save clinics’ time, encourage broader screening practices, and increase early detection of cognitive decline. Objective: The goal of this pilot study was to examine the feasibility, acceptability, and initial psychometric properties of MyCog Mobile. Research questions included (1) Can older adults complete MyCog Mobile remotely without staff support? (2) Are the internal consistency and test-retest reliability of the measures acceptable? and (3) How do participants rate the user experience of MyCog Mobile? Methods: A sample of adults aged 65 years and older (N=51) self-administered the MyCog Mobile measures remotely on their smartphones twice within a 2- to 3-week interval. The pilot version of MyCog Mobile includes 4 activities: MyFaces measures facial memory, MySorting measures executive functioning, MySequences measures working memory, and MyPictures measures episodic memory. After their first administration, participants also completed a modified version of the Simplified System Usability Scale (S-SUS) and 2 custom survey items. Results: All participants in the sample passed the practice items and completed each measure. Findings indicate that the Mobile Toolbox assessments measure the constructs well (internal consistency 0.73 to 0.91) and are stable over an approximately 2-week delay (test-retest reliability 0.61 to 0.71). Participants’ rating of the user experience (mean S-SUS score 73.17, SD 19.27) indicated that older adults found the usability of MyCog Mobile to be above average. On free-response feedback items, most participants provided positive feedback or no feedback at all, but some indicated a need for clarity in certain task instructions, concerns about participants’ abilities, desire to be able to contact a support person or use in-app technical support, and desire for additional practice items. Conclusions: Pilot evidence suggests that the MyCog Mobile cognitive screener can be reliably self-administered by older adults on their smartphones. Participants in our study generally provided positive feedback about the MyCog Mobile experience and rated the usability of the app highly. Based on participant feedback, we will conduct further usability research to improve support functionality, optimize task instructions and practice opportunities, and ensure that patients feel comfortable using MyCog Mobile. The next steps include a clinical validation study that compares MyCog Mobile to gold-standard assessments and tests the sensitivity and specificity of the measures for identifying dementia.
- 7 February 2024
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 8 (1) , e54299
Open AccessRemote Self-Administration of Cognitive Screeners for Older Adults Prior to a Primary Care Visit: Pilot Cross-Sectional Study of the Reliability and Usability of the MyCog Mobile Screening App
Background: Routine cognitive screening is essential in the early detection of dementia, but time constraints in primary care settings often limit clinicians’ ability to conduct screenings. MyCog Mobile is a newly developed cognitive screening system that patients can self-administer on their smartphones before a primary care visit, which can help save clinics’ time, encourage broader screening practices, and increase early detection of cognitive decline. Objective: The goal of this pilot study was to examine the feasibility, acceptability, and initial psychometric properties of MyCog Mobile. Research questions included (1) Can older adults complete MyCog Mobile remotely without staff support? (2) Are the internal consistency and test-retest reliability of the measures acceptable? and (3) How do participants rate the user experience of MyCog Mobile? Methods: A sample of adults aged 65 years and older (N=51) self-administered the MyCog Mobile measures remotely on their smartphones twice within a 2- to 3-week interval. The pilot version of MyCog Mobile includes 4 activities: MyFaces measures facial memory, MySorting measures executive functioning, MySequences measures working memory, and MyPictures measures episodic memory. After their first administration, participants also completed a modified version of the Simplified System Usability Scale (S-SUS) and 2 custom survey items. Results: All participants in the sample passed the practice items and completed each measure. Findings indicate that the Mobile Toolbox assessments measure the constructs well (internal consistency 0.73 to 0.91) and are stable over an approximately 2-week delay (test-retest reliability 0.61 to 0.71). Participants’ rating of the user experience (mean S-SUS score 73.17, SD 19.27) indicated that older adults found the usability of MyCog Mobile to be above average. On free-response feedback items, most participants provided positive feedback or no feedback at all, but some indicated a need for clarity in certain task instructions, concerns about participants’ abilities, desire to be able to contact a support person or use in-app technical support, and desire for additional practice items. Conclusions: Pilot evidence suggests that the MyCog Mobile cognitive screener can be reliably self-administered by older adults on their smartphones. Participants in our study generally provided positive feedback about the MyCog Mobile experience and rated the usability of the app highly. Based on participant feedback, we will conduct further usability research to improve support functionality, optimize task instructions and practice opportunities, and ensure that patients feel comfortable using MyCog Mobile. The next steps include a clinical validation study that compares MyCog Mobile to gold-standard assessments and tests the sensitivity and specificity of the measures for identifying dementia.
- Open Access
- 15 November 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Remote Self-Administration of Cognitive Screeners for Older Adults Prior to a Primary Care Visit: Pilot Cross-Sectional Study of the Reliability and Usability of the MyCog Mobile Screening App (Preprint)
BACKGROUND Routine cognitive screening is essential in the early detection of dementia, but time constraints in primary care settings often limit clinicians’ ability to conduct screenings. MyCog Mobile is a newly developed cognitive screening system that patients can self-administer on their smartphones before a primary care visit, which can help save clinics’ time, encourage broader screening practices, and increase early detection of cognitive decline. OBJECTIVE The goal of this pilot study was to examine the feasibility, acceptability, and initial psychometric properties of MyCog Mobile. Research questions included (1) Can older adults complete MyCog Mobile remotely without staff support? (2) Are the internal consistency and test-retest reliability of the measures acceptable? and (3) How do participants rate the user experience of MyCog Mobile? METHODS A sample of adults aged 65 years and older (N=51) self-administered the MyCog Mobile measures remotely on their smartphones twice within a 2- to 3-week interval. The pilot version of MyCog Mobile includes 4 activities: MyFaces measures facial memory, MySorting measures executive functioning, MySequences measures working memory, and MyPictures measures episodic memory. After their first administration, participants also completed a modified version of the Simplified System Usability Scale (S-SUS) and 2 custom survey items. RESULTS All participants in the sample passed the practice items and completed each measure. Findings indicate that the Mobile Toolbox assessments measure the constructs well (internal consistency 0.73 to 0.91) and are stable over an approximately 2-week delay (test-retest reliability 0.61 to 0.71). Participants’ rating of the user experience (mean S-SUS score 73.17, SD 19.27) indicated that older adults found the usability of MyCog Mobile to be above average. On free-response feedback items, most participants provided positive feedback or no feedback at all, but some indicated a need for clarity in certain task instructions, concerns about participants’ abilities, desire to be able to contact a support person or use in-app technical support, and desire for additional practice items. CONCLUSIONS Pilot evidence suggests that the MyCog Mobile cognitive screener can be reliably self-administered by older adults on their smartphones. Participants in our study generally provided positive feedback about the MyCog Mobile experience and rated the usability of the app highly. Based on participant feedback, we will conduct further usability research to improve support functionality, optimize task instructions and practice opportunities, and ensure that patients feel comfortable using MyCog Mobile. The next steps include a clinical validation study that compares MyCog Mobile to gold-standard assessments and tests the sensitivity and specificity of the measures for identifying dementia.
- 15 November 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessRemote Self-Administration of Cognitive Screeners for Older Adults Prior to a Primary Care Visit: Pilot Cross-Sectional Study of the Reliability and Usability of the MyCog Mobile Screening App (Preprint)
BACKGROUND Routine cognitive screening is essential in the early detection of dementia, but time constraints in primary care settings often limit clinicians’ ability to conduct screenings. MyCog Mobile is a newly developed cognitive screening system that patients can self-administer on their smartphones before a primary care visit, which can help save clinics’ time, encourage broader screening practices, and increase early detection of cognitive decline. OBJECTIVE The goal of this pilot study was to examine the feasibility, acceptability, and initial psychometric properties of MyCog Mobile. Research questions included (1) Can older adults complete MyCog Mobile remotely without staff support? (2) Are the internal consistency and test-retest reliability of the measures acceptable? and (3) How do participants rate the user experience of MyCog Mobile? METHODS A sample of adults aged 65 years and older (N=51) self-administered the MyCog Mobile measures remotely on their smartphones twice within a 2- to 3-week interval. The pilot version of MyCog Mobile includes 4 activities: MyFaces measures facial memory, MySorting measures executive functioning, MySequences measures working memory, and MyPictures measures episodic memory. After their first administration, participants also completed a modified version of the Simplified System Usability Scale (S-SUS) and 2 custom survey items. RESULTS All participants in the sample passed the practice items and completed each measure. Findings indicate that the Mobile Toolbox assessments measure the constructs well (internal consistency 0.73 to 0.91) and are stable over an approximately 2-week delay (test-retest reliability 0.61 to 0.71). Participants’ rating of the user experience (mean S-SUS score 73.17, SD 19.27) indicated that older adults found the usability of MyCog Mobile to be above average. On free-response feedback items, most participants provided positive feedback or no feedback at all, but some indicated a need for clarity in certain task instructions, concerns about participants’ abilities, desire to be able to contact a support person or use in-app technical support, and desire for additional practice items. CONCLUSIONS Pilot evidence suggests that the MyCog Mobile cognitive screener can be reliably self-administered by older adults on their smartphones. Participants in our study generally provided positive feedback about the MyCog Mobile experience and rated the usability of the app highly. Based on participant feedback, we will conduct further usability research to improve support functionality, optimize task instructions and practice opportunities, and ensure that patients feel comfortable using MyCog Mobile. The next steps include a clinical validation study that compares MyCog Mobile to gold-standard assessments and tests the sensitivity and specificity of the measures for identifying dementia.