Piper Fromy

BackgroundCognitive impairment is a major factor impacting the functioning and quality of life of individuals with schizophrenia. Several clinical outcome assessment (COA) measures have been developed to measure various aspects of cognition. This study aimed to identify thresholds for interpreting changes that would be meaningful to patients.MethodsThe study utilised COA data from a multinational program of randomised controlled trials evaluating iclepertin for the treatment of cognitive impairment associated with schizophrenia (CIAS). Endpoints included the MATRICS Consensus Cognitive Battery (MCCB), Schizophrenia Cognition Rating Scale (SCoRS), Virtual Reality Functional Capacity Assessment Tool (VRFCAT) and Patient Reported Experience of Cognitive Impairment in Schizophrenia (PRECIS). Analysis utilised anchor-based methods supported by distribution-based methods to derive interpretation thresholds. Triangulation resulted in a single value or small range of values for between-group difference and meaningful within-patient change (MWPC) thresholds.ResultsMWPC thresholds for improvement were increases of 4 to 7 points for MCCB overall composite T-score and 10 points for VRFCAT overall T-score and decreases of 5 to 7 points for SCoRS interviewer total score and 0.4 to 0.6 points for PRECIS total score. MCCB and VRFCAT thresholds were estimated using distribution-based methods due to insufficient correlation between anchors and the target scores.ConclusionThe proposed score interpretation thresholds may be valuable for assessing improvement in CIAS in future clinical research. Anchor-based analyses may be limited in this patient population as patient, caregiver and clinician-based global ratings of severity tend to have low correlations with multi-item COA scores, particularly with performance-based measures.

Background: Sensor-based digital health technologies (sDHTs) are increasingly used to support scientific and clinical decision-making. The digital measures (DMs) they generate offer significant potential to accelerate the drug development timeline, decrease clinical trial costs, and improve access to care. However, choosing an appropriate statistical methodology when conducting analytical validation (AV) of a DM is complicated, particularly for novel DMs, for which appropriate, established reference measures (RMs) may not exist. More understanding of, and a standardized approach to, AV in these scenarios is needed. Objective: In a prior simulation study, 3 statistical methods were tested for their ability to estimate a simulated relationship between a sDHT-derived DM and several clinical outcome assessment (COA) RMs. The aim of this work was to assess the feasibility of implementation of these methods in real data and to examine the impact of AV study design factors on the relationships estimated. Methods: Four real-world datasets, captured using sDHTs, were used to prepare hypothetical AV studies representing a range of scenarios with respect to 3 key study design properties: temporal coherence, construct coherence, and data completeness. The datasets analyzed were as follows: Urban Poor (comparing nighttime awakenings to measures of psychological well-being), STAGES (comparing daily step count to psychological and fatigue measures), mPower (comparing daily smartphone screen taps to measures of function in Parkinson’s disease), and Brighten (comparing smartphone communication activity to measures of psychological well-being). For each hypothetical AV study, 3 statistical methods were leveraged: the Pearson correlation coefficient (PCC) between DM and RM, simple linear regression (SLR) between DM and RM, multiple linear regression (MLR) between DMs and combinations of RMs, and 2-factor, correlated-factor confirmatory factor analysis (CFA) models. Performance measures were the PCC magnitudes (for PCC), R ² and adjusted R ² statistics (for SLR and MLR, respectively), and factor correlations (for CFA). Results: Most of the CFA models exhibited an acceptable fit according to the majority of the fit statistics employed, and each model was able to estimate a factor correlation. For each model, these correlations were greater than or equal to the corresponding PCC in magnitude. Correlations were the strongest in the hypothetical studies with strong temporal and construct coherence. Conclusions: The performance of the selected statistical methods shown in this work supports their feasibility when implemented in real-world data. Our findings, in particular, support the use of CFA to assess the relationship between a novel DM and a COA RM. The observed impact of AV study design factors on the relationships estimated allowed the authors to determine practical recommendations for study design in AV of novel DMs. By using a standardized methodology for evaluating novel DMs, sDHT developers, biostatisticians, and clinical researchers can navigate the complex validation landscape more easily, with more certainty, and with more tools at their disposal.

Sensor-based digital health technologies (sDHTs) are increasingly used to support scientific and clinical decision-making. The digital clinical measures they generate offer enormous benefits, including providing more patient-relevant data, improving patient access, reducing costs, and driving inclusion across health care ecosystems. Scientific best practices and regulatory guidance now provide clear direction to investigators seeking to evaluate sDHTs for use in different contexts. However, the quality of the evidence reported for analytical validation of sDHTs—evaluation of algorithms converting sample-level sensor data into a measure that is clinically interpretable—is inconsistent and too often insufficient to support a particular digital measure as fit-for-purpose. We propose a hierarchical framework to address challenges related to selecting the most appropriate reference measure for conducting analytical validation and codify best practices and an approach that will help capture the greatest value of sDHTs for public health, patient care, and medical product development.

BACKGROUND Sensor-based digital health technologies (sDHTs) are increasingly used to support scientific and clinical decision-making. The digital measures (DMs) they generate offer significant potential to accelerate the drug development timeline, decrease clinical trial costs, and improve access to care. However, choosing appropriate statistical methodology when conducting analytical validation (AV) of a DM is complicated, particularly for novel DMs, for which appropriate, established reference measures may not exist. More understanding of, and a standardization of approach to, AV in these scenarios is needed. OBJECTIVE In a prior simulation study, three statistical methods were tested for their ability to estimate a simulated relationship between a sDHT-derived DM and several clinical outcome assessment (COA) reference measures. The aim of this work was to assess the feasibility of these methods’ implementation in real data, and to examine the impact of AV study design factors on the relationships estimated. METHODS Four real-world datasets, captured using sDHTs, were used to prepare hypothetical AV studies that represented a range of scenarios with respect to three key study design properties: temporal coherence, construct coherence and data completeness. For each hypothetical study, two-factor, correlated-factors confirmatory factor analysis (CFA) models, and a combination of simple and multiple linear regression models, were built using the DM and reference measure data. The factor correlation, R2 and adjusted R2 statistics were calculated, in addition to the Pearson Correlation Coefficients (PCCs). RESULTS The majority of CFA models exhibited an acceptable fit according to the majority of the fit statistics employed, and each model was able to estimate a factor correlation. For each model, these correlations were greater than or equal to the corresponding PCC in magnitude. Correlations were strongest in the hypothetical studies with strong temporal and construct coherence. CONCLUSIONS The performance of the measures shown in this work supports the feasibility of the selected statistical methods when implemented in real-world data. Our findings in particular support the use of CFA to assess the relationship between a novel digital measure and a COA reference measure. The observed impact of AV study design factors on the relationships estimated allowed the authors to determine practical recommendations to aid in appropriate study design for AV of novel digital measures. By utilizing a standardized methodology for evaluating novel digital measures, sDHT developers, biostatisticians and clinical researchers will be able to navigate the complex validation landscape more easily, with more certainty, and with more tools at their disposal, expediting the pathway to validation and regulatory review.

Introduction Chronic Hand Eczema (CHE) is an inflammatory skin disease that causes significant impact on health-related quality of life (HRQoL). The Hand Eczema Impact Scale (HEIS) is a new patient-reported outcome (PRO) measure designed to assess the impact of CHE on key domains of HRQoL. This study aimed to develop and evaluate content and psychometric validity of the HEIS. Methods The HEIS was initially developed on the basis of a literature review and concept elicitation interviews. Qualitative cognitive debriefing interviews (n = 20) were conducted with patients with CHE to assess relevance and understanding of items, response options, and recall period. Psychometric properties (item performance, dimensionality, reliability, validity, responsiveness, and estimation of meaningful change thresholds) were then assessed using data (n = 258) from a phase 2b trial (NCT03683719). Results Cognitive debriefing confirmed all items were understood and relevant to patients. Inter-item correlations (all > 0.50) and confirmatory factor analysis (factor loadings ≥ 0.80) supported unidimensionality of the HEIS score, and mostly provided support for the HEIS Proximal Daily Activity Limitations (PDAL) score, with only one item loading below the prespecified threshold. Item properties and previous qualitative work supported retaining this item in the total score but removed from the HEIS PDAL domain. Internal consistency (Cronbach’s alpha ≥ 0.89) and test–retest reliability (intra-class correlation coefficient ≥ 0.79) results were very strong. Strong correlations with concurrent measures (0.66–0.87) and significant differences between severity groups (p < 0.001) supported construct validity. Large effect sizes for mean change scores in participants that improved and significant differences between groups indicated ability to detect change. Anchor-based analyses supported within-individual responder definitions of ≥ 1.3 points for improvements in both HEIS score and HEIS PDAL score (covering three items) and of ≥ 1.5 points for HEIS embarrassment with the appearance of hands (Emb) score (covering two items). Conclusions The 9-item HEIS is the first CHE-specific PRO measure developed and validated according to regulatory guidance for assessment of the impact of CHE on key domains of HRQoL. This article provides evidence of strong content and psychometric validity and shows improvements of ≥ 1.3 points in HEIS score and HEIS PDAL score, and improvements of ≥ 1.5 points in HEIS Emb score represent clinically meaningful, important changes. Trial Registration NCT03683719.

UNSTRUCTURED Sensor-based digital health technologies (sDHTs) are increasingly used to support scientific and clinical decision-making. The digital clinical measures they generate offer enormous benefits, including providing more patient-relevant data, improving patient access, reducing costs, and driving inclusion across healthcare ecosystems. Scientific best practices and regulatory guidance now provide clear direction to investigators seeking to evaluate sDHTs and document claims related to their suitability for use in different contexts. However, the quality of the evidence reported for analytical validation of sDHTs —evaluation of algorithms converting sample-level sensor data into a measure that is clinically interpretable— is inconsistent and too often insufficient to support rigorous claims based on digital clinical measures. We propose a hierarchical framework to address challenges related to selecting the most appropriate reference measure for conducting analytical validation, as dictated by the medical claims related to the sDHT algorithm, and codify best practices and an approach that will help capture the greatest value of sDHTs for public health, patient care, and medical product development.

Digital health technologies offer valuable advantages to dementia researchers and clinicians as screening tools, diagnostic aids, and monitoring instruments. To support the use and advancement of these resources, a comprehensive overview of the current technological landscape is essential. A multi-stakeholder working group, convened by the Digital Medicine Society (DiMe), conducted a landscape review to identify digital health technologies for Alzheimer’s disease and related dementia populations. We searched studies indexed in PubMed, Embase, and APA PsycInfo to identify manuscripts published between May 2003 to May 2023 reporting analytical validation, clinical validation, or usability/feasibility results for relevant digital health technologies. Additional technologies were identified through community outreach. We collated peer-reviewed manuscripts, poster presentations, or regulatory documents for 106 different technologies for Alzheimer’s disease and related dementia assessment covering diverse populations such as Lewy Body, vascular dementias, frontotemporal dementias, and all severities of Alzheimer’s disease. Wearable sensors represent 32% of included technologies, non-wearables 61%, and technologies with components of both account for the remaining 7%. Neurocognition is the most prevalent concept of interest, followed by physical activity and sleep. Clinical validation is reported in 69% of evidence, analytical validation in 34%, and usability/feasibility in 20% (not mutually exclusive). These findings provide clinicians and researchers a landscape overview describing the range of technologies for assessing Alzheimer’s disease and related dementias. A living library of technologies is presented for the clinical and research communities which will keep findings up-to-date as the field develops.