Are ecological momentary assessment measures of intervention change worth the trouble?: Evaluation in four digital mental health trials (Preprint)
Open Access
- 26 November 2024
- journal article
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27, e69297
Abstract
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Ecological momentary assessment (EMA) is increasingly being incorporated into intervention studies to acquire a more fine-grained and ecologically valid assessment of change. The added utility of including relatively burdensome EMA measures in a clinical trial hinges on several psychometric assumptions, including that these measure are: (1) reliable, (2) related but not redundant with conventional self-report measures (convergent and discriminant validity), (3) sensitive to intervention-related change, and (4) associated with a clinically-relevant criterion of improvement (criterion validity) above conventional self-report measures (incremental validity). Using data from 4 app-based meditation trials (N = 412), we examined the reliability, validity, and sensitivity to change of conventional self-report and EMA measures of improvement in rumination. Conventional self-report and EMA measures of rumination were only modestly correlated, particularly with regards to change over time, which may be due to their lower reliability. Changes in rumination were larger for conventional self-report than EMA. Notably, change in both self-report and EMA rumination each accounted for unique variance in depressive symptom improvement, demonstrating incremental predictive validity. Conventional self-report and EMA measures of rumination provide distinct and clinically meaningful information. Researchers using EMA should consider the psychometric properties of their measures and the precise construct they intend to capture.Keywords
This publication has 102 references indexed in Scilit:
- Response times in Ecological Momentary Assessment (EMA): shedding light on the response process with a drift diffusion modelCurrent Psychology, 2023
- Acceptability and Preliminary Effects of a Mindfulness Mobile Application for Ruminative AdolescentsBehavior Therapy, 2021
- App-Based Mindfulness Training for Adolescent Rumination: Predictors of Immediate and Cumulative BenefitMindfulness, 2021
- Comparing integrative cognitive‐affective therapy and guided self‐help cognitive‐behavioral therapy to treat binge‐eating disorder using standard and naturalistic momentary outcome measures: A randomized controlled trialInternational Journal of Eating Disorders, 2020
- Hedonic Evaluation over Short and Long Retention Intervals: The Mechanism of the Peak–End RuleJournal of Behavioral Decision Making, 2013
- Getting Out of Rumination: Comparison of Three Brief Interventions in a Sample of YouthJournal of Abnormal Child Psychology, 2012
- Ecological Validity for Patient Reported Outcomes2010
- Memory for fatigue in chronic fatigue syndrome: The relation between weekly recall and momentary ratingsInternational Journal of Behavioral Medicine, 2008
- Factor structure, concurrent validity, and internal consistency of the beck depression inventory?second edition in a sample of college studentsDepression and Anxiety, 2004
- Patients' memories of painful medical treatments: real-time and retrospective evaluations of two minimally invasive proceduresPain, 1996