State-Dependent Missingness in Hidden Markov Models, with an Application to Drop-Out in a Clinical Trial
Open Access
- 3 January 2025
- journal article
- research article
- Published by Cambridge University Press (CUP) in Psychometrika
- Vol. 90 (2) , 476-507
Abstract
Establishing the effectiveness of treatments for psychopathology requires accurate models of its progression over time and the factors that impact it. Longitudinal data is however fraught with missingness, hindering accurate modeling. We re-analyse data on schizophrenia severity in a clinical trial using hidden Markov models (HMMs). We consider missing data in HMMs with a focus on situations where data is missing not at random (MNAR) and missingness depends on the latent states, allowing symptom severity to indirectly impact probability of missingness. In simulations, we show that including a submodel for state-dependent missingness reduces bias when data is MNAR and state-dependent, whilst not reducing accuracy when data is missing-at-random (MAR). When missingness depends on time, a model that allows missingness to be both state- and time-dependent is unbiased. Overall, these results show that modelling missingness as state-dependent and including relevant covariates is a useful strategy in applications of HMMs to time-series with missing data. Applying the model to data from a clinical trial, we find that drop-out is more likely for patients with less severe symptoms, which may lead to a biased assessment of treatment effectiveness.Keywords
This publication has 13 references indexed in Scilit:
- A hidden Markov model for continuous longitudinal data with missing responses and dropoutBiometrical Journal, 2023
- Refining our understanding of depressive states and state transitions in response to cognitive behavioural therapy using latent Markov modellingPsychological Medicine, 2020
- Identification and initial validation of empirically derived bipolar symptom states from a large longitudinal dataset: an application of hidden Markov modeling to the Systematic Treatment Enhancement Program for Bipolar Disorder (STEP-BD) studyPsychological Medicine, 2018
- Major depressive disorder as a nonlinear dynamic system: bimodality in the frequency distribution of depressive symptoms over timeBMC Psychiatry, 2015
- A Discrete Time Event-History Approach to Informative Drop-Out in Mixed Latent Markov Models with CovariatesBiometrics, 2014
- Intermittent Missing Observations in Discrete-Time Hidden Markov ModelsCommunications in Statistics - Simulation and Computation, 2012
- Estimating Transition Probabilities for Ignorable Intermittent Missing Data in a Discrete-Time Markov ChainCommunications in Statistics - Simulation and Computation, 2010
- A hidden semi-Markov model with missing data and multiple observation sequences for mobility trackingSignal Processing, 2003
- A Transitional Model for Longitudinal Binary Data Subject to Nonignorable Missing DataBiometrics, 2000
- Stochastic Algorithms for Markov Models Estimation with Intermittent Missing DataBiometrics, 1999