Medical Data Wrangling With Sequential Variational Autoencoders
- 29 October 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 26 (6) , 2737-2745
Abstract
Medical data sets are usually corrupted by noise and missing data. These missing patterns are commonly assumed to be completely random, but in medical scenarios, the reality is that these patterns occur in bursts due to sensors that are off for some time or data collected in a misaligned uneven fashion, among other causes. This paper proposes to model medical data records with heterogeneous data types and bursty missing data using sequential variational autoencoders (VAEs). In particular, we propose a new methodology, the Shi-VAE, which extends the capabilities of VAEs to sequential streams of data with missing observations. We compare our model against state-of-the-art solutions in an intensive care unit database (ICU) and a dataset of passive human monitoring. Furthermore, we find that standard error metrics such as RMSE are not conclusive enough to assess temporal models and include in our analysis the cross-correlation between the ground truth and the imputed signal. We show that Shi-VAE achieves the best performance in terms of using both metrics, with lower computational complexity than the GP-VAE model, which is the state-of-the-art method for medical records.Keywords
All Related Versions
Funding Information
- Ministerio de Economa Industria y Competitividad Gobierno de Espaa (RTI2018-099655-B-100, TEC2017-92552- EXP)
- H2020 European Research Council (714161)
- Comunidad de Madrid (IND2020/TIC-17372, Y2018/TCS-4705)
- Fundacin BBVA (Deep-DARWiN project)
This publication has 14 references indexed in Scilit:
- VIGAN: Missing view imputation with generative adversarial networks2017
- MyTracesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2017
- Trajectories of depression2015
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- MoodScope2013
- Predicting postpartum changes in emotion and behavior via social media2013
- StressSense2012
- Multiple imputation using chained equations: Issues and guidance for practiceStatistics in Medicine, 2010
- The Elements of Statistical Learning2001
- Long Short-Term MemoryNeural Computation, 1997