End-to-End Incomplete Time-Series Modeling From Linear Memory of Latent Variables
- 11 April 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 50 (12) , 4908-4920
Abstract
Time series with missing values (incomplete time series) are ubiquitous in real life on account of noise or malfunctioning sensors. Time-series imputation (replacing missing data) remains a challenge due to the potential for nonlinear dependence on concurrent and previous values of the time series. In this paper, we propose a novel framework for modeling incomplete time series, called a linear memory vector recurrent neural network (LIME-RNN), a recurrent neural network (RNN) with a learned linear combination of previous history states. The technique bears some similarity to residual networks and graph-based temporal dependency imputation. In particular, we introduce a linear memory vector [called the residual sum vector (RSV)] that integrates over previous hidden states of the RNN, and is used to fill in missing values. A new loss function is developed to train our model with time series in the presence of missing values in an end-to-end way. Our framework can handle imputation of both missing-at-random and consecutive missing inputs. Moreover, when conducting time-series prediction with missing values, LIME-RNN allows imputation and prediction simultaneously. We demonstrate the efficacy of the model via extensive experimental evaluation on univariate and multivariate time series, achieving state-of-the-art performance on synthetic and real-world data. The statistical results show that our model is significantly better than most existing time-series univariate or multivariate imputation methods.Keywords
Funding Information
- National Natural Science Foundation of China (61502174, 61872148, 61722205, 61751205, 61572199, 61751202)
- Natural Science Foundation of Guangdong Province (2017A030313355, 2017A030313358)
- Guangdong Province Higher Vocational Colleges and Schools Pearl River Scholar Funded Scheme
- Special Project for Research and Development in Key areas of Guangdong Province (2018B010107002)
- Guangzhou Science and Technology Planning Project (201704030051)
- Opening Project of Guangdong Province Key Laboratory of Big Data Analysis and Processing (2017014)
- Guangdong University of Finance and Economics Big Data and Educational Statistics Application Laboratory (2017WSYS001)
- National Science Foundation, USA, to the Temporal Dynamics of Learning Center, an NSF Science of Learning Center (SMA 1041755)
This publication has 31 references indexed in Scilit:
- Learning a Mahalanobis Distance-Based Dynamic Time Warping Measure for Multivariate Time Series ClassificationIEEE Transactions on Cybernetics, 2015
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Projection-Based Ensemble Learning for Ordinal RegressionIEEE Transactions on Cybernetics, 2013
- Multiple imputation using chained equations: Issues and guidance for practiceStatistics in Medicine, 2010
- Kalman Filtering With Intermittent ObservationsIEEE Transactions on Automatic Control, 2004
- An analysis of four missing data treatment methods for supervised learningApplied Artificial Intelligence, 2003
- Functional-Coefficient Regression Models for Nonlinear Time SeriesJournal of the American Statistical Association, 2000
- Long Short-Term MemoryNeural Computation, 1997
- Learning representations by back-propagating errorsNature, 1986
- Inference and missing dataBiometrika, 1976