TrajGPT: Irregular Time-Series Representation Learning of Health Trajectory

Authors:
Publisher website
Google Scholar
Abstract
In the healthcare domain, time-series data are often irregularly sampled with varying intervals through outpatient visits, posing challenges for existing models designed for equally spaced sequential data. To address this, we propose Trajectory Generative Pre-trained Transformer (TrajGPT) for representation learning on irregularly-sampled healthcare time series. TrajGPT introduces a novel Selective Recurrent Attention (SRA) module that leverages a data-dependent decay to adaptively filter irrelevant past information. As a discretized ordinary differential equation (ODE) framework, TrajGPT captures underlying continuous dynamics and enables a time-specific inference for forecasting arbitrary target timesteps without auto-regressive prediction. Experimental results based on the longitudinal EHR data PopHR from Montreal health system and eICU from PhysioNet showcase TrajGPT's superior zero-shot performance in disease forecasting, drug usage prediction, and sepsis detection. The inferred trajectories of diabetic and cardiac patients reveal meaningful comorbidity conditions, underscoring TrajGPT as a useful tool for forecasting patient health evolution.
Funding Information
  • Nature et technologies
  • NOVA-FRQNT-NSERC (FRQ-NT 2023-NOVA-328677)
  • Canada Research Chairs
  • Machine Learning for Genomics and Healthcare (CRC-2021-00547)
  • Natural Sciences and Engineering Research Council of Canada (RGPIN-2016-05174)

This publication has 25 references indexed in Scilit: