Exploring a Graph-based Approach to Offline Reinforcement Learning for Sepsis Treatment
Preprint
Abstract
Sepsis is a serious, life-threatening condition. When treating sepsis, it is challenging to determine the correct amount of intravenous fluids and vasopressors for a given patient. While automated reinforcement learning (RL)-based methods have been used to support these decisions with promising results, previous studies have relied on relational data. Given the complexity of modern healthcare data, representing data as a graph may provide a more natural and effective approach. This study models patient data from the well-known MIMIC-III dataset as a heterogeneous graph that evolves over time. Subsequently, we explore two Graph Neural Network architectures - GraphSAGE and GATv2 - for learning patient state representations, adopting the approach of decoupling representation learning from policy learning. The encoders are trained to produce latent state representations, jointly with decoders that predict the next patient state. These representations are then used for policy learning with the dBCQ algorithm. The results of our experimental evaluation confirm the potential of a graph-based approach, while highlighting the complexity of representation learning in this domain.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Offline Safe Reinforcement Learning for Sepsis Treatment: Tackling Variable-Length Episodes with Sparse RewardsHuman-Centric Intelligent Systems, 2025
- Deep reinforcement learning extracts the optimal sepsis treatment policy from treatment recordsCommunications Medicine, 2024
- Reinforced Sequential Decision-Making for Sepsis Treatment: The PosNegDM Framework With Mortality Classifier and TransformerIEEE Journal of Biomedical and Health Informatics, 2024
- Optimizing sepsis treatment strategies via a reinforcement learning modelBiomedical Engineering Letters, 2024
- A value-based deep reinforcement learning model with human expertise in optimal treatment of sepsisnpj Digital Medicine, 2023
- The treatment of sepsis: an episodic memory-assisted deep reinforcement learning approachApplied Intelligence, 2022
- Predicting Patient Outcomes with Graph Representation Learning2022
- Transatlantic transferability of a new reinforcement learning model for optimizing haemodynamic treatment for critically ill patients with sepsisArtificial Intelligence in Medicine, 2021
- Graph neural networks: A review of methods and applicationsAI Open, 2020
- The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive careNature Medicine, 2018