Employing Deep Reinforcement Learning to Maximize Lower Limb Blood Flow Using Intermittent Pneumatic Compression
- 5 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 28 (10) , 6193-6200
Abstract
Intermittent pneumatic compression (IPC) systems apply external pressure to the lower limbs and enhance peripheral blood flow. We previously introduced a cardiac-gated compression system that enhanced arterial blood velocity (BV) in the lower limb compared to fixed compression timing (CT) for seated and standing sub7 jects. However, these pilot studies found that the CT that maximized BV was not constant across individuals and could change over time. Current CT modelling methods for IPC are limited to predictions for a single day and one heartbeat ahead. However, IPC therapy for may span weeks or longer, the BV response to compression can vary with physiological state, and the best CT for eliciting the desired physiological outcome may change, even for the same individual. We propose that a deep reinforcement learning (DRL) algorithm can learn and adaptively modify CT to achieve a selected outcome using IPC. Herein, we target maximizing lower limb arterial BV as the desired out19 come and build participant-specific simulated lower limb environments for 6 participants. We show that DRL can adaptively learn the CT for IPC that maximized arterial BV. Compared to previous work, the DRL agent achieves 98% ± 2 of the resultant blood flow and is faster at maximizing BV; the DRL agent can learn an “optimal” policy in 15 minutes ± 2 on average and can adapt on the fly. Given a desired objective, we posit that the proposed DRL agent can be implemented in IPC systems to rapidly learn the (potentially time-varying) “optimal” CT with a human-in-the-loop.Keywords
Funding Information
- Natural Sciences and Engineering Research Council of Canada
- Ontario Graduate Scholarship in Science and Technology
This publication has 21 references indexed in Scilit:
- Investigating Optimal Intermittent Pneumatic Compression Timing Across Two Days2023
- Optimization of Dry Weight Assessment in Hemodialysis Patients via Reinforcement LearningIEEE Journal of Biomedical and Health Informatics, 2022
- Accurate Blood Pressure Estimation During Activities of Daily Living: A Wearable Cuffless SolutionIEEE Journal of Biomedical and Health Informatics, 2021
- Intermittent compression of the calf muscle as a countermeasure to protect blood pressure and brain blood flow in upright posture in older adultsEuropean Journal of Applied Physiology, 2021
- Toward a Fully Automated Artificial Pancreas System Using a Bioinspired Reinforcement Learning Design: In Silico ValidationIEEE Journal of Biomedical and Health Informatics, 2020
- Reinforcement-learning optimal control for type-1 diabetes2018
- Optimization of anemia treatment in hemodialysis patients via reinforcement learningArtificial Intelligence in Medicine, 2014
- Partially Observable Markov Decision Processes2012
- Randomized clinical trial of intermittent pneumatic compression and low molecular weight heparin in traumaBritish Journal of Surgery, 2003
- Optimising the Performance of Intermittent Pneumatic Compression DevicesEuropean Journal of Vascular and Endovascular Surgery, 2000