LSTM-Based Modeling and Reinforcement Learning Control of a Magnetically Actuated Catheter
Preprint
Abstract
Autonomous magnetic catheter systems are emerging as a promising approach for the future of minimally invasive interventions. This study presents a novel approach that begins by modeling the nonlinear and hysteretic dynamics of a magnetically actuated catheter system, consists of a magnetic catheter manipulated by servo-controlled magnetic fields generated by two external permanent magnets, and its complex behavior is captured using a Long Short-Term Memory (LSTM) neural network. This model validated against experimental setup's data with a root mean square error (RMSE) of 0.42 mm and 99.8% coverage within 3 mm, establishing it as a reliable surrogate model. This LSTM enables the training of Reinforcement Learning (RL) agents for controlling the system and avoiding damage to the real setup, with the potential for subsequent fine-tuning on the physical system. We implemented Deep Q-Network (DQN) and actor-critic RL controllers, comparing these two agents first for regulation and subsequently for path following along linear and half-sinusoidal paths for the catheter tip. The actor-critic outperforms DQN, offering greater accuracy and faster performance with less error, along with smoother trajectories at a 10 Hz sampling rate, in both regulation and path following compared to the DQN controller. This performance, due to the continuous action space, suits dynamic navigation tasks like navigating curved vascular structures for practical applications.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Landing control of a magnetically actuated robotic catheter on beating heart surfaceScientific Reports, 2025
- Comparison of Classical, Neural Network and Hybrid Models for Hysteretic Single-Tendon Catheter KinematicsIEEE Robotics and Automation Letters, 2024
- A zero-shot reinforcement learning strategy for autonomous guidewire navigationInternational Journal of Computer Assisted Radiology and Surgery, 2024
- A Reinforcement Learning Approach for Continuum Robot ControlJournal of Intelligent & Robotic Systems, 2023
- Finite Element Analysis-Based Soft Robotic Modeling: Simulating a Soft Actuator in SOFAIEEE Robotics & Automation Magazine, 2023
- Metaheuristic algorithms for PID controller parameters tuning: review, approaches and open problemsHeliyon, 2022
- Continuum Robots for Medical InterventionsProceedings of the IEEE, 2022
- Hysteresis Modeling of Robotic Catheters Based on Long Short-Term Memory Network for Improved Environment ReconstructionIEEE Robotics and Automation Letters, 2021
- Untethered soft roboticsNature Electronics, 2018
- Model predictive control: past, present and futureComputers & Chemical Engineering, 1999