CASOG: Conservative Actor–Critic With SmOoth Gradient for Skill Learning in Robot-Assisted Intervention
- 18 September 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 71 (7) , 7722-7731
Abstract
The robot-assisted intervention has shown reduced radiation exposure to physicians and improved precision in clinical trials. However, existing vascular robotic systems follow master-slave control mode and entirely rely on manual commands. This article proposes a novel offline reinforcement learning algorithm, Conservative Actor–critic with SmOoth Gradient (CASOG), to learn manipulation skills on vascular robotic systems. The proposed algorithm conservatively estimates Q-function and smooths gradients of convolution layers to deal with distribution shift and overfitting issues. Furthermore, to focus on complex manipulations, transitions with larger absolute temporal-difference error are sampled with higher probability. Comparative experiments on multiple vascular models and offline data demonstrate that CASOG delivers guidewire to the target with higher success rates and fewer backward steps than prior offline reinforcement learning methods. These results indicate that the proposed algorithm is promising to improve the autonomy of vascular robotic systems.Keywords
Funding Information
- National Natural Science Foundation of China (62003343, 62222316, U1913601, 62073325, U20A20224, U1913210)
- Beijing Natural Science Foundation (M22008)
- Youth Innovation Promotion Association of the Chinese Academy of Sciences (2020140)
- CIE-Tencent Robotics X Rhino-Bird Focused Research Program
This publication has 23 references indexed in Scilit:
- RepVGG: Making VGG-style ConvNets Great Again2021
- A Real-Time Multi-Task Framework for Guidewire Segmentation and Endpoint Localization in Endovascular Interventions2021
- Collaborative Robot-Assisted Endovascular Catheterization with Generative Adversarial Imitation Learning2020
- The Global Burden of Cardiovascular Diseases and Risk FactorsJACC, 2019
- A Novel Robot-Assisted Endovascular Catheterization System With Haptic Force FeedbackIEEE Transactions on Robotics, 2019
- UNet++: A Nested U-Net Architecture for Medical Image Segmentation2018
- Radiation‐associated lens changes in the cardiac catheterization laboratory: Results from the IC‐CATARACT (CATaracts Attributed to RAdiation in the CaTh lab) studyCatheterization and Cardiovascular Interventions, 2017
- Occupational health hazards of interventional cardiologists in the current decade: Results of the 2014 SCAI membership surveyCatheterization and Cardiovascular Interventions, 2015
- Current and Emerging Robot-Assisted Endovascular Catheterization Technologies: A ReviewAnnals of Biomedical Engineering, 2013
- First-in-Human Evaluation of a Novel Robotic-Assisted Coronary Angioplasty SystemJACC: Cardiovascular Interventions, 2011