A Heuristically Accelerated Reinforcement Learning-Based Neurosurgical Path Planner
Authors:
Open Access
- 1 January 2023
- journal article
- research article
- Published by American Association for the Advancement of Science (AAAS) in Cyborg and Bionic Systems
- Vol. 4, 0026
Abstract
The steerable needle becomes appealing in the neurosurgery intervention procedure because of its flexibility to bypass critical regions inside the brain; with proper path planning, it can also minimize the potential damage by setting constraints and optimizing the insertion path. Recently, reinforcement learning (RL)-based path planning algorithm has shown promising results in neurosurgery, but because of the trial and error mechanism, it can be computationally expensive and insecure with low training efficiency. In this paper, we propose a heuristically accelerated deep Q network (DQN) algorithm to safely preoperatively plan a needle insertion path in a neurosurgical environment. Furthermore, a fuzzy inference system is integrated into the framework as a balance of the heuristic policy and the RL algorithm. Simulations are conducted to test the proposed method in comparison to the traditional greedy heuristic searching algorithm and DQN algorithms. Tests showed promising results of our algorithm in saving over 50 training episodes, calculating path lengths of 0.35 after normalization, which is 0.61 and 0.39 for DQN and traditional greedy heuristic searching algorithm, respectively. Moreover, the maximum curvature during planning is reduced to 0.046 from 0.139 mm −1 using the proposed algorithm compared to DQN.Keywords
This publication has 27 references indexed in Scilit:
- Deep Reinforcement Learning: A Brief SurveyIEEE Signal Processing Magazine, 2017
- Concentric Tube Robots: Rapid, Stable Path-Planning and Guidance for Surgical UseIEEE Robotics & Automation Magazine, 2017
- The role of automatic computer-aided surgical trajectory planning in improving the expected safety of stereotactic neurosurgeryInternational Journal of Computer Assisted Radiology and Surgery, 2014
- The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information RepositoryJournal of Digital Imaging, 2013
- Heuristically-Accelerated Multiagent Reinforcement LearningIEEE Transactions on Cybernetics, 2013
- 3D Slicer as an image computing platform for the Quantitative Imaging NetworkMagnetic Resonance Imaging, 2012
- Risk-based path planning for a steerable flexible probe for neurosurgical intervention2012
- Torsional dynamics compensation enhances robotic control of tip-steerable needles2012
- Modeling and Control of Needles With Torsional FrictionIEEE Transactions on Biomedical Engineering, 2009
- Accelerating autonomous learning by using heuristic selection of actionsJournal of Heuristics, 2007