Deep Reinforcement Learning Framework-Based Flow Rate Rejection Control of Soft Magnetic Miniature Robots
- 7 September 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 53 (12) , 7699-7711
Abstract
Soft magnetic miniature robots (SMMRs) have potential biomedical applications due to their flexible size and mobility to access confined environments. However, navigating the robot to a goal site with precise control performance and high repeatability in unstructured environments, especially in flow rate conditions, still remains a challenge. In this study, drawing inspiration from the control requirements of drug delivery and release to the goal lesion site in the presence of dynamic biofluids, we propose a flow rate rejection control strategy based on a deep reinforcement learning (DRL) framework to actuate an SMMR to achieve goal-reaching and hovering in fluidic tubes. To this end, an SMMR is first fabricated, which can be operated by an external magnetic field to realize its desired functionalities. Subsequently, a simulator is constructed based on neural networks to map the relationship between the applied magnetic field and robot locomotion states. With minimal prior knowledge about the environment and dynamics, a gated recurrent unit (GRU)-based DRL algorithm is formulated by considering the designed history state–action and estimated flow rates. In addition, the randomization technique is applied during training to distill the general control policy for the physical SMMR. The results of numerical simulations and experiments are illustrated to demonstrate the robustness and efficacy of the presented control framework. Finally, in-depth analyses and discussions indicate the potentiality of DRL for soft magnetic robots in biomedical applications.Keywords
Funding Information
- Croucher Foundation (CAS20403)
- Hong Kong RGC (RFS2122-4S03)
- CUHK internal grants
- Fundamental Research Funds for the Central Universities (2242022R10051)
- Natural Science Foundation of Jiangsu Province (BK20220834)
This publication has 38 references indexed in Scilit:
- Direction Priority Control Method for Magnetic Manipulation System in Current and Voltage LimitsIEEE Transactions on Industrial Electronics, 2016
- Quantitative blood flow velocity imaging using laser speckle flowmetryScientific Reports, 2016
- Planar Path Following of 3-D Steering Scaled-Up Helical MicroswimmersIEEE Transactions on Robotics, 2015
- Magnetic-Based Motion Control of Paramagnetic Microparticles With Disturbance CompensationIEEE Transactions on Magnetics, 2014
- Electromagnetic Steering of a Magnetic Cylindrical Microrobot Using Optical Feedback Closed-Loop ControlInternational Journal of Optomechatronics, 2014
- Data-Based Techniques Focused on Modern Industry: An OverviewIEEE Transactions on Industrial Electronics, 2014
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- On the Properties of Neural Machine Translation: Encoder–Decoder Approaches2014
- OctoMag: An Electromagnetic System for 5-DOF Wireless MicromanipulationIEEE Transactions on Robotics, 2010
- Microrobots for Minimally Invasive MedicineAnnual Review of Biomedical Engineering, 2010