Sinan Ozgun Demir
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Sinan Ozgun Demir has a total of 37 co-authors. Here are the top co-authors by number of shared publications:
- Metin Sitti
Physical Intelligence Department, Max Planck Institute for Intelligent Systems, Stuttgart 70569, Germany
10 shared publications - Alp Can Karacakol
Physical Intelligence Department, Max Planck Institute for Intelligent Systems, Stuttgart 70569, Germany
3 shared publications - Zhiqiang Zheng
Department of Biomedical Engineering, City University of Hong Kong, Hong Kong SAR 999077, China
3 shared publications - Aude Bolopion
FEMTO-ST Institute, Université Bourgogne Franche-Comté, CNRS, 24 rue Savary, F-25000 Besançon, France
2 shared publications - Huaping Wang
Key Laboratory of Biomimetic Robots and Systems (Beijing Institute of Technology), Ministry of Education, Beijing 100081, China
2 shared publications - Mehmet Efe Tiryaki
Physical Intelligence Department, Max Planck Institute for Intelligent Systems, Stuttgart, Germany
2 shared publications
Latest Publications
- Open Access
- 23 February 2026
- Published by Springer Nature in Nature Machine Intelligence
- Vol. 8 (2) , 276-291
Synthetic X‑ray‑driven tracking and control of miniature medical devices
The clinical translation of miniature medical devices (MMDs) for minimally invasive surgery promises transformative advances in biomedical engineering, offering enhanced precision, reduced patient trauma and faster recovery times. However, their effective deployment in complex anatomies under real-time X-ray guidance—a widely used surgical imaging modality—presents challenges such as low imaging quality and difficulties of spatial MMD control. Manual identification and operation are labour intensive and error prone. Meanwhile, deep learning-based automation is limited by the scarcity of annotated X-ray datasets of MMDs owing to costly data collection, laborious annotation and privacy constraints. Here we introduce MicroSyn-X, a framework for training computer vision models to enable robotic teleoperation of MMDs using synthesized high-fidelity, pixel-accurate, auto-labelled and domain-randomized X-ray images, eliminating manual data curation. Integrating MicroSyn-X into a teleoperated robotic system enables real-time localization and navigation of magnetic soft and magnetic liquid MMDs within both ex vivo and dynamic in vivo environments, demonstrating robustness under challenging imaging conditions of low contrast, high noise and occlusion. With these promises, we open source the X-ray MMD dataset to enable benchmarking. Addressing data scarcity and enabling real-time robotic navigation, this work advances MMD-assisted minimally invasive surgery towards next-generation precision interventions.
- 23 February 2026
- Published by Springer Nature in Nature Machine Intelligence
- Vol. 8 (2) , 276-291
Open AccessSynthetic X‑ray‑driven tracking and control of miniature medical devices
The clinical translation of miniature medical devices (MMDs) for minimally invasive surgery promises transformative advances in biomedical engineering, offering enhanced precision, reduced patient trauma and faster recovery times. However, their effective deployment in complex anatomies under real-time X-ray guidance—a widely used surgical imaging modality—presents challenges such as low imaging quality and difficulties of spatial MMD control. Manual identification and operation are labour intensive and error prone. Meanwhile, deep learning-based automation is limited by the scarcity of annotated X-ray datasets of MMDs owing to costly data collection, laborious annotation and privacy constraints. Here we introduce MicroSyn-X, a framework for training computer vision models to enable robotic teleoperation of MMDs using synthesized high-fidelity, pixel-accurate, auto-labelled and domain-randomized X-ray images, eliminating manual data curation. Integrating MicroSyn-X into a teleoperated robotic system enables real-time localization and navigation of magnetic soft and magnetic liquid MMDs within both ex vivo and dynamic in vivo environments, demonstrating robustness under challenging imaging conditions of low contrast, high noise and occlusion. With these promises, we open source the X-ray MMD dataset to enable benchmarking. Addressing data scarcity and enabling real-time robotic navigation, this work advances MMD-assisted minimally invasive surgery towards next-generation precision interventions.
- Open Access
- 5 December 2025
- Published by American Association for the Advancement of Science (AAAS) in Science Advances
- Vol. 11 (49) , eadw9133
Ultrasoft hydrogel immune millirobot with multimodal locomotion
Advancements in cellular immunotherapy demanded efficient immune cell delivery. To meet this need, we introduced hydrogel-based immune millirobots designed for high immune cell loading and precise tumor targeting. These ultrasoft robots, embedded with magnetic nanoparticles, exhibited adaptable locomotion: walking, rolling, climbing, and undulating, enabling navigation through complex biological environments and alignment with varied tumor morphologies. They responded to magnetic fields and ionic or pH changes, facilitating propulsion, grasping, and localized delivery. In vitro, the millirobots eradicated three-dimensional tumor models in four days; in vivo, they notably reduced tumor growth in HepG2-luc tumor-bearing nude mice within 15 days. Bioluminescence imaging confirmed enhanced natural killer cell activity at tumor sites. The robots demonstrated excellent biocompatibility and biodegradability and caused no adverse effects postimplantation. This work showcased a responsive, soft robotic system with potential for advancing immune cell delivery and exploring tumor-immune dynamics in cancer therapy.
- 5 December 2025
- Published by American Association for the Advancement of Science (AAAS) in Science Advances
- Vol. 11 (49) , eadw9133
Open AccessUltrasoft hydrogel immune millirobot with multimodal locomotion
Advancements in cellular immunotherapy demanded efficient immune cell delivery. To meet this need, we introduced hydrogel-based immune millirobots designed for high immune cell loading and precise tumor targeting. These ultrasoft robots, embedded with magnetic nanoparticles, exhibited adaptable locomotion: walking, rolling, climbing, and undulating, enabling navigation through complex biological environments and alignment with varied tumor morphologies. They responded to magnetic fields and ionic or pH changes, facilitating propulsion, grasping, and localized delivery. In vitro, the millirobots eradicated three-dimensional tumor models in four days; in vivo, they notably reduced tumor growth in HepG2-luc tumor-bearing nude mice within 15 days. Bioluminescence imaging confirmed enhanced natural killer cell activity at tumor sites. The robots demonstrated excellent biocompatibility and biodegradability and caused no adverse effects postimplantation. This work showcased a responsive, soft robotic system with potential for advancing immune cell delivery and exploring tumor-immune dynamics in cancer therapy.
- Top Cited PapersOpen Access
- 26 March 2025
- Published by Springer Nature in Nature Communications
- Vol. 16 (1) , 1-13
Data-driven design of shape-programmable magnetic soft materials
Magnetically responsive soft materials with spatially-encoded magnetic and material properties enable versatile shape morphing for applications ranging from soft medical robots to biointerfaces. Although high-resolution encoding of 3D magnetic and material properties create a vast design space, their intrinsic coupling makes trial-and-error based design exploration infeasible. Here, we introduce a data-driven strategy that uses stochastic design alterations guided by a predictive neural network, combined with cost-efficient simulations, to optimize distributed magnetization profile and morphology of magnetic soft materials for desired shape-morphing and robotic behaviors. Our approach uncovers non-intuitive 2D designs that morph into complex 2D/3D structures and optimizes morphological behaviors, such as maximizing rotation or minimizing volume. We further demonstrate enhanced jumping performance over an intuitive reference design and showcase fabrication- and scale-agnostic, inherently 3D, multi-material soft structures for robotic tasks including traversing and jumping. This generic, data-driven framework enables efficient exploration of design space of stimuli-responsive soft materials, providing functional shape morphing and behavior for the next generation of soft robots and devices.
- 26 March 2025
- Published by Springer Nature in Nature Communications
- Vol. 16 (1) , 1-13
Top Cited PapersOpen AccessData-driven design of shape-programmable magnetic soft materials
Magnetically responsive soft materials with spatially-encoded magnetic and material properties enable versatile shape morphing for applications ranging from soft medical robots to biointerfaces. Although high-resolution encoding of 3D magnetic and material properties create a vast design space, their intrinsic coupling makes trial-and-error based design exploration infeasible. Here, we introduce a data-driven strategy that uses stochastic design alterations guided by a predictive neural network, combined with cost-efficient simulations, to optimize distributed magnetization profile and morphology of magnetic soft materials for desired shape-morphing and robotic behaviors. Our approach uncovers non-intuitive 2D designs that morph into complex 2D/3D structures and optimizes morphological behaviors, such as maximizing rotation or minimizing volume. We further demonstrate enhanced jumping performance over an intuitive reference design and showcase fabrication- and scale-agnostic, inherently 3D, multi-material soft structures for robotic tasks including traversing and jumping. This generic, data-driven framework enables efficient exploration of design space of stimuli-responsive soft materials, providing functional shape morphing and behavior for the next generation of soft robots and devices.
- Open Access
- 18 June 2024
- Published by Wiley in Advanced Science
- Vol. 11 (30) , e2308881
Learning Soft Millirobot Multimodal Locomotion with Sim‐to‐Real Transfer
With wireless multimodal locomotion capabilities, magnetic soft millirobots have emerged as potential minimally invasive medical robotic platforms. Due to their diverse shape programming capability, they can generate various locomotion modes, and their locomotion can be adapted to different environments by controlling the external magnetic field signal. Existing adaptation methods, however, are based on hand‐tuned signals. Here, a learning‐based adaptive magnetic soft millirobot multimodal locomotion framework empowered by sim‐to‐real transfer is presented. Developing a data‐driven magnetic soft millirobot simulation environment, the periodic magnetic actuation signal is learned for a given soft millirobot in simulation. Then, the learned locomotion strategy is deployed to the real world using Bayesian optimization and Gaussian processes. Finally, automated domain recognition and locomotion adaptation for unknown environments using a Kullback‐Leibler divergence‐based probabilistic method are illustrated. This method can enable soft millirobot locomotion to quickly and continuously adapt to environmental changes and explore the actuation space for unanticipated solutions with minimum experimental cost.
- 18 June 2024
- Published by Wiley in Advanced Science
- Vol. 11 (30) , e2308881
Open AccessLearning Soft Millirobot Multimodal Locomotion with Sim‐to‐Real Transfer
With wireless multimodal locomotion capabilities, magnetic soft millirobots have emerged as potential minimally invasive medical robotic platforms. Due to their diverse shape programming capability, they can generate various locomotion modes, and their locomotion can be adapted to different environments by controlling the external magnetic field signal. Existing adaptation methods, however, are based on hand‐tuned signals. Here, a learning‐based adaptive magnetic soft millirobot multimodal locomotion framework empowered by sim‐to‐real transfer is presented. Developing a data‐driven magnetic soft millirobot simulation environment, the periodic magnetic actuation signal is learned for a given soft millirobot in simulation. Then, the learned locomotion strategy is deployed to the real world using Bayesian optimization and Gaussian processes. Finally, automated domain recognition and locomotion adaptation for unknown environments using a Kullback‐Leibler divergence‐based probabilistic method are illustrated. This method can enable soft millirobot locomotion to quickly and continuously adapt to environmental changes and explore the actuation space for unanticipated solutions with minimum experimental cost.
- Open Access
- 21 March 2024
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 121 (13)
Single-step precision programming of decoupled multiresponsive soft millirobots
Stimuli-responsive soft robots offer new capabilities for the fields of medical and rehabilitation robotics, artificial intelligence, and soft electronics. Precisely programming the shape morphing and decoupling the multiresponsiveness of such robots is crucial to enable them with ample degrees of freedom and multifunctionality, while ensuring high fabrication accuracy. However, current designs featuring coupled multiresponsiveness or intricate assembly processes face limitations in executing complex transformations and suffer from a lack of precision. Therefore, we propose a one-stepped strategy to program multistep shape-morphing soft millirobots (MSSMs) in response to decoupled environmental stimuli. Our approach involves employing a multilayered elastomer and laser scanning technology to selectively process the structure of MSSMs, achieving a minimum machining precision of 30 μm. The resulting MSSMs are capable of imitating the shape morphing of plants and hand gestures and resemble kirigami, pop-up, and bistable structures. The decoupled multistimuli responsiveness of the MSSMs allows them to conduct shape morphing during locomotion, perform logic circuit control, and remotely repair circuits in response to humidity, temperature, and magnetic field. This strategy presents a paradigm for the effective design and fabrication of untethered soft miniature robots with physical intelligence, advancing the decoupled multiresponsive materials through modular tailoring of robotic body structures and properties to suit specific applications.
- 21 March 2024
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 121 (13)
Open AccessSingle-step precision programming of decoupled multiresponsive soft millirobots
Stimuli-responsive soft robots offer new capabilities for the fields of medical and rehabilitation robotics, artificial intelligence, and soft electronics. Precisely programming the shape morphing and decoupling the multiresponsiveness of such robots is crucial to enable them with ample degrees of freedom and multifunctionality, while ensuring high fabrication accuracy. However, current designs featuring coupled multiresponsiveness or intricate assembly processes face limitations in executing complex transformations and suffer from a lack of precision. Therefore, we propose a one-stepped strategy to program multistep shape-morphing soft millirobots (MSSMs) in response to decoupled environmental stimuli. Our approach involves employing a multilayered elastomer and laser scanning technology to selectively process the structure of MSSMs, achieving a minimum machining precision of 30 μm. The resulting MSSMs are capable of imitating the shape morphing of plants and hand gestures and resemble kirigami, pop-up, and bistable structures. The decoupled multistimuli responsiveness of the MSSMs allows them to conduct shape morphing during locomotion, perform logic circuit control, and remotely repair circuits in response to humidity, temperature, and magnetic field. This strategy presents a paradigm for the effective design and fabrication of untethered soft miniature robots with physical intelligence, advancing the decoupled multiresponsive materials through modular tailoring of robotic body structures and properties to suit specific applications.
- Open Access
- 16 December 2022
- Published by American Association for the Advancement of Science (AAAS) in Science Advances
- Vol. 8 (50) , eade6135
Programmable aniso-electrodeposited modular hydrogel microrobots
Systems with programmable and complex shape morphing are highly desired in many fields wherein sensing, actuation, and manipulation must be performed. Living organisms use nonuniform distributions of their body structural composition to achieve diverse shape morphing, motion, and functionality. However, for the microrobot fabrication, these designs often involve complicated robotic architectures requiring time-consuming and arduous fabrication processes. This paper proposes a single-step aniso-electrodeposition method for fabricating modular microrobots (MMRs) with distinct functions in each modular segment. By programming the electric field, the microscale stripe-shaped structure can be endowed with diverse shape-morphing capabilities, such as spiraling, twisting, bending, and coiling. The proposed fabrication method can develop MMRs with multiple independent modules onto which cells, drugs, and magnetic nanoparticles can be loaded to achieve multifunctionality. Thus, MMRs can perform multiple tasks, such as propulsion, grasping, and object delivery, simultaneously under magnetic control and ionic and pH stimuli.
- 16 December 2022
- Published by American Association for the Advancement of Science (AAAS) in Science Advances
- Vol. 8 (50) , eade6135
Open AccessProgrammable aniso-electrodeposited modular hydrogel microrobots
Systems with programmable and complex shape morphing are highly desired in many fields wherein sensing, actuation, and manipulation must be performed. Living organisms use nonuniform distributions of their body structural composition to achieve diverse shape morphing, motion, and functionality. However, for the microrobot fabrication, these designs often involve complicated robotic architectures requiring time-consuming and arduous fabrication processes. This paper proposes a single-step aniso-electrodeposition method for fabricating modular microrobots (MMRs) with distinct functions in each modular segment. By programming the electric field, the microscale stripe-shaped structure can be endowed with diverse shape-morphing capabilities, such as spiraling, twisting, bending, and coiling. The proposed fabrication method can develop MMRs with multiple independent modules onto which cells, drugs, and magnetic nanoparticles can be loaded to achieve multifunctionality. Thus, MMRs can perform multiple tasks, such as propulsion, grasping, and object delivery, simultaneously under magnetic control and ionic and pH stimuli.
- 1 June 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (3) , 6982-6989
Deep Learning-based 3D Magnetic Microrobot Tracking using 2D MR Images
Magnetic resonance imaging (MRI)-guided robots emerged as a promising tool for minimally invasive medical operations. Recently, MRI scanners have been proposed for actuating and localizing magnetic microrobots in the patient’s body using two-dimensional (2D) MR images. However, three-dimensional (3D) magnetic microrobots tracking during motion is still an untackled issue in MRI-powered microrobotics. Here, we present a deep learning-based 3D magnetic microrobot tracking method using 2D MR images during microrobot motion. The proposed method comprises a convolutional neural network (CNN) and complementary particle filter for 3D microrobot tracking. The CNN localizes the microrobot position relative to the 2D MRI slice and classifies the microrobot visibility in the MR images. First, we create an ultrasound (US) imaging-mentored MRI-based microrobot imaging and actuation system to train the CNN. Then, we trained the CNN using the MRI data generated by automated experiments using US image-based visual servoing of a microrobot with a $500\,{\mu }$ m-diameter magnetic core. We showed that the proposed CNN can localize the microrobot and classified its visibility in an in vitro environment with $\pm \mathbf {0.56}$ mm and $\mathbf {87.5}$ % accuracy in 2D MR images, respectively. Furthermore, we demonstrated ex-vivo 3D microrobot tracking with $\pm \mathbf {1.43}$ mm accuracy, improving tracking accuracy by 60% compared to the previous studies. The presented tracking strategy will enable MRI-powered microrobots to be used in high-precision targeted medical applications in the future.
- 1 June 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (3) , 6982-6989
Deep Learning-based 3D Magnetic Microrobot Tracking using 2D MR Images
Magnetic resonance imaging (MRI)-guided robots emerged as a promising tool for minimally invasive medical operations. Recently, MRI scanners have been proposed for actuating and localizing magnetic microrobots in the patient’s body using two-dimensional (2D) MR images. However, three-dimensional (3D) magnetic microrobots tracking during motion is still an untackled issue in MRI-powered microrobotics. Here, we present a deep learning-based 3D magnetic microrobot tracking method using 2D MR images during microrobot motion. The proposed method comprises a convolutional neural network (CNN) and complementary particle filter for 3D microrobot tracking. The CNN localizes the microrobot position relative to the 2D MRI slice and classifies the microrobot visibility in the MR images. First, we create an ultrasound (US) imaging-mentored MRI-based microrobot imaging and actuation system to train the CNN. Then, we trained the CNN using the MRI data generated by automated experiments using US image-based visual servoing of a microrobot with a $500\,{\mu }$ m-diameter magnetic core. We showed that the proposed CNN can localize the microrobot and classified its visibility in an in vitro environment with $\pm \mathbf {0.56}$ mm and $\mathbf {87.5}$ % accuracy in 2D MR images, respectively. Furthermore, we demonstrated ex-vivo 3D microrobot tracking with $\pm \mathbf {1.43}$ mm accuracy, improving tracking accuracy by 60% compared to the previous studies. The presented tracking strategy will enable MRI-powered microrobots to be used in high-precision targeted medical applications in the future.
- 28 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 27 (5) , 3506-3516
A Localization Method for Untethered Small-Scale Robots Using Electrical Impedance Tomography
Untethered small-scale robots can be potentially used in medical applications, such as minimally invasive surgeries and targeted drug delivery. This article introduces a new localization method using electrical impedance tomography, which is an emerging medical imaging technique, to dynamically track small-scale robots. The proposed approach provides the electrical conductivity distribution within the robot workspace from a set of electrical stimulations and voltage measurements gathered from eight electrodes placed at its boundary. The position of the robot can be deduced from the conductivity map that is reconstructed with the contrast in electrical properties between the robot and the background medium. This method is experimentally validated by successfully tracking the 2-D motion of four different magnetically actuated robots within a cylindrical arena (30 mm in diameter and 4.2 mm high). The smallest detected robot is 1.5 × 1.5 × 1 mm 3 . The proposed tracking method provides a noninvasive technology with low-cost and high-speed potential that would be significant and useful for the position feedback control of untethered devices for biomedical applications in the future.
- 28 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 27 (5) , 3506-3516
A Localization Method for Untethered Small-Scale Robots Using Electrical Impedance Tomography
Untethered small-scale robots can be potentially used in medical applications, such as minimally invasive surgeries and targeted drug delivery. This article introduces a new localization method using electrical impedance tomography, which is an emerging medical imaging technique, to dynamically track small-scale robots. The proposed approach provides the electrical conductivity distribution within the robot workspace from a set of electrical stimulations and voltage measurements gathered from eight electrodes placed at its boundary. The position of the robot can be deduced from the conductivity map that is reconstructed with the contrast in electrical properties between the robot and the background medium. This method is experimentally validated by successfully tracking the 2-D motion of four different magnetically actuated robots within a cylindrical arena (30 mm in diameter and 4.2 mm high). The smallest detected robot is 1.5 × 1.5 × 1 mm 3 . The proposed tracking method provides a noninvasive technology with low-cost and high-speed potential that would be significant and useful for the position feedback control of untethered devices for biomedical applications in the future.
- 14 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 2156-2161
Control and Transport of Passive Particles Using Self-Organized Spinning Micro-Disks
Traditional robotic systems have proven to be instrumental in object manipulation tasks for automated manufacturing processes. Object manipulation in such cases typically involves transport, pick-and-place and assembly of objects using automated conveyors and robotic arms. However, the forces at microscopic scales (e.g., surface tension, Van der Waals, electrostatic) can be qualitatively and quantitatively different from those at macroscopic scales. These forces render the release of objects difficult, and hence, traditional systems cannot be directly transferred to small scales (below a few millimeters). Consequently, novel micro-robotic manipulation systems have to be designed to take into account these scaling effects. Such systems could be beneficial for micro-fabrication processes and for biological studies. Here, we show autonomous position control of passive particles floating at the air-water interface using a collective of self-organized spinning micro-disks with a diameter of ${300}\,{\mathrm {\mu m}}$ . First, we show that the spinning micro-disks collectives generate azimuthal flows that cause passive particles to orbit around them. We then develop a closed-loop controller to demonstrate autonomous position control of passive particles without physical contact. Finally, we showcase the capability of our system to split from an expanded to several circular collectives while holding the particle at a fixed target. Our system’s contact-free object manipulation capability could be used for transporting delicate biological objects and for guiding self-assembly of passive objects for micro-fabrication.
- 14 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 2156-2161
Control and Transport of Passive Particles Using Self-Organized Spinning Micro-Disks
Traditional robotic systems have proven to be instrumental in object manipulation tasks for automated manufacturing processes. Object manipulation in such cases typically involves transport, pick-and-place and assembly of objects using automated conveyors and robotic arms. However, the forces at microscopic scales (e.g., surface tension, Van der Waals, electrostatic) can be qualitatively and quantitatively different from those at macroscopic scales. These forces render the release of objects difficult, and hence, traditional systems cannot be directly transferred to small scales (below a few millimeters). Consequently, novel micro-robotic manipulation systems have to be designed to take into account these scaling effects. Such systems could be beneficial for micro-fabrication processes and for biological studies. Here, we show autonomous position control of passive particles floating at the air-water interface using a collective of self-organized spinning micro-disks with a diameter of ${300}\,{\mathrm {\mu m}}$ . First, we show that the spinning micro-disks collectives generate azimuthal flows that cause passive particles to orbit around them. We then develop a closed-loop controller to demonstrate autonomous position control of passive particles without physical contact. Finally, we showcase the capability of our system to split from an expanded to several circular collectives while holding the particle at a fixed target. Our system’s contact-free object manipulation capability could be used for transporting delicate biological objects and for guiding self-assembly of passive objects for micro-fabrication.
- Open Access
- 16 June 2021
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 40 (12-14) , 1331-1351
Task space adaptation via the learning of gait controllers of magnetic soft millirobots
Untethered small-scale soft robots have promising applications in minimally invasive surgery, targeted drug delivery, and bioengineering applications as they can directly and non-invasively access confined and hard-to-reach spaces in the human body. For such potential biomedical applications, the adaptivity of the robot control is essential to ensure the continuity of the operations, as task environment conditions show dynamic variations that can alter the robot’s motion and task performance. The applicability of the conventional modeling and control methods is further limited for soft robots at the small-scale owing to their kinematics with virtually infinite degrees of freedom, inherent stochastic variability during fabrication, and changing dynamics during real-world interactions. To address the controller adaptation challenge to dynamically changing task environments, we propose using a probabilistic learning approach for a millimeter-scale magnetic walking soft robot using Bayesian optimization (BO) and Gaussian processes (GPs). Our approach provides a data-efficient learning scheme by finding the gait controller parameters while optimizing the stride length of the walking soft millirobot using a small number of physical experiments. To demonstrate the controller adaptation, we test the walking gait of the robot in task environments with different surface adhesion and roughness, and medium viscosity, which aims to represent the possible conditions for future robotic tasks inside the human body. We further utilize the transfer of the learned GP parameters among different task spaces and robots and compare their efficacy on the improvement of data-efficient controller learning.
- 16 June 2021
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 40 (12-14) , 1331-1351
Open AccessTask space adaptation via the learning of gait controllers of magnetic soft millirobots
Untethered small-scale soft robots have promising applications in minimally invasive surgery, targeted drug delivery, and bioengineering applications as they can directly and non-invasively access confined and hard-to-reach spaces in the human body. For such potential biomedical applications, the adaptivity of the robot control is essential to ensure the continuity of the operations, as task environment conditions show dynamic variations that can alter the robot’s motion and task performance. The applicability of the conventional modeling and control methods is further limited for soft robots at the small-scale owing to their kinematics with virtually infinite degrees of freedom, inherent stochastic variability during fabrication, and changing dynamics during real-world interactions. To address the controller adaptation challenge to dynamically changing task environments, we propose using a probabilistic learning approach for a millimeter-scale magnetic walking soft robot using Bayesian optimization (BO) and Gaussian processes (GPs). Our approach provides a data-efficient learning scheme by finding the gait controller parameters while optimizing the stride length of the walking soft millirobot using a small number of physical experiments. To demonstrate the controller adaptation, we test the walking gait of the robot in task environments with different surface adhesion and roughness, and medium viscosity, which aims to represent the possible conditions for future robotic tasks inside the human body. We further utilize the transfer of the learned GP parameters among different task spaces and robots and compare their efficacy on the improvement of data-efficient controller learning.