Hiroyuki Ishii
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Hiroyuki Ishii has a total of 86 co-authors. Here are the top co-authors by number of shared publications:
- Haitham El-Hussieny
Department of Mechatronics and Robotics, School of Innovative Design Engineering, Egypt-Japan University of Science and Technology (E-JUST), New Borg El-Arab, Alexandria 21934, Egypt
8 shared publications - Atsuo Takanishi
Waseda University
7 shared publications - Yuki Satake
Department of Robotics, Ritsumeikan University, Kusatsu 525-8577, Shiga, Japan
6 shared publications - Ayman A. Nada
School of Innovative Design Engineering, Faculty of Engineering, Egypt-Japan University of Science and Technology E-JUST, Alexandria, Egypt
5 shared publications - Qing Shi
Intelligent Robotics Institute, School of Mechatronical Engineering, Beijing Institute of Technology, Beijing 100081, China
3 shared publications Shinichiro Konno
Waseda University, Tokyo, Japan
3 shared publications
Latest Publications
- 7 October 2026
- Published by IOP Publishing in Bioinspiration & Biomimetics
- Vol. 21 (5) , 056021
Engineering analysis on periodic pedal motion of clams during foot penetration
Clams are bivalve species that burrow into substrate using a characteristic organ termed ‘foot’. Their burrowing process involves a repetitive sequence of movements termed the digging cycle. This study focuses on the initial phase of the digging cycle: foot penetration into substrate, as a step toward clarifying the burrowing mechanism from robotics and soil mechanics perspectives. We proposed that the periodic pedal motion plays a key role in facilitating foot penetration through reduction of substrate resistance. To identify the essential characteristics, this motion was decomposed into three directional components focusing on the physical interaction between the biological structure and the substrate. To examine these components, two hypotheses were proposed: firstly, that longitudinal substrate agitation facilitates foot penetration; and secondly, that lateral and transverse agitation contributes to this process. Three experiments using 3D-printed foot models were conducted to test these hypotheses, defining the penetration (longitudinal) direction as vertically downward. The first experiment examined the effects of longitudinal agitation by measuring vertical resistive forces during penetration into saturated sand, both with and without vertical reciprocating motion. The second experiment evaluated the effects of the lateral and transverse agitation by measuring the penetration displacement of the model resulting from the application of horizontal reciprocating motion under a constant vertical load. Finally, the third experiment further tested the second hypothesis to examine the effect of lateral agitation under conditions that more closely mimic a biological foot, by incorporating a deformable foot model into the experimental setup of the second experiment. The results indicate that lateral motion is a key factor in facilitating foot penetration under reduced force, whereas the longitudinal component contributes little when acting alone. Based on fundamental theories in soil mechanics, the primary mechanism of this effect was attributed to a reduction in effective stress caused by factors such as liquefaction.
- 7 October 2026
- Published by IOP Publishing in Bioinspiration & Biomimetics
- Vol. 21 (5) , 056021
Engineering analysis on periodic pedal motion of clams during foot penetration
Clams are bivalve species that burrow into substrate using a characteristic organ termed ‘foot’. Their burrowing process involves a repetitive sequence of movements termed the digging cycle. This study focuses on the initial phase of the digging cycle: foot penetration into substrate, as a step toward clarifying the burrowing mechanism from robotics and soil mechanics perspectives. We proposed that the periodic pedal motion plays a key role in facilitating foot penetration through reduction of substrate resistance. To identify the essential characteristics, this motion was decomposed into three directional components focusing on the physical interaction between the biological structure and the substrate. To examine these components, two hypotheses were proposed: firstly, that longitudinal substrate agitation facilitates foot penetration; and secondly, that lateral and transverse agitation contributes to this process. Three experiments using 3D-printed foot models were conducted to test these hypotheses, defining the penetration (longitudinal) direction as vertically downward. The first experiment examined the effects of longitudinal agitation by measuring vertical resistive forces during penetration into saturated sand, both with and without vertical reciprocating motion. The second experiment evaluated the effects of the lateral and transverse agitation by measuring the penetration displacement of the model resulting from the application of horizontal reciprocating motion under a constant vertical load. Finally, the third experiment further tested the second hypothesis to examine the effect of lateral agitation under conditions that more closely mimic a biological foot, by incorporating a deformable foot model into the experimental setup of the second experiment. The results indicate that lateral motion is a key factor in facilitating foot penetration under reduced force, whereas the longitudinal component contributes little when acting alone. Based on fundamental theories in soil mechanics, the primary mechanism of this effect was attributed to a reduction in effective stress caused by factors such as liquefaction.
- Open Access
A Parametric Multi-Bladed Continuum Robot Arm: Design-Family Construction and Locking-Free ANCF Static Modeling
Continuum robot arms offer compliance and confined-space dexterity that rigid-link manipulators cannot match, but each published design is built around a fixed structural configuration serving a single motion type, so that finger-like, planar, and spatial motion each demand a separate design and model. This work introduces a tendon-driven, multi-bladed continuum robot arm in which the blade count is a design parameter, together with a procedure that assembles the corresponding multibody model automatically for any blade count. Static response is modeled in the absolute nodal coordinate formulation using the fully parameterized BE24 beam element with the strain-split method, which alleviates the Poisson locking the element otherwise exhibits for the near-incompressible material considered. The static model characterizes force-displacement response, stiffness modulation by axial preload, and the reachable set of each variant. Blade count governs both the compliance of the arm and the dimensionality of the reachable set—a curve, a planar region, and a spatial surface for one, two, and three blades—and the three-blade set is trilobed rather than circular, reflecting pull-only tendon actuation. One construction and one modeling procedure thus span finger-like, planar, and spatial manipulators, and form the parametric model base for subsequent optimal design, targeted experimental validation, and inverse kinematics.
Open AccessA Parametric Multi-Bladed Continuum Robot Arm: Design-Family Construction and Locking-Free ANCF Static Modeling
Continuum robot arms offer compliance and confined-space dexterity that rigid-link manipulators cannot match, but each published design is built around a fixed structural configuration serving a single motion type, so that finger-like, planar, and spatial motion each demand a separate design and model. This work introduces a tendon-driven, multi-bladed continuum robot arm in which the blade count is a design parameter, together with a procedure that assembles the corresponding multibody model automatically for any blade count. Static response is modeled in the absolute nodal coordinate formulation using the fully parameterized BE24 beam element with the strain-split method, which alleviates the Poisson locking the element otherwise exhibits for the near-incompressible material considered. The static model characterizes force-displacement response, stiffness modulation by axial preload, and the reachable set of each variant. Blade count governs both the compliance of the arm and the dimensionality of the reachable set—a curve, a planar region, and a spatial surface for one, two, and three blades—and the three-blade set is trilobed rather than circular, reflecting pull-only tendon actuation. One construction and one modeling procedure thus span finger-like, planar, and spatial manipulators, and form the parametric model base for subsequent optimal design, targeted experimental validation, and inverse kinematics.
- Open Access
Robust Adaptive Koopman MPC Under Structured and Stochastic Uncertainty for Soft Continuum Robots
Soft continuum robots exhibit highly nonlinear and configuration-dependent dynamics, making accurate trajectory tracking challenging under model uncertainty and external disturbances. This paper presents an adaptive Koopman-based model predictive control (MPC) framework for a tendon-driven soft continuum robot and evaluates its performance through comprehensive closed-loop simulations. A lifted linear Koopman model is identified from experimentally collected robot data and incorporated into an MPC formulation to provide computationally efficient prediction while capturing dominant nonlinear behavior. To compensate for plant–model mismatch and time-varying uncertainties, an online Recursive Least Squares (RLS) adaptation mechanism is integrated into the Koopman–MPC framework, enabling continuous model refinement during closed-loop operation without repeated offline retraining. The proposed controller is evaluated through comprehensive closed-loop simulations on circular, triangular, helical, and figure-eight trajectories under stochastic disturbances and structured parametric bias conditions using a Koopman model identified from experimentally collected robot data. Results demonstrate consistent improvements in tracking performance compared with fixed Koopman MPC while maintaining real-time computational feasibility. Under structured parametric bias, the proposed controller reduces the mean tracking error from 9.30 mm to 1.36 mm during circular trajectory tracking and achieves sub-millimeter accuracy in several operating conditions. These findings highlight the potential of online Koopman model adaptation for predictive control of soft continuum robots operating under uncertainty.
Open AccessRobust Adaptive Koopman MPC Under Structured and Stochastic Uncertainty for Soft Continuum Robots
Soft continuum robots exhibit highly nonlinear and configuration-dependent dynamics, making accurate trajectory tracking challenging under model uncertainty and external disturbances. This paper presents an adaptive Koopman-based model predictive control (MPC) framework for a tendon-driven soft continuum robot and evaluates its performance through comprehensive closed-loop simulations. A lifted linear Koopman model is identified from experimentally collected robot data and incorporated into an MPC formulation to provide computationally efficient prediction while capturing dominant nonlinear behavior. To compensate for plant–model mismatch and time-varying uncertainties, an online Recursive Least Squares (RLS) adaptation mechanism is integrated into the Koopman–MPC framework, enabling continuous model refinement during closed-loop operation without repeated offline retraining. The proposed controller is evaluated through comprehensive closed-loop simulations on circular, triangular, helical, and figure-eight trajectories under stochastic disturbances and structured parametric bias conditions using a Koopman model identified from experimentally collected robot data. Results demonstrate consistent improvements in tracking performance compared with fixed Koopman MPC while maintaining real-time computational feasibility. Under structured parametric bias, the proposed controller reduces the mean tracking error from 9.30 mm to 1.36 mm during circular trajectory tracking and achieves sub-millimeter accuracy in several operating conditions. These findings highlight the potential of online Koopman model adaptation for predictive control of soft continuum robots operating under uncertainty.
- Open Access
Dynamic Modeling and Model Predictive Control of Soft Growing Robot for Safe and Assisted Patient Repositioning
The growing demand for elderly and bedridden patient care in hospitals, nursing homes, and long-term care facilities has increased the need for safe and efficient repositioning methods. Repositioning immobile patients is essential for preventing pressure injuries and other complications associated with prolonged immobility. However, this task is still commonly performed manually using bed sheets, pillows, and similar support aids, making it physically demanding and increasing the risk of musculoskeletal injury among caregivers. This paper presents a two-stage soft growing robot for safe and assisted patient repositioning from a supine posture to a side-lying position. The proposed mechanism consists of two soft pneumatic chambers with distinct roles. The first chamber enables pressure-driven eversion, allowing the robot to deploy smoothly beneath the patient with minimal friction. The second chamber is then pressurized to generate the lifting and rolling motion required for repositioning. A first-principles dynamic model of the pressure-driven vine robot is developed by integrating pneumatic supply dynamics, internal pressure evolution, and tip-extension mechanics within a Lagrangian framework. Based on this model, a robust multi-stage nonlinear model predictive control strategy is formulated to regulate deployment beneath the patient under parameter uncertainty. The rolling dynamics of the second stage are also analyzed to determine the minimum pressure required for repositioning as a function of patient weight and roll angle. Simulation results show that the proposed controller achieves smooth and accurate deployment while satisfying input and state constraints under uncertainty. The rolling analysis further indicates that the required pressure increases with patient weight and decreases with roll angle. These findings demonstrate the potential of the proposed mechanism to reduce caregiver effort and enable safe, controlled patient repositioning.
Open AccessDynamic Modeling and Model Predictive Control of Soft Growing Robot for Safe and Assisted Patient Repositioning
The growing demand for elderly and bedridden patient care in hospitals, nursing homes, and long-term care facilities has increased the need for safe and efficient repositioning methods. Repositioning immobile patients is essential for preventing pressure injuries and other complications associated with prolonged immobility. However, this task is still commonly performed manually using bed sheets, pillows, and similar support aids, making it physically demanding and increasing the risk of musculoskeletal injury among caregivers. This paper presents a two-stage soft growing robot for safe and assisted patient repositioning from a supine posture to a side-lying position. The proposed mechanism consists of two soft pneumatic chambers with distinct roles. The first chamber enables pressure-driven eversion, allowing the robot to deploy smoothly beneath the patient with minimal friction. The second chamber is then pressurized to generate the lifting and rolling motion required for repositioning. A first-principles dynamic model of the pressure-driven vine robot is developed by integrating pneumatic supply dynamics, internal pressure evolution, and tip-extension mechanics within a Lagrangian framework. Based on this model, a robust multi-stage nonlinear model predictive control strategy is formulated to regulate deployment beneath the patient under parameter uncertainty. The rolling dynamics of the second stage are also analyzed to determine the minimum pressure required for repositioning as a function of patient weight and roll angle. Simulation results show that the proposed controller achieves smooth and accurate deployment while satisfying input and state constraints under uncertainty. The rolling analysis further indicates that the required pressure increases with patient weight and decreases with roll angle. These findings demonstrate the potential of the proposed mechanism to reduce caregiver effort and enable safe, controlled patient repositioning.
- Open Access
Novel Design of a Soft–Rigid Hybrid Pneumatic Actuator Incorporating a Spine-like Internal Structure
Soft pneumatic actuators (SPAs) are widely used in robotic systems due to their inherent compliance and safety during human–robot interaction. However, their intrinsic softness often leads to insufficient stiffness and a low load-bearing capacity, which limit their applicability. In this work, a novel soft–rigid hybrid pneumatic actuator incorporating a spine-like internal structure is proposed to enhance the effective stiffness while preserving bending flexibility. Inspired by the biomechanical structure of the human spine, the embedded spine-like structure consists of interconnected rigid vertebrae integrated along the central axis of a soft pneumatic actuator. Static bending experiments under different base orientations and external loads are conducted to evaluate the actuator’s performance. The experimental results demonstrate that the proposed actuator exhibits improved posture retention, enhanced load-bearing capacity, and higher robustness against gravitational loading compared to a soft pneumatic actuator without a spine-like structure. These results confirm that the spine-like internal structure effectively increases the actuator’s effective stiffness, enabling stable bending behavior under various working conditions.
Open AccessNovel Design of a Soft–Rigid Hybrid Pneumatic Actuator Incorporating a Spine-like Internal Structure
Soft pneumatic actuators (SPAs) are widely used in robotic systems due to their inherent compliance and safety during human–robot interaction. However, their intrinsic softness often leads to insufficient stiffness and a low load-bearing capacity, which limit their applicability. In this work, a novel soft–rigid hybrid pneumatic actuator incorporating a spine-like internal structure is proposed to enhance the effective stiffness while preserving bending flexibility. Inspired by the biomechanical structure of the human spine, the embedded spine-like structure consists of interconnected rigid vertebrae integrated along the central axis of a soft pneumatic actuator. Static bending experiments under different base orientations and external loads are conducted to evaluate the actuator’s performance. The experimental results demonstrate that the proposed actuator exhibits improved posture retention, enhanced load-bearing capacity, and higher robustness against gravitational loading compared to a soft pneumatic actuator without a spine-like structure. These results confirm that the spine-like internal structure effectively increases the actuator’s effective stiffness, enabling stable bending behavior under various working conditions.
- Open Access
Multibody Based Parameter Estimation of Stewart Platform Using Particles Swarm Optimization
Parameter estimation plays an important role in improving the accuracy, control, and diagnostic performance of mechanisms, particularly in parallel mechanisms such as the Stewart platform, which are increasingly used in high-precision automation, advanced manufacturing, and machine-centric applications. This paper presents a multibody–based framework for generalized dynamic modeling and inertial parameter estimation of parallel robotic manipulators, demonstrated on the DeltaLab-SMT EX800 Stewart platform. A systematic constrained multibody dynamic formulation is developed using an iterative kinematic–dynamic coupling scheme to compute generalized coordinates and their time derivatives under prescribed motion trajectories. The proposed identification manifold is experimentally validated on the physical test rig, in which the platform motion is executed via the control/DAQ system, while inertial measurements are acquired using an external 6-axis motion sensor to obtain direct acceleration data from the moving platform. Platform acceleration measurements are mapped through the inverse dynamics of the multibody model to derive the corresponding generalized forces, providing a practical and cost-effective alternative to direct force measurement with transducers. A Kalman filter is subsequently employed to combine the measured and the model-predicted data, yielding optimally filtered estimates of the inertial coordinates for accurate parameter identification. Inertial parameters are estimated using particle swarm optimization and bench marked against a gradient-based Levenberg–Marquardt approach, with comparison in terms of convergence behavior, robustness, and estimation accuracy. The results support the proposed framework as a measurement-informed benchmark methodology for parameter estimation of parallel manipulators.
Open AccessMultibody Based Parameter Estimation of Stewart Platform Using Particles Swarm Optimization
Parameter estimation plays an important role in improving the accuracy, control, and diagnostic performance of mechanisms, particularly in parallel mechanisms such as the Stewart platform, which are increasingly used in high-precision automation, advanced manufacturing, and machine-centric applications. This paper presents a multibody–based framework for generalized dynamic modeling and inertial parameter estimation of parallel robotic manipulators, demonstrated on the DeltaLab-SMT EX800 Stewart platform. A systematic constrained multibody dynamic formulation is developed using an iterative kinematic–dynamic coupling scheme to compute generalized coordinates and their time derivatives under prescribed motion trajectories. The proposed identification manifold is experimentally validated on the physical test rig, in which the platform motion is executed via the control/DAQ system, while inertial measurements are acquired using an external 6-axis motion sensor to obtain direct acceleration data from the moving platform. Platform acceleration measurements are mapped through the inverse dynamics of the multibody model to derive the corresponding generalized forces, providing a practical and cost-effective alternative to direct force measurement with transducers. A Kalman filter is subsequently employed to combine the measured and the model-predicted data, yielding optimally filtered estimates of the inertial coordinates for accurate parameter identification. Inertial parameters are estimated using particle swarm optimization and bench marked against a gradient-based Levenberg–Marquardt approach, with comparison in terms of convergence behavior, robustness, and estimation accuracy. The results support the proposed framework as a measurement-informed benchmark methodology for parameter estimation of parallel manipulators.
- Open Access
- 29 December 2025
- Published by Universidad Tecnológica Centroamericana in Innovare: Revista de ciencia y tecnología
- Vol. 14 (2) , 1-4
Gobernanza de la IA: el proceso de IA de Hiroshima comparado con los avances de Honduras y América Latina
Introducción. El rápido avance de la inteligencia artificial (IA) ha transformado las industrias y las sociedades a nivel mundial, lo que ha hecho necesario contar con marcos de gobernanza sólidos para gestionar su desarrollo de manera responsable. Presentación de caso. En este documento se comparan el Proceso de Hiroshima sobre IA del G7 y la Política Nacional de Inteligencia Artificial de Chile, destacando sus enfoques distintos y sus objetivos comunes. El Proceso de Hiroshima sobre IA, puesto en marcha durante la Cumbre del G7 de 2023, hace hincapié en la colaboración mundial, las directrices éticas y un marco centrado en el ser humano y basado en el riesgo. Por el contrario, la gobernanza de Chile se centra en las prioridades nacionales, con un plan de acción decenal y una legislación para integrar la IA en su tejido socioeconómico. Discusión. Ambos marcos abordan la transparencia, la rendición de cuentas y las consideraciones éticas, aunque a diferentes escalas. Mientras que el Proceso de Hiroshima sobre IA establece un estándar global para la interoperabilidad y la gobernanza compartida, los esfuerzos de Chile ejemplifican la implementación localizada con colaboración internacional. Conclusión. En conjunto, estas iniciativas subrayan la necesidad de estrategias diversas para navegar eficazmente por las complejidades de la IA, lo que en el futuro podría servir de base al Gobierno hondureño para establecer o adoptar medidas existentes para la gobernanza de la IA.
- 29 December 2025
- Published by Universidad Tecnológica Centroamericana in Innovare: Revista de ciencia y tecnología
- Vol. 14 (2) , 1-4
Open AccessGobernanza de la IA: el proceso de IA de Hiroshima comparado con los avances de Honduras y América Latina
Introducción. El rápido avance de la inteligencia artificial (IA) ha transformado las industrias y las sociedades a nivel mundial, lo que ha hecho necesario contar con marcos de gobernanza sólidos para gestionar su desarrollo de manera responsable. Presentación de caso. En este documento se comparan el Proceso de Hiroshima sobre IA del G7 y la Política Nacional de Inteligencia Artificial de Chile, destacando sus enfoques distintos y sus objetivos comunes. El Proceso de Hiroshima sobre IA, puesto en marcha durante la Cumbre del G7 de 2023, hace hincapié en la colaboración mundial, las directrices éticas y un marco centrado en el ser humano y basado en el riesgo. Por el contrario, la gobernanza de Chile se centra en las prioridades nacionales, con un plan de acción decenal y una legislación para integrar la IA en su tejido socioeconómico. Discusión. Ambos marcos abordan la transparencia, la rendición de cuentas y las consideraciones éticas, aunque a diferentes escalas. Mientras que el Proceso de Hiroshima sobre IA establece un estándar global para la interoperabilidad y la gobernanza compartida, los esfuerzos de Chile ejemplifican la implementación localizada con colaboración internacional. Conclusión. En conjunto, estas iniciativas subrayan la necesidad de estrategias diversas para navegar eficazmente por las complejidades de la IA, lo que en el futuro podría servir de base al Gobierno hondureño para establecer o adoptar medidas existentes para la gobernanza de la IA.
- Open Access
- 6 November 2025
- Published by MDPI AG in Journal of Imaging
- Vol. 11 (11) , 395
Multi-Weather DomainShifter: A Comprehensive Multi-Weather Transfer LLM Agent for Handling Domain Shift in Aerial Image Processing
Recent deep learning-based remote sensing analysis models often struggle with performance degradation due to domain shifts caused by illumination variations (clear to overcast), changing atmospheric conditions (clear to foggy, dusty), and physical scene changes (clear to snowy). Addressing domain shift in aerial image segmentation is challenging due to limited training data availability, including costly data collection and annotation. We propose Multi-Weather DomainShifter, a comprehensive multi-weather domain transfer system that augments single-domain images into various weather conditions without additional laborious annotation, coordinated by a large language model (LLM) agent. Specifically, we utilize Unreal Engine to construct a synthetic dataset featuring images captured under diverse conditions such as overcast, foggy, and dusty settings. We then propose a latent space style transfer model that generates alternate domain versions based on real aerial datasets. Additionally, we present a multi-modal snowy scene diffusion model with LLM-assisted scene descriptors to add snowy elements into scenes. Multi-weather DomainShifter integrates these two approaches into a tool library and leverages the agent for tool selection and execution. Extensive experiments on the ISPRS Vaihingen and Potsdam dataset demonstrate that domain shift caused by weather change in aerial image-leads to significant performance drops, then verify our proposal’s capacity to adapt models to perform well in shifted domains while maintaining their effectiveness in the original domain.
- 6 November 2025
- Published by MDPI AG in Journal of Imaging
- Vol. 11 (11) , 395
Open AccessMulti-Weather DomainShifter: A Comprehensive Multi-Weather Transfer LLM Agent for Handling Domain Shift in Aerial Image Processing
Recent deep learning-based remote sensing analysis models often struggle with performance degradation due to domain shifts caused by illumination variations (clear to overcast), changing atmospheric conditions (clear to foggy, dusty), and physical scene changes (clear to snowy). Addressing domain shift in aerial image segmentation is challenging due to limited training data availability, including costly data collection and annotation. We propose Multi-Weather DomainShifter, a comprehensive multi-weather domain transfer system that augments single-domain images into various weather conditions without additional laborious annotation, coordinated by a large language model (LLM) agent. Specifically, we utilize Unreal Engine to construct a synthetic dataset featuring images captured under diverse conditions such as overcast, foggy, and dusty settings. We then propose a latent space style transfer model that generates alternate domain versions based on real aerial datasets. Additionally, we present a multi-modal snowy scene diffusion model with LLM-assisted scene descriptors to add snowy elements into scenes. Multi-weather DomainShifter integrates these two approaches into a tool library and leverages the agent for tool selection and execution. Extensive experiments on the ISPRS Vaihingen and Potsdam dataset demonstrate that domain shift caused by weather change in aerial image-leads to significant performance drops, then verify our proposal’s capacity to adapt models to perform well in shifted domains while maintaining their effectiveness in the original domain.
- Open Access
- 16 September 2025
- Published by MDPI AG in Preprints.org
Multi-Weather DomainShifter: A Comprehensive Multi-Weather Transfer LLM Agent for Handling Domain Shift in Aerial Image Processing
Recent deep learning-based remote sensing analysis models often struggle with performance degradation due to domain shifts caused by illumination variations (clear to overcast), changing atmospheric conditions (clear to foggy, dusty), and physical scene changes (clear to snowy). Addressing domain shift in aerial image segmentation is challenging due to limited training data availability, including costly data collection and annotation. We propose Multi-Weather DomainShifter, a comprehensive multi-weather domain transfer system that augments single-domain images into various weather conditions without additional laborious annotation, coordinated by a large language model (LLM) agent. Specifically, we utilize Unreal Engine to construct a synthetic dataset featuring images captured under diverse conditions such as overcast, foggy, and dusty settings. We then propose a latent space style transfer model that generates alternate domain versions based on real aerial datasets. Additionally, we present a multi-modal snowy scene diffusion model with LLM-assisted scene descriptors to add snowy elements into scenes. Multiweather DomainShifter integrates these two approaches into a tool library and leverages the agent for tool selection and execution. Extensive experiments on the ISPRS Vaihingen and Potsdam dataset demonstrate that domain shift caused by weather change in aerial image-leads to significant performance drops, then verify our proposal’s capacity to adapt models to perform well in shifted domains while maintaining their effectiveness in the original domain. The code is available at https://github.com/WayBob/domainshifter.
- 16 September 2025
- Published by MDPI AG in Preprints.org
Open AccessMulti-Weather DomainShifter: A Comprehensive Multi-Weather Transfer LLM Agent for Handling Domain Shift in Aerial Image Processing
Recent deep learning-based remote sensing analysis models often struggle with performance degradation due to domain shifts caused by illumination variations (clear to overcast), changing atmospheric conditions (clear to foggy, dusty), and physical scene changes (clear to snowy). Addressing domain shift in aerial image segmentation is challenging due to limited training data availability, including costly data collection and annotation. We propose Multi-Weather DomainShifter, a comprehensive multi-weather domain transfer system that augments single-domain images into various weather conditions without additional laborious annotation, coordinated by a large language model (LLM) agent. Specifically, we utilize Unreal Engine to construct a synthetic dataset featuring images captured under diverse conditions such as overcast, foggy, and dusty settings. We then propose a latent space style transfer model that generates alternate domain versions based on real aerial datasets. Additionally, we present a multi-modal snowy scene diffusion model with LLM-assisted scene descriptors to add snowy elements into scenes. Multiweather DomainShifter integrates these two approaches into a tool library and leverages the agent for tool selection and execution. Extensive experiments on the ISPRS Vaihingen and Potsdam dataset demonstrate that domain shift caused by weather change in aerial image-leads to significant performance drops, then verify our proposal’s capacity to adapt models to perform well in shifted domains while maintaining their effectiveness in the original domain. The code is available at https://github.com/WayBob/domainshifter.
- Open Access
- 1 January 2025
- Published by Walter de Gruyter GmbH in Nonlinear Engineering
- Vol. 14 (1)
Real-time force/position control of soft growing robots: A data-driven model predictive approach
Vine robots represent a novel class of soft robots that achieve mobility through tip extension, a mechanism inspired by the natural growth processes of vine plants. This unique movement strategy enables effective navigation in constrained and cluttered environments, offering significant advantages over conventional robotic systems. However, the continuum nature and inherent compliance of vine robots introduce complex modeling and control challenges. Deep learning offer a powerful alternative for modeling systems with such complex dynamics. In this article, we present a data-driven dynamic model for a pressure-driven everting vine robot, utilizing a deep neural network (DNN)-based discrete-time dynamic model. This model was integrated into a model predictive control (MPC) framework, and a comparative analysis was conducted against the MPC framework using a nonlinear first-principle model of the vine robot. The results demonstrate that the DNN-MPC framework offers a better control performance and significantly improved computational efficiency compared to the MPC based on the nonlinear first-principles model. The DNN-MPC reduced computation time by a factor of 11, making it highly viable for real-time control applications.
- 1 January 2025
- Published by Walter de Gruyter GmbH in Nonlinear Engineering
- Vol. 14 (1)
Open AccessReal-time force/position control of soft growing robots: A data-driven model predictive approach
Vine robots represent a novel class of soft robots that achieve mobility through tip extension, a mechanism inspired by the natural growth processes of vine plants. This unique movement strategy enables effective navigation in constrained and cluttered environments, offering significant advantages over conventional robotic systems. However, the continuum nature and inherent compliance of vine robots introduce complex modeling and control challenges. Deep learning offer a powerful alternative for modeling systems with such complex dynamics. In this article, we present a data-driven dynamic model for a pressure-driven everting vine robot, utilizing a deep neural network (DNN)-based discrete-time dynamic model. This model was integrated into a model predictive control (MPC) framework, and a comparative analysis was conducted against the MPC framework using a nonlinear first-principle model of the vine robot. The results demonstrate that the DNN-MPC framework offers a better control performance and significantly improved computational efficiency compared to the MPC based on the nonlinear first-principles model. The DNN-MPC reduced computation time by a factor of 11, making it highly viable for real-time control applications.