YJYusheng Jiao

Yusheng Jiao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Yusheng Jiao has a total of 9 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    DEEP DIVE INTO MODEL-FREE REINFORCEMENT LEARNING FOR UNDERWATER LOCOMOTION: THEORY AND PRACTICE

    by Yusheng Jiao, Feng Ling, Sina Heydari, Nicolas Heess, Josh Merel, Eva Kanso

    Aquatic animals and underwater robots operate in a complex physical world and must coordinate their bodies to achieve behavioral objectives such as navigation and predation. With recent developments in deep reinforcement learning (RL), it is now possible for scientists and engineers to synthesize sensorimotor strategies (policies) for specific tasks using physically simulated bodies and environments. However, beyond solving individual control problems, these methods offer an exciting framework for understanding the organization of an animal sensorimotor system in connection with its morphology and physical interaction with the environment, as well as for deriving general design rules for bioinspired underwater robots. Although algorithms and code implementing both learning agents and environments are increasingly available, the basic assumptions and modeling choices that go into the formulation of an embodied feedback control problem using deep reinforcement learning may not be immediately apparent. In this tutorial, we provide a self-contained introduction to model-free reinforcement learning for embodied agents in underwater environments, with a focus on actor-critic methods. We first present the mathematical formulation of RL, highlighting where physical modeling choices enter. We then discuss the practical aspects of implementing actor-critic algorithms. Drawing on recent examples of RL-controlled swimmers, we provide guidelines for choosing observations, actions, and rewards consistent with biological behavior, and we outline how RL can be used as a tool to explore hypotheses about the feedback control underlying animal and robotic behavior.

  • Open Access

    Flow currents support simple and versatile trail-tracking strategies

    by Haotian Hang, Yusheng Jiao, Josh Merel, Eva Kanso

    Aquatic animals offer compelling evidence that flow sensing alone, without vision, is sufficient to guide a swimming organism to the source of an unsteady hydrodynamic trail. However, the sensory feedback strategies that allow these remarkable trail-tracking abilities remain opaque. Here, by integrating mechanistic flow simulations with reinforcement learning techniques, we discovered two simple and equally effective strategies for hydrodynamic trail following. Though not obvious, these strategies possess parsimonious interpretations, analogous to Braitenberg’s simplest vehicles, where the agent senses local flow signals and turns away from or toward the direction of stronger signals. A rigorous stability analysis shows that the effectiveness of these strategies in robustly tracking flow currents is independent of the type of sensor but depends on sensor placement and the traveling nature of the flow signal. Importantly, these results inform a suite of versatile strategies for hydrodynamic trail following applicable to both vortical and turbulent flows. These insights support the future design and implementation of adaptive real-time sensory feedback strategies for autonomous robots in dynamic flow environments.


  • Open Access

    Flow Currents Support Simple and Versatile Trail-Tracking Strategies

    by Haotian Hang, Yusheng Jiao, Josh Merel, Eva Kanso, Sina Heydari, Feng Ling

    Aquatic animals offer compelling evidence that flow sensing alone, without vision, is sufficient to guide a swimming organism to the source of an unsteady hydrodynamic trail. However, the sensory feedback strategies that allow these remarkable trail tracking abilities remain opaque. Here, by integrating mechanistic flow simulations with reinforcement learning techniques, we discovered two simple and equally effective strategies for hydrodynamic trail following. Though not a priori obvious, these strategies possess parsimonious interpretations, analogous to Braitenberg’s simplest vehicles, where the agent senses local flow signals and turns away from or toward the direction of stronger signals. A rigorous stability analysis shows that the effectiveness of these strategies in robustly tracking flow currents is independent of the type of sensor but depends on sensor placement and the traveling nature of the flow signal. Importantly, these results inform a suite of versatile strategies for hydrodynamic trail following applicable to both vortical and turbulent flows. These insights support the future design and implementation of adaptive real-time sensory feedback strategies for autonomous robots in dynamic flow environments.


  • Open Access

    Evaluating evasion strategies in zebrafish larvae

    by Yusheng Jiao, Brendan Colvert, Yi Man, Matthew J. McHenry, Eva Kanso

    An effective evasion strategy allows prey to survive encounters with predators. Prey are generally thought to escape in a direction that is either random or serves to maximize the minimum distance from the predator. Here, we introduce a comprehensive approach to determine the most likely evasion strategy among multiple hypotheses and the role of biomechanical constraints on the escape response of prey fish. Through a consideration of six strategies with sensorimotor noise and previous kinematic measurements, our analysis shows that zebrafish larvae generally escape in a direction orthogonal to the predator’s heading. By sensing only the predator’s heading, this orthogonal strategy maximizes the distance from fast-moving predators, and, when operating within the biomechanical constraints of the escape response, it provides the best predictions of prey behavior among all alternatives. This work demonstrates a framework for resolving the strategic basis of evasion in predator–prey interactions, which could be applied to a broad diversity of animals.


  • Open Access

    Evaluating Evasion Strategies in Zebrafish Larvae

    by Yusheng Jiao, Brendan Colvert, Yi Man, Matthew J. McHenry, Eva Kanso

    An effective evasion strategy allows prey to survive encounters with predators. Prey are generally thought to escape in a direction that is either random or serves to maximize the minimum distance from the predator. Here we introduce a comprehensive approach to determine the most likely evasion strategy among multiple hypotheses and the role of biomechanical constraints on the escape response of prey fish. Through a consideration of six strategies with sensorimotor noise and previous kinematic measurements, our analysis shows that zebrafish larvae generally escape in a direction orthogonal to the predator’s heading. By sensing only the predator’s heading, this orthogonal strategy maximizes the distance from fast-moving predators, and, when operating within the biomechanical constraints of the escape response, it provides the best predictions of prey behavior among all alternatives. This work demonstrates a framework for resolving the strategic basis of evastion in predator-prey interactions, which could be applied to a broad diversity of animals.

  • Learning to swim in potential flow

    by Yusheng Jiao, Feng Ling, Sina Heydari, Nicolas Heess, Josh Merel, Eva Kanso

    Fish swim by undulating their bodies. These propulsive motions require coordinated shape changes of a body that interacts with its fluid environment, but the specific shape coordination that leads to robust turning and swimming motions remains unclear. To address the problem of underwater motion planning, we propose a simple model of a three-link fish swimming in a potential flow environment and we use model-free reinforcement learning for shape control. We arrive at optimal shape changes for two swimming tasks: swimming in a desired direction and swimming towards a known target. This fish model belongs to a class of problems in geometric mechanics, known as driftless dynamical systems, which allow us to analyze the swimming behavior in terms of geometric phases over the shape space of the fish. These geometric methods are less intuitive in the presence of drift. Here, we use the shape space analysis as a tool for assessing, visualizing, and interpreting the control policies obtained via reinforcement learning in the absence of drift. We then examine the robustness of these policies to drift-related perturbations. Although the fish has no direct control over the drift itself, it learns to take advantage of the presence of moderate drift to reach its target.