Sina Heydari
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Sina Heydari has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Eva Kanso
Department of Aerospace and Mechanical Engineering, University of Southern California, Los Angeles, California 90089, USA
7 shared publications - Feng Ling
University of Southern California Viterbi School of Engineering, 920 Downey Way, Los Angeles, California, 90089-0111, United States
3 shared publications - Haotian Hang
Department of Aerospace and Mechanical Engineering, University of Southern California, Los Angeles, USA
3 shared publications Josh Merel
Fauna Robotics, office, New York, 10003, United States
3 shared publications- Yusheng Jiao
University of Southern California
3 shared publications Nicolas Heess
DeepMind, 6-8 Handyside Street, London, England, N1C 4AG, United Kingdom of Great Britain and Northern Ireland
2 shared publications
Latest Publications
- Open Access
- 23 February 2026
- Published by IOP Publishing in Bioinspiration & Biomimetics
- Vol. 21 (2)
DEEP DIVE INTO MODEL-FREE REINFORCEMENT LEARNING FOR UNDERWATER LOCOMOTION: THEORY AND PRACTICE
Aquatic animals and underwater robots operate in a complex physical world and must coordinate their bodies to achieve behavioral objectives such as navigation and predation. With recent developments in deep reinforcement learning (RL), it is now possible for scientists and engineers to synthesize sensorimotor strategies (policies) for specific tasks using physically simulated bodies and environments. However, beyond solving individual control problems, these methods offer an exciting framework for understanding the organization of an animal sensorimotor system in connection with its morphology and physical interaction with the environment, as well as for deriving general design rules for bioinspired underwater robots. Although algorithms and code implementing both learning agents and environments are increasingly available, the basic assumptions and modeling choices that go into the formulation of an embodied feedback control problem using deep reinforcement learning may not be immediately apparent. In this tutorial, we provide a self-contained introduction to model-free reinforcement learning for embodied agents in underwater environments, with a focus on actor-critic methods. We first present the mathematical formulation of RL, highlighting where physical modeling choices enter. We then discuss the practical aspects of implementing actor-critic algorithms. Drawing on recent examples of RL-controlled swimmers, we provide guidelines for choosing observations, actions, and rewards consistent with biological behavior, and we outline how RL can be used as a tool to explore hypotheses about the feedback control underlying animal and robotic behavior.
- 23 February 2026
- Published by IOP Publishing in Bioinspiration & Biomimetics
- Vol. 21 (2)
Open AccessDEEP DIVE INTO MODEL-FREE REINFORCEMENT LEARNING FOR UNDERWATER LOCOMOTION: THEORY AND PRACTICE
Aquatic animals and underwater robots operate in a complex physical world and must coordinate their bodies to achieve behavioral objectives such as navigation and predation. With recent developments in deep reinforcement learning (RL), it is now possible for scientists and engineers to synthesize sensorimotor strategies (policies) for specific tasks using physically simulated bodies and environments. However, beyond solving individual control problems, these methods offer an exciting framework for understanding the organization of an animal sensorimotor system in connection with its morphology and physical interaction with the environment, as well as for deriving general design rules for bioinspired underwater robots. Although algorithms and code implementing both learning agents and environments are increasingly available, the basic assumptions and modeling choices that go into the formulation of an embodied feedback control problem using deep reinforcement learning may not be immediately apparent. In this tutorial, we provide a self-contained introduction to model-free reinforcement learning for embodied agents in underwater environments, with a focus on actor-critic methods. We first present the mathematical formulation of RL, highlighting where physical modeling choices enter. We then discuss the practical aspects of implementing actor-critic algorithms. Drawing on recent examples of RL-controlled swimmers, we provide guidelines for choosing observations, actions, and rewards consistent with biological behavior, and we outline how RL can be used as a tool to explore hypotheses about the feedback control underlying animal and robotic behavior.
- Top Cited PapersOpen Access
- 13 January 2026
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 123 (3)
Tube feet dynamics drive adaptation in sea star locomotion
Sea stars use hundreds of tube feet on their oral surface to crawl, climb, and navigate complex environments, despite lacking a central brain. While tube foot morphology and function as muscular hydrostats are well described, the mechanisms that coordinate their collective dynamics remain poorly understood. To investigate these dynamics, we employed an optical imaging method based on frustrated total internal reflection (FTIR) to visualize and quantify tube foot adhesive contacts in real time in the species Asterias rubens across individuals spanning a wide size range. Our results reveal an inverse relationship between crawling speed and tube foot adhesion time, indicating that sea stars regulate locomotion by modulating contact duration in response to mechanical load. To test this, we conducted perturbation experiments using 3D-printed backpacks that increased body mass by 25 and 50%, along with biomechanical modeling of decentralized feedback control of the tube feet. The added load significantly increased adhesion time, supporting the role of a load-dependent mechanical adaptation. We further investigated inverted locomotion, both experimentally and through simulation, and found that tube feet adjust their contact behavior when the animal is oriented upside down relative to gravity. Together, these findings demonstrate that sea stars adapt their locomotion to changing mechanical demands by modulating tube foot–substrate interactions, revealing a robust decentralized control strategy in a brainless organism and highlighting general principles of distributed control in biology and soft robotics.
- 13 January 2026
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 123 (3)
Top Cited PapersOpen AccessTube feet dynamics drive adaptation in sea star locomotion
Sea stars use hundreds of tube feet on their oral surface to crawl, climb, and navigate complex environments, despite lacking a central brain. While tube foot morphology and function as muscular hydrostats are well described, the mechanisms that coordinate their collective dynamics remain poorly understood. To investigate these dynamics, we employed an optical imaging method based on frustrated total internal reflection (FTIR) to visualize and quantify tube foot adhesive contacts in real time in the species Asterias rubens across individuals spanning a wide size range. Our results reveal an inverse relationship between crawling speed and tube foot adhesion time, indicating that sea stars regulate locomotion by modulating contact duration in response to mechanical load. To test this, we conducted perturbation experiments using 3D-printed backpacks that increased body mass by 25 and 50%, along with biomechanical modeling of decentralized feedback control of the tube feet. The added load significantly increased adhesion time, supporting the role of a load-dependent mechanical adaptation. We further investigated inverted locomotion, both experimentally and through simulation, and found that tube feet adjust their contact behavior when the animal is oriented upside down relative to gravity. Together, these findings demonstrate that sea stars adapt their locomotion to changing mechanical demands by modulating tube foot–substrate interactions, revealing a robust decentralized control strategy in a brainless organism and highlighting general principles of distributed control in biology and soft robotics.
- 16 June 2025
- Published by American Physical Society (APS) in Physical Review Fluids
- Vol. 10 (6)
Navigation of a three-link microswimmer via deep reinforcement learning
- 16 June 2025
- Published by American Physical Society (APS) in Physical Review Fluids
- Vol. 10 (6)
Navigation of a three-link microswimmer via deep reinforcement learning
- Open Access
- 19 December 2024
- Published by eLife Sciences Publications, Ltd in eLife
- Vol. 13
Mapping spatial patterns to energetic benefits in groups of flow-coupled swimmers
The coordinated motion of animal groups through fluids is thought to reduce the cost of locomotion to individuals in the group. However, the connection between the spatial patterns observed in collectively moving animals and the energetic benefits at each position within the group remains unclear. To address this knowledge gap, we study the spontaneous emergence of cohesive formations in groups of fish, modeled as flapping foils, all heading in the same direction. We show in pairwise formations and with increasing group size that (1) in side-by-side arrangements, the reciprocal nature of flow coupling results in an equal distribution of energy requirements among all members, with reduction in cost of locomotion for swimmers flapping inphase but an increase in cost for swimmers flapping antiphase, and (2) in inline arrangements, flow coupling is non-reciprocal for all flapping phase, with energetic savings in favor of trailing swimmers, but only up to a finite number of swimmers, beyond which school cohesion and energetic benefits are lost at once. We explain these findings mechanistically and we provide efficient diagnostic tools for identifying locations in the wake of single and multiple swimmers that offer opportunities for hydrodynamic benefits to aspiring followers. Our results imply a connection between the resources generated by flow physics and social traits that influence greedy and cooperative group behavior.
- 19 December 2024
- Published by eLife Sciences Publications, Ltd in eLife
- Vol. 13
Open AccessMapping spatial patterns to energetic benefits in groups of flow-coupled swimmers
The coordinated motion of animal groups through fluids is thought to reduce the cost of locomotion to individuals in the group. However, the connection between the spatial patterns observed in collectively moving animals and the energetic benefits at each position within the group remains unclear. To address this knowledge gap, we study the spontaneous emergence of cohesive formations in groups of fish, modeled as flapping foils, all heading in the same direction. We show in pairwise formations and with increasing group size that (1) in side-by-side arrangements, the reciprocal nature of flow coupling results in an equal distribution of energy requirements among all members, with reduction in cost of locomotion for swimmers flapping inphase but an increase in cost for swimmers flapping antiphase, and (2) in inline arrangements, flow coupling is non-reciprocal for all flapping phase, with energetic savings in favor of trailing swimmers, but only up to a finite number of swimmers, beyond which school cohesion and energetic benefits are lost at once. We explain these findings mechanistically and we provide efficient diagnostic tools for identifying locations in the wake of single and multiple swimmers that offer opportunities for hydrodynamic benefits to aspiring followers. Our results imply a connection between the resources generated by flow physics and social traits that influence greedy and cooperative group behavior.
- Open Access
Flow Currents Support Simple and Versatile Trail-Tracking Strategies
Aquatic animals offer compelling evidence that flow sensing alone, without vision, is sufficient to guide a swimming organism to the source of an unsteady hydrodynamic trail. However, the sensory feedback strategies that allow these remarkable trail tracking abilities remain opaque. Here, by integrating mechanistic flow simulations with reinforcement learning techniques, we discovered two simple and equally effective strategies for hydrodynamic trail following. Though not a priori obvious, these strategies possess parsimonious interpretations, analogous to Braitenberg’s simplest vehicles, where the agent senses local flow signals and turns away from or toward the direction of stronger signals. A rigorous stability analysis shows that the effectiveness of these strategies in robustly tracking flow currents is independent of the type of sensor but depends on sensor placement and the traveling nature of the flow signal. Importantly, these results inform a suite of versatile strategies for hydrodynamic trail following applicable to both vortical and turbulent flows. These insights support the future design and implementation of adaptive real-time sensory feedback strategies for autonomous robots in dynamic flow environments.
Open AccessFlow Currents Support Simple and Versatile Trail-Tracking Strategies
Aquatic animals offer compelling evidence that flow sensing alone, without vision, is sufficient to guide a swimming organism to the source of an unsteady hydrodynamic trail. However, the sensory feedback strategies that allow these remarkable trail tracking abilities remain opaque. Here, by integrating mechanistic flow simulations with reinforcement learning techniques, we discovered two simple and equally effective strategies for hydrodynamic trail following. Though not a priori obvious, these strategies possess parsimonious interpretations, analogous to Braitenberg’s simplest vehicles, where the agent senses local flow signals and turns away from or toward the direction of stronger signals. A rigorous stability analysis shows that the effectiveness of these strategies in robustly tracking flow currents is independent of the type of sensor but depends on sensor placement and the traveling nature of the flow signal. Importantly, these results inform a suite of versatile strategies for hydrodynamic trail following applicable to both vortical and turbulent flows. These insights support the future design and implementation of adaptive real-time sensory feedback strategies for autonomous robots in dynamic flow environments.
- Open Access
- 9 December 2021
- Published by Cambridge University Press (CUP) in Journal of Fluid Mechanics
- Vol. 932, A35
Active tail flexion in concert with passive hydrodynamic forces improves swimming speed and efficiency
Fish typically swim by periodic bending of their bodies. Bending seems to follow a universal rule; it occurs at about one-third from the posterior end of the fish body with a maximum bending angle of about $30^{\circ }$ . However, the hydrodynamic mechanisms that shaped this convergent design and its potential benefit to fish in terms of swimming speed and efficiency are not well understood. It is also unclear to what extent this bending is active or follows passively from the interaction of a flexible posterior with the fluid environment. Here, we use a self-propelled two-link model, with fluid–structure interactions described in the context of the vortex sheet method, to analyse the effects of both active and passive body bending on the swimming performance. We find that passive bending is more efficient but could reduce swimming speed compared with rigid flapping, but the addition of active bending could enhance both speed and efficiency. Importantly, we find that the phase difference between the posterior and anterior sections of the body is an important kinematic factor that influences performance, and that active antiphase flexion, consistent with the passive flexion phase, can simultaneously enhance speed and efficiency in a region of the design space that overlaps with biological observations. Our results are consistent with the hypothesis that fish that actively bend their bodies in a fashion that exploits passive hydrodynamics can at once improve speed and efficiency.- 9 December 2021
- Published by Cambridge University Press (CUP) in Journal of Fluid Mechanics
- Vol. 932, A35
Open AccessActive tail flexion in concert with passive hydrodynamic forces improves swimming speed and efficiency
Fish typically swim by periodic bending of their bodies. Bending seems to follow a universal rule; it occurs at about one-third from the posterior end of the fish body with a maximum bending angle of about $30^{\circ }$ . However, the hydrodynamic mechanisms that shaped this convergent design and its potential benefit to fish in terms of swimming speed and efficiency are not well understood. It is also unclear to what extent this bending is active or follows passively from the interaction of a flexible posterior with the fluid environment. Here, we use a self-propelled two-link model, with fluid–structure interactions described in the context of the vortex sheet method, to analyse the effects of both active and passive body bending on the swimming performance. We find that passive bending is more efficient but could reduce swimming speed compared with rigid flapping, but the addition of active bending could enhance both speed and efficiency. Importantly, we find that the phase difference between the posterior and anterior sections of the body is an important kinematic factor that influences performance, and that active antiphase flexion, consistent with the passive flexion phase, can simultaneously enhance speed and efficiency in a region of the design space that overlaps with biological observations. Our results are consistent with the hypothesis that fish that actively bend their bodies in a fashion that exploits passive hydrodynamics can at once improve speed and efficiency. - 13 July 2021
- Published by Cambridge University Press (CUP) in Journal of Fluid Mechanics
- Vol. 922
School cohesion, speed and efficiency are modulated by the swimmers flapping motion
Fish schools are ubiquitous in marine life. Although flow interactions are thought to be beneficial for schooling, their exact effects on the speed, energetics and stability of the group remain elusive. Recent numerical simulations and experimental models suggest that flow interactions stabilize in-tandem formations of flapping foils. Here, we employ a minimal vortex sheet model that captures salient features of the flow interactions among flapping swimmers, and we study the free swimming of a pair of in-line swimmers driven with identical heaving or pitching motions. We find that, independent of the flapping mode, heaving or pitching, the follower passively stabilizes at discrete locations in the wake of the leader, consistent with the heaving foil experiments, but pitching swimmers exhibit tighter and more cohesive formations. Further, in comparison to swimming alone, pitching motions increase the energetic efficiency of the group while heaving motions result in a slight increase in the swimming speed. A deeper analysis of the wake of a single swimmer sheds light on the hydrodynamic mechanisms underlying pairwise formations. These results recapitulate that flow interactions provide a passive mechanism that promotes school cohesion, and afford novel insights into the role of the flapping mode in controlling the emergent properties of the school.
- 13 July 2021
- Published by Cambridge University Press (CUP) in Journal of Fluid Mechanics
- Vol. 922
School cohesion, speed and efficiency are modulated by the swimmers flapping motion
Fish schools are ubiquitous in marine life. Although flow interactions are thought to be beneficial for schooling, their exact effects on the speed, energetics and stability of the group remain elusive. Recent numerical simulations and experimental models suggest that flow interactions stabilize in-tandem formations of flapping foils. Here, we employ a minimal vortex sheet model that captures salient features of the flow interactions among flapping swimmers, and we study the free swimming of a pair of in-line swimmers driven with identical heaving or pitching motions. We find that, independent of the flapping mode, heaving or pitching, the follower passively stabilizes at discrete locations in the wake of the leader, consistent with the heaving foil experiments, but pitching swimmers exhibit tighter and more cohesive formations. Further, in comparison to swimming alone, pitching motions increase the energetic efficiency of the group while heaving motions result in a slight increase in the swimming speed. A deeper analysis of the wake of a single swimmer sheds light on the hydrodynamic mechanisms underlying pairwise formations. These results recapitulate that flow interactions provide a passive mechanism that promotes school cohesion, and afford novel insights into the role of the flapping mode in controlling the emergent properties of the school.
- 12 May 2021
- Published by American Physical Society (APS) in Physical Review Fluids
- Vol. 6 (5) , 050505
Learning to swim in potential flow
Fish swim by undulating their bodies. These propulsive motions require coordinated shape changes of a body that interacts with its fluid environment, but the specific shape coordination that leads to robust turning and swimming motions remains unclear. To address the problem of underwater motion planning, we propose a simple model of a three-link fish swimming in a potential flow environment and we use model-free reinforcement learning for shape control. We arrive at optimal shape changes for two swimming tasks: swimming in a desired direction and swimming towards a known target. This fish model belongs to a class of problems in geometric mechanics, known as driftless dynamical systems, which allow us to analyze the swimming behavior in terms of geometric phases over the shape space of the fish. These geometric methods are less intuitive in the presence of drift. Here, we use the shape space analysis as a tool for assessing, visualizing, and interpreting the control policies obtained via reinforcement learning in the absence of drift. We then examine the robustness of these policies to drift-related perturbations. Although the fish has no direct control over the drift itself, it learns to take advantage of the presence of moderate drift to reach its target.
- 12 May 2021
- Published by American Physical Society (APS) in Physical Review Fluids
- Vol. 6 (5) , 050505
Learning to swim in potential flow
Fish swim by undulating their bodies. These propulsive motions require coordinated shape changes of a body that interacts with its fluid environment, but the specific shape coordination that leads to robust turning and swimming motions remains unclear. To address the problem of underwater motion planning, we propose a simple model of a three-link fish swimming in a potential flow environment and we use model-free reinforcement learning for shape control. We arrive at optimal shape changes for two swimming tasks: swimming in a desired direction and swimming towards a known target. This fish model belongs to a class of problems in geometric mechanics, known as driftless dynamical systems, which allow us to analyze the swimming behavior in terms of geometric phases over the shape space of the fish. These geometric methods are less intuitive in the presence of drift. Here, we use the shape space analysis as a tool for assessing, visualizing, and interpreting the control policies obtained via reinforcement learning in the absence of drift. We then examine the robustness of these policies to drift-related perturbations. Although the fish has no direct control over the drift itself, it learns to take advantage of the presence of moderate drift to reach its target.