Annan Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Annan Zhang has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Daniela Rus
Massachusetts Institute of Technology
3 shared publications - Lillian Chin
University of Texas at Austin, Austin, TX, USA
2 shared publications - Ryan Landon Truby
Department of Mechanical Engineering, Northwestern University, Evanston, IL 60208, USA
2 shared publications - Alexandre Amice
Massachusetts Institute of Technology (MIT), Cambridge, MA, USA
1 shared publication Boyuan Chen
Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, USA
1 shared publication- Chao Liu
Mechanical Engineering, The University of British Columbia, Vancouver, British Columbia, CANADA
1 shared publication
Latest Publications
- Open Access
- 25 June 2025
- Published by Springer Nature in Nature
- Vol. 643 (8070) , 89-95
Controlling diverse robots by inferring Jacobian fields with deep networks
Mirroring the complex structures and diverse functions of natural organisms is a long-standing challenge in robotics1–4. Modern fabrication techniques have greatly expanded the feasible hardware5–8, but using these systems requires control software to translate the desired motions into actuator commands. Conventional robots can easily be modelled as rigid links connected by joints, but it remains an open challenge to model and control biologically inspired robots that are often soft or made of several materials, lack sensing capabilities and may change their material properties with use9–12. Here, we introduce a method that uses deep neural networks to map a video stream of a robot to its visuomotor Jacobian field (the sensitivity of all 3D points to the robot’s actuators). Our method enables the control of robots from only a single camera, makes no assumptions about the robots’ materials, actuation or sensing, and is trained without expert intervention by observing the execution of random commands. We demonstrate our method on a diverse set of robot manipulators that vary in actuation, materials, fabrication and cost. Our approach achieves accurate closed-loop control and recovers the causal dynamic structure of each robot. Because it enables robot control using a generic camera as the only sensor, we anticipate that our work will broaden the design space of robotic systems and serve as a starting point for lowering the barrier to robotic automation.
- 25 June 2025
- Published by Springer Nature in Nature
- Vol. 643 (8070) , 89-95
Open AccessControlling diverse robots by inferring Jacobian fields with deep networks
Mirroring the complex structures and diverse functions of natural organisms is a long-standing challenge in robotics1–4. Modern fabrication techniques have greatly expanded the feasible hardware5–8, but using these systems requires control software to translate the desired motions into actuator commands. Conventional robots can easily be modelled as rigid links connected by joints, but it remains an open challenge to model and control biologically inspired robots that are often soft or made of several materials, lack sensing capabilities and may change their material properties with use9–12. Here, we introduce a method that uses deep neural networks to map a video stream of a robot to its visuomotor Jacobian field (the sensitivity of all 3D points to the robot’s actuators). Our method enables the control of robots from only a single camera, makes no assumptions about the robots’ materials, actuation or sensing, and is trained without expert intervention by observing the execution of random commands. We demonstrate our method on a diverse set of robot manipulators that vary in actuation, materials, fabrication and cost. Our approach achieves accurate closed-loop control and recovers the causal dynamic structure of each robot. Because it enables robot control using a generic camera as the only sensor, we anticipate that our work will broaden the design space of robotic systems and serve as a starting point for lowering the barrier to robotic automation.
- 3 November 2023
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 43 (9) , 1322-1341
Certified polyhedral decompositions of collision-free configuration space
Understanding the geometry of collision-free configuration space (C-free) in the presence of Cartesian-space obstacles is an essential ingredient for collision-free motion planning. While it is possible to check for collisions at a point using standard algorithms, to date no practical method exists for computing C-free regions with rigorous certificates due to the complexity of mapping Cartesian-space obstacles through the kinematics. In this work, we present the first to our knowledge rigorous method for approximately decomposing a rational parametrization of C-free into certified polyhedral regions. Our method, called C-Iris (C-space Iterative Regional Inflation by Semidefinite programming), generates large, convex polytopes in a rational parameterization of the configuration space which are rigorously certified to be collision-free. Such regions have been shown to be useful for both optimization-based and randomized motion planning. Based on convex optimization, our method works in arbitrary dimensions, only makes assumptions about the convexity of the obstacles in the 3D Cartesian space, and is fast enough to scale to realistic problems in manipulation. We demonstrate our algorithm’s ability to fill a non-trivial amount of collision-free C-space in several 2-DOF examples where the C-space can be visualized, as well as the scalability of our algorithm on a 7-DOF KUKA iiwa, a 6-DOF UR3e, and 12-DOF bimanual manipulators. An implementation of our algorithm is open-sourced in Drake . We furthermore provide examples of our algorithm in interactive Python notebooks .
- 3 November 2023
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 43 (9) , 1322-1341
Certified polyhedral decompositions of collision-free configuration space
Understanding the geometry of collision-free configuration space (C-free) in the presence of Cartesian-space obstacles is an essential ingredient for collision-free motion planning. While it is possible to check for collisions at a point using standard algorithms, to date no practical method exists for computing C-free regions with rigorous certificates due to the complexity of mapping Cartesian-space obstacles through the kinematics. In this work, we present the first to our knowledge rigorous method for approximately decomposing a rational parametrization of C-free into certified polyhedral regions. Our method, called C-Iris (C-space Iterative Regional Inflation by Semidefinite programming), generates large, convex polytopes in a rational parameterization of the configuration space which are rigorously certified to be collision-free. Such regions have been shown to be useful for both optimization-based and randomized motion planning. Based on convex optimization, our method works in arbitrary dimensions, only makes assumptions about the convexity of the obstacles in the 3D Cartesian space, and is fast enough to scale to realistic problems in manipulation. We demonstrate our algorithm’s ability to fill a non-trivial amount of collision-free C-space in several 2-DOF examples where the C-space can be visualized, as well as the scalability of our algorithm on a 7-DOF KUKA iiwa, a 6-DOF UR3e, and 12-DOF bimanual manipulators. An implementation of our algorithm is open-sourced in Drake . We furthermore provide examples of our algorithm in interactive Python notebooks .
- Open Access
- 25 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (4) , 11509-11516
Vision-Based Sensing for Electrically-Driven Soft Actuators
Developing reliable control strategies in soft robotics requires advances in soft robot perception. However, current soft robotic sensors pose many performance limitations, and available materials and manufacturing techniques complicate soft sensorized robot design. To address these long-standing needs, we introduce a method for using vision to sensorize robust, electrically-driven soft robotic actuators constructed from a new class of architected materials. Specifically, we use cameras positioned within the hollow interiors of handed shearing auxetic (HSA) actuators to record deformation during motion. We train a convolutional neural network (CNN) that maps the visual feedback to the actuator's tip pose. Our model provides predictions with sub-millimeter accuracy from only six minutes of training data, while remaining lightweight with an inference time of 18 milliseconds per frame. We also develop a model that additionally predicts the horizontal tip force acting on the actuator and generalizes to previously unseen forces. Finally, we demonstrate the viability of our sensorization strategy for contact-rich applications by training a CNN that predicts the tip pose accurately during tactile interactions. Overall, our methods present a reliable vision-based approach for designing sensorized soft robots built from electrically-actuated, architected materials.
- 25 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (4) , 11509-11516
Open AccessVision-Based Sensing for Electrically-Driven Soft Actuators
Developing reliable control strategies in soft robotics requires advances in soft robot perception. However, current soft robotic sensors pose many performance limitations, and available materials and manufacturing techniques complicate soft sensorized robot design. To address these long-standing needs, we introduce a method for using vision to sensorize robust, electrically-driven soft robotic actuators constructed from a new class of architected materials. Specifically, we use cameras positioned within the hollow interiors of handed shearing auxetic (HSA) actuators to record deformation during motion. We train a convolutional neural network (CNN) that maps the visual feedback to the actuator's tip pose. Our model provides predictions with sub-millimeter accuracy from only six minutes of training data, while remaining lightweight with an inference time of 18 milliseconds per frame. We also develop a model that additionally predicts the horizontal tip force acting on the actuator and generalizes to previously unseen forces. Finally, we demonstrate the viability of our sensorization strategy for contact-rich applications by training a CNN that predicts the tip pose accurately during tactile interactions. Overall, our methods present a reliable vision-based approach for designing sensorized soft robots built from electrically-actuated, architected materials.
- Open Access
- 12 August 2022
- Published by American Association for the Advancement of Science (AAAS) in Science Advances
- Vol. 8 (32) , eabq4385
Fluidic innervation sensorizes structures from a single build material
Multifunctional materials with distributed sensing and programmed mechanical properties are required for myriad emerging technologies. However, current fabrication techniques constrain these materials’ design and sensing capabilities. We address these needs with a method for sensorizing architected materials through fluidic innervation, where distributed networks of empty, air-filled channels are directly embedded within an architected material’s sparse geometry. By measuring pressure changes within these channels, we receive feedback regarding material deformation. Thus, this technique allows for three-dimensional printing of sensorized structures from a single material. With this strategy, we fabricate sensorized soft robotic actuators on the basis of handed shearing auxetics and accurately predict their kinematics from the sensors’ proprioceptive feedback using supervised learning. Our strategy for facilitating structural, sensing, and actuation capabilities through control of form alone simplifies sensorized material design for applications spanning wearables, smart structures, and robotics.
- 12 August 2022
- Published by American Association for the Advancement of Science (AAAS) in Science Advances
- Vol. 8 (32) , eabq4385
Open AccessFluidic innervation sensorizes structures from a single build material
Multifunctional materials with distributed sensing and programmed mechanical properties are required for myriad emerging technologies. However, current fabrication techniques constrain these materials’ design and sensing capabilities. We address these needs with a method for sensorizing architected materials through fluidic innervation, where distributed networks of empty, air-filled channels are directly embedded within an architected material’s sparse geometry. By measuring pressure changes within these channels, we receive feedback regarding material deformation. Thus, this technique allows for three-dimensional printing of sensorized structures from a single material. With this strategy, we fabricate sensorized soft robotic actuators on the basis of handed shearing auxetics and accurately predict their kinematics from the sensors’ proprioceptive feedback using supervised learning. Our strategy for facilitating structural, sensing, and actuation capabilities through control of form alone simplifies sensorized material design for applications spanning wearables, smart structures, and robotics.
- Top Cited Papers
- 1 September 2020
- Published by Elsevier in International Journal of Plasticity
- Vol. 132
Using neural networks to represent von Mises plasticity with isotropic hardening
- 1 September 2020
- Published by Elsevier in International Journal of Plasticity
- Vol. 132
Top Cited PapersUsing neural networks to represent von Mises plasticity with isotropic hardening