Xiang Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Xiang Zhang has a total of 32 co-authors. Here are the top co-authors by number of shared publications:
- Masayoshi Tomizuka
UC Berkeley
8 shared publications - Changhao Wang
University of California, Berkeley, CA, USA
4 shared publications - Zhian Kuang
Space Control and Inertial Technology Research Center, Harbin Institute of Technology, Harbin China 150086
3 shared publications - Liting Sun
University of California Berkeley
2 shared publications - Xinghao Zhu
Robotics and AI Institute (RAI), Boston, MA, USA
2 shared publications Aaron D. Ames
California Institute of Technology, Pasadena, CA, USA
1 shared publication
Latest Publications
- 29 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 31 (3) , 3202-3213
DexH2R: Task-Oriented Dexterous Manipulation From Human to Robots
Dexterous manipulation remains a challenging problem due to the high-dimensional action space of robot hands. To solve this challenge, recent advances focus on transferring human hand motions to dexterous robot motions. However, the most straightforward method, retargeting, fails to ensure successful task execution due to the ignorance of hand–object interaction and environmental feedback, while other methods combining retargeting with human feedback significantly increase human labor and degrade motion quality. To address this limitation, we propose a novel framework that enhances retargeted motions with a residual policy based on reinforcement learning, allowing robots to autonomously correct action errors without human intervention. Specifically, our approach leverages human demonstrations to provide natural and task-relevant finger motions while incorporating object and environmental information to improve manipulation performance. Extensive experiments in multiple robot hand embodiments demonstrate that our method outperforms baseline approaches by approximately 40% in task success rates on both seen and unseen objects. Furthermore, our approach eliminates the need for real-time human correction, significantly reducing labor costs and facilitating scalable data collection for dexterous manipulation.
- 29 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 31 (3) , 3202-3213
DexH2R: Task-Oriented Dexterous Manipulation From Human to Robots
Dexterous manipulation remains a challenging problem due to the high-dimensional action space of robot hands. To solve this challenge, recent advances focus on transferring human hand motions to dexterous robot motions. However, the most straightforward method, retargeting, fails to ensure successful task execution due to the ignorance of hand–object interaction and environmental feedback, while other methods combining retargeting with human feedback significantly increase human labor and degrade motion quality. To address this limitation, we propose a novel framework that enhances retargeted motions with a residual policy based on reinforcement learning, allowing robots to autonomously correct action errors without human intervention. Specifically, our approach leverages human demonstrations to provide natural and task-relevant finger motions while incorporating object and environmental information to improve manipulation performance. Extensive experiments in multiple robot hand embodiments demonstrate that our method outperforms baseline approaches by approximately 40% in task success rates on both seen and unseen objects. Furthermore, our approach eliminates the need for real-time human correction, significantly reducing labor costs and facilitating scalable data collection for dexterous manipulation.
- 29 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 30 (4) , 2955-2963
Prismatic-Bending Transformable (PBT) Joint for a Modular, Foldable Manipulator With Enhanced Reachability and Dexterity
Robotic manipulators, traditionally designed with classical joint-link articulated structures, excel in industrial applications but face challenges in human-centered and general-purpose tasks requiring greater dexterity and adaptability. To address these challenges, we propose the prismatic-bending transformable (PBT) joint—a novel, scissors-inspired mechanism with directional maintenance capability that provides bending, rotation, and elongation/contraction within a single module. This design enables transformable kinematic chains that are modular, reconfigurable, and scalable for diverse tasks. We detail the mechanical design, optimization, kinematic and dynamic modeling, and experimental validation of the PBT joint, demonstrating its integration into foldable, modular robotic manipulators. The PBT joint functions as a single stock keeping unit, enabling manipulators to be constructed entirely from standardized PBT joints. It also serves as a modular extension for existing systems, such as wrist modules, streamlining design, deployment, transportation, and maintenance. Three joint sizes have been developed and tested, showcasing enhanced dexterity, reachability, and adaptability, particularly in confined and cluttered spaces. This work presents a promising approach to robotic manipulator development, providing a compact and versatile solution for operation in dynamic and constrained environments.
- 29 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 30 (4) , 2955-2963
Prismatic-Bending Transformable (PBT) Joint for a Modular, Foldable Manipulator With Enhanced Reachability and Dexterity
Robotic manipulators, traditionally designed with classical joint-link articulated structures, excel in industrial applications but face challenges in human-centered and general-purpose tasks requiring greater dexterity and adaptability. To address these challenges, we propose the prismatic-bending transformable (PBT) joint—a novel, scissors-inspired mechanism with directional maintenance capability that provides bending, rotation, and elongation/contraction within a single module. This design enables transformable kinematic chains that are modular, reconfigurable, and scalable for diverse tasks. We detail the mechanical design, optimization, kinematic and dynamic modeling, and experimental validation of the PBT joint, demonstrating its integration into foldable, modular robotic manipulators. The PBT joint functions as a single stock keeping unit, enabling manipulators to be constructed entirely from standardized PBT joints. It also serves as a modular extension for existing systems, such as wrist modules, streamlining design, deployment, transportation, and maintenance. Three joint sizes have been developed and tested, showcasing enhanced dexterity, reachability, and adaptability, particularly in confined and cluttered spaces. This work presents a promising approach to robotic manipulator development, providing a compact and versatile solution for operation in dynamic and constrained environments.
- 25 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (2) , 1508-1515
Contact-Rich SE(3)-Equivariant Robot Manipulation Task Learning via Geometric Impedance Control
This letter presents a differential geometric control approach that leverages SE(3) group invariance and equivariance to increase transferability in learning robot manipulation tasks that involve interaction with the environment. Specifically, we employ a control law and a learning representation framework that remain invariant under arbitrary SE(3) transformations of the manipulation task definition. Furthermore, the control law and learning representation framework are shown to be SE(3) equivariant when represented relative to the spatial frame. The proposed approach is based on utilizing a recently presented geometric impedance control (GIC) combined with a learning variable impedance control framework, where the gain scheduling policy is trained in a supervised learning fashion from expert demonstrations. A geometrically consistent error vector (GCEV) is fed to a neural network to achieve a gain scheduling policy that remains invariant to arbitrary translation and rotations. A comparison of our proposed control and learning framework with a well-known Cartesian space learning impedance control, equipped with a Cartesian error vector-based gain scheduling policy, confirms the significantly superior learning transferability of our proposed approach. A hardware implementation on a peg-in-hole task is conducted to validate the learning transferability and feasibility of the proposed approach.
- 25 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (2) , 1508-1515
Contact-Rich SE(3)-Equivariant Robot Manipulation Task Learning via Geometric Impedance Control
This letter presents a differential geometric control approach that leverages SE(3) group invariance and equivariance to increase transferability in learning robot manipulation tasks that involve interaction with the environment. Specifically, we employ a control law and a learning representation framework that remain invariant under arbitrary SE(3) transformations of the manipulation task definition. Furthermore, the control law and learning representation framework are shown to be SE(3) equivariant when represented relative to the spatial frame. The proposed approach is based on utilizing a recently presented geometric impedance control (GIC) combined with a learning variable impedance control framework, where the gain scheduling policy is trained in a supervised learning fashion from expert demonstrations. A geometrically consistent error vector (GCEV) is fed to a neural network to achieve a gain scheduling policy that remains invariant to arbitrary translation and rotations. A comparison of our proposed control and learning framework with a well-known Cartesian space learning impedance control, equipped with a Cartesian error vector-based gain scheduling policy, confirms the significantly superior learning transferability of our proposed approach. A hardware implementation on a peg-in-hole task is conducted to validate the learning transferability and feasibility of the proposed approach.
- 10 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 5544-5551
Offline-Online Learning of Deformation Model for Cable Manipulation With Graph Neural Networks
Manipulating deformable linear objects by robots has a wide range of applications, e.g., manufacturing and medical surgery. To complete such tasks, an accurate dynamics model for predicting the deformation is critical for robust control. In this letter, we deal with this challenge by proposing a hybrid offline-online method to learn the dynamics of cables in a robust and data-efficient manner. In the offline phase, we adopt Graph Neural Network (GNN) to learn the deformation dynamics purely from the simulation data. Then a linear residual model is learned in real-time to bridge the sim-to-real gap. The learned model is then utilized as the dynamics constraint of a trust region based Model Predictive Controller (MPC) to calculate the optimal robot movements. The online learning and MPC run in a closed-loop manner to robustly accomplish the task. Finally, comparative results with existing methods are provided to quantitatively show the effectiveness and robustness.
- 10 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 5544-5551
Offline-Online Learning of Deformation Model for Cable Manipulation With Graph Neural Networks
Manipulating deformable linear objects by robots has a wide range of applications, e.g., manufacturing and medical surgery. To complete such tasks, an accurate dynamics model for predicting the deformation is critical for robust control. In this letter, we deal with this challenge by proposing a hybrid offline-online method to learn the dynamics of cables in a robust and data-efficient manner. In the offline phase, we adopt Graph Neural Network (GNN) to learn the deformation dynamics purely from the simulation data. Then a linear residual model is learned in real-time to bridge the sim-to-real gap. The learned model is then utilized as the dynamics constraint of a trust region based Model Predictive Controller (MPC) to calculate the optimal robot movements. The online learning and MPC run in a closed-loop manner to robustly accomplish the task. Finally, comparative results with existing methods are provided to quantitatively show the effectiveness and robustness.
- Open Access
Safe Online Gain Optimization for Variable Impedance Control
Smooth behaviors are preferable for many contact-rich manipulation tasks. Impedance control arises as an effective way to regulate robot movements by mimicking a mass-spring-damping system. Consequently, the robot behavior can be determined by the impedance gains. However, tuning the impedance gains for different tasks is tricky, especially for unstructured environments. Moreover, online adapting the optimal gains to meet the time-varying performance index is even more challenging. In this paper, we present Safe Online Gain Optimization for Variable Impedance Control (Safe OnGO-VIC). By reformulating the dynamics of impedance control as a control-affine system, in which the impedance gains are the inputs, we provide a novel perspective to understand variable impedance control. Additionally, we innovatively formulate an optimization problem with online collected force information to obtain the optimal impedance gains in real-time. Safety constraints are also embedded in the proposed framework to avoid unwanted collisions. We experimentally validated the proposed algorithm on three manipulation tasks. Comparison results with a constant gain baseline and an adaptive control method prove that the proposed algorithm is effective and generalizable to different scenarios.
Open AccessSafe Online Gain Optimization for Variable Impedance Control
Smooth behaviors are preferable for many contact-rich manipulation tasks. Impedance control arises as an effective way to regulate robot movements by mimicking a mass-spring-damping system. Consequently, the robot behavior can be determined by the impedance gains. However, tuning the impedance gains for different tasks is tricky, especially for unstructured environments. Moreover, online adapting the optimal gains to meet the time-varying performance index is even more challenging. In this paper, we present Safe Online Gain Optimization for Variable Impedance Control (Safe OnGO-VIC). By reformulating the dynamics of impedance control as a control-affine system, in which the impedance gains are the inputs, we provide a novel perspective to understand variable impedance control. Additionally, we innovatively formulate an optimization problem with online collected force information to obtain the optimal impedance gains in real-time. Safety constraints are also embedded in the proposed framework to avoid unwanted collisions. We experimentally validated the proposed algorithm on three manipulation tasks. Comparison results with a constant gain baseline and an adaptive control method prove that the proposed algorithm is effective and generalizable to different scenarios.
- 30 August 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (4) , 8442-8449
Online Learning of Unknown Dynamics for Model-Based Controllers in Legged Locomotion
The performance of a model-based controller can severely suffer when its model inaccurately represents the real world dynamics. We propose to learn a time-varying, locally linear residual model along the robot’s current trajectory, to compensate for the prediction errors of the controller’s model. Supervised learning is performed online, as the robot is running in the unknown environment, using data collected from its immediate past. We theoretically investigate our method in its general formulation, then apply it to a bipedal controller derived from the full-order dynamics of virtual constraints, and a quadrupedal controller derived from a simplified model of contact forces. For a biped in simulation, our method consistently outperforms the baseline and a recent learning-based method. We also experiment with a 12 kg quadruped in simulation and real world, where the baseline fails to walk with 10 kg of payload but our method succeeds.
- 30 August 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (4) , 8442-8449
Online Learning of Unknown Dynamics for Model-Based Controllers in Legged Locomotion
The performance of a model-based controller can severely suffer when its model inaccurately represents the real world dynamics. We propose to learn a time-varying, locally linear residual model along the robot’s current trajectory, to compensate for the prediction errors of the controller’s model. Supervised learning is performed online, as the robot is running in the unknown environment, using data collected from its immediate past. We theoretically investigate our method in its general formulation, then apply it to a bipedal controller derived from the full-order dynamics of virtual constraints, and a quadrupedal controller derived from a simplified model of contact forces. For a biped in simulation, our method consistently outperforms the baseline and a recent learning-based method. We also experiment with a 12 kg quadruped in simulation and real world, where the baseline fails to walk with 10 kg of payload but our method succeeds.
- 23 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (2) , 2225-2232
Learning Variable Impedance Control via Inverse Reinforcement Learning for Force-Related Tasks
Many manipulation tasks require robots to interact with unknown environments. In such applications, the ability to adapt the impedance according to different task phases and environment constraints is crucial for safety and performance. Although many approaches based on deep reinforcement learning (RL) and learning from demonstration (LfD) have been proposed to obtain variable impedance skills on contact-rich manipulation tasks, these skills are typically task-specific and could be sensitive to changes in task settings. This letter proposes an inverse reinforcement learning (IRL) based approach to recover both the variable impedance policy and reward function from expert demonstrations. We explore different action space of the reward functions to achieve a more general representation of expert variable impedance skills. Experiments on two variable impedance tasks (Peg-in-Hole and Cup-on-Plate) were conducted in both simulations and on a real FANUC LR Mate 200iD/7 L industrial robot. The comparison results with behavior cloning and force-based IRL proved that the learned reward function in the gain action space has better transferability than in the force space. Experiment videos are available at https://msc.berkeley.edu/research/impedance-irl.html .
- 23 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (2) , 2225-2232
Learning Variable Impedance Control via Inverse Reinforcement Learning for Force-Related Tasks
Many manipulation tasks require robots to interact with unknown environments. In such applications, the ability to adapt the impedance according to different task phases and environment constraints is crucial for safety and performance. Although many approaches based on deep reinforcement learning (RL) and learning from demonstration (LfD) have been proposed to obtain variable impedance skills on contact-rich manipulation tasks, these skills are typically task-specific and could be sensitive to changes in task settings. This letter proposes an inverse reinforcement learning (IRL) based approach to recover both the variable impedance policy and reward function from expert demonstrations. We explore different action space of the reward functions to achieve a more general representation of expert variable impedance skills. Experiments on two variable impedance tasks (Peg-in-Hole and Cup-on-Plate) were conducted in both simulations and on a real FANUC LR Mate 200iD/7 L industrial robot. The comparison results with behavior cloning and force-based IRL proved that the learned reward function in the gain action space has better transferability than in the force space. Experiment videos are available at https://msc.berkeley.edu/research/impedance-irl.html .
- Open Access
Learning Variable Impedance Control via Inverse Reinforcement Learning for Force-Related Tasks
Many manipulation tasks require robots to interact with unknown environments. In such applications, the ability to adapt the impedance according to different task phases and environment constraints is crucial for safety and performance. Although many approaches based on deep reinforcement learning (RL) and learning from demonstration (LfD) have been proposed to obtain variable impedance skills on contact-rich manipulation tasks, these skills are typically task-specific and could be sensitive to changes in task settings. This paper proposes an inverse reinforcement learning (IRL) based approach to recover both the variable impedance policy and reward function from expert demonstrations. We explore different action space of the reward functions to achieve a more general representation of expert variable impedance skills. Experiments on two variable impedance tasks (Peg-in-Hole and Cup-on-Plate) were conducted in both simulations and on a real FANUC LR Mate 200iD/7L industrial robot. The comparison results with behavior cloning and force-based IRL proved that the learned reward function in the gain action space has better transferability than in the force space. Experiment videos are available at https://msc.berkeley.edu/research/impedance-irl.html.
Open AccessLearning Variable Impedance Control via Inverse Reinforcement Learning for Force-Related Tasks
Many manipulation tasks require robots to interact with unknown environments. In such applications, the ability to adapt the impedance according to different task phases and environment constraints is crucial for safety and performance. Although many approaches based on deep reinforcement learning (RL) and learning from demonstration (LfD) have been proposed to obtain variable impedance skills on contact-rich manipulation tasks, these skills are typically task-specific and could be sensitive to changes in task settings. This paper proposes an inverse reinforcement learning (IRL) based approach to recover both the variable impedance policy and reward function from expert demonstrations. We explore different action space of the reward functions to achieve a more general representation of expert variable impedance skills. Experiments on two variable impedance tasks (Peg-in-Hole and Cup-on-Plate) were conducted in both simulations and on a real FANUC LR Mate 200iD/7L industrial robot. The comparison results with behavior cloning and force-based IRL proved that the learned reward function in the gain action space has better transferability than in the force space. Experiment videos are available at https://msc.berkeley.edu/research/impedance-irl.html.
- Open Access
- 15 March 2019
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 38 (5) , 518-562
Action representations in robotics: A taxonomy and systematic classification
Understanding and defining the meaning of “action” is substantial for robotics research. This becomes utterly evident when aiming at equipping autonomous robots with robust manipulation skills for action execution. Unfortunately, to this day we still lack both a clear understanding of the concept of an action and a set of established criteria that ultimately characterize an action. In this survey, we thus first review existing ideas and theories on the notion and meaning of action. Subsequently, we discuss the role of action in robotics and attempt to give a seminal definition of action in accordance with its use in robotics research. Given this definition we then introduce a taxonomy for categorizing action representations in robotics along various dimensions. Finally, we provide a meticulous literature survey on action representations in robotics where we categorize relevant literature along our taxonomy. After discussing the current state of the art we conclude with an outlook towards promising research directions.
- 15 March 2019
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 38 (5) , 518-562
Open AccessAction representations in robotics: A taxonomy and systematic classification
Understanding and defining the meaning of “action” is substantial for robotics research. This becomes utterly evident when aiming at equipping autonomous robots with robust manipulation skills for action execution. Unfortunately, to this day we still lack both a clear understanding of the concept of an action and a set of established criteria that ultimately characterize an action. In this survey, we thus first review existing ideas and theories on the notion and meaning of action. Subsequently, we discuss the role of action in robotics and attempt to give a seminal definition of action in accordance with its use in robotics research. Given this definition we then introduce a taxonomy for categorizing action representations in robotics along various dimensions. Finally, we provide a meticulous literature survey on action representations in robotics where we categorize relevant literature along our taxonomy. After discussing the current state of the art we conclude with an outlook towards promising research directions.