Lipeng Chen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Lipeng Chen has a total of 71 co-authors. Here are the top co-authors by number of shared publications:
- Yu Zheng
Research Institute, UBTECH Robotics Inc., Shenzhen, China
4 shared publications - Jianing Qiu
The Chinese University of Hong Kong
3 shared publications - Benny Lo
Hamlyn Centre, Department of Surgery and Cancer, Imperial College London, London SW7 2AZ, UK
2 shared publications - Chaofan Zhang
Chinese Academy of Sciences
2 shared publications - Cong Xiao
State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, China
2 shared publications - Geng Yang
State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering Zhejiang University Hangzhou China
2 shared publications
Latest Publications
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (11) , 12496-12503
Toward Bimanual Dexterous Grasp of Partially Observable Objects With Data Synthesis and Diffusion-Based Generation Approach
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (11) , 12496-12503
Toward Bimanual Dexterous Grasp of Partially Observable Objects With Data Synthesis and Diffusion-Based Generation Approach
- 8 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Human-Machine Systems
- Vol. 56 (1) , 135-146
A Proactive Safety Architecture Based on Proximity Sensing for Enhanced Human-Robot Interaction in Tele-Homecare
With the advancement of robot technologies, robot-assisted tele-homecare systems enter human living environments to provide homecare services. In such a context, physical human-robot contact becomes inevitable. Ensuring safety during human-robot interaction in home environments becomes a critical challenge. However, conventional methods mainly rely on creating a collision-free environment, which is insufficient for inevitable or even desirable human-robot contact. This article proposes a proactive safety architecture that smoothly switches between collision avoidance and contact reaction. To enable proactive sensing for homecare robots, a proximity sensor is customized with the abilities of approach and contact awareness. Based on the proximity sensing data, a proactive safety architecture is proposed to ensure continuous task execution while avoiding potential collisions. For inevitable contact, a pretouch and contact reaction strategy is designed to enable a seamless transition from proximity-sensing-based collision avoidance to contact reaction. Comparative experiments are conducted to validate the proactive safety architecture on a telerobotic system prototype. Compared with the three state-of-the-art approaches, the proposed strategy reduces contact force (by at least 35.21% along the primary collision direction) while maximizing motion-tracking performance. A user study is conducted to investigate the user experience. Feedback from 10 participants highlights positive evaluations of this system’s usability, indicating the feasibility of the proposed strategy in enhancing safety during human-robot interaction for the tele-homecare system.
- 8 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Human-Machine Systems
- Vol. 56 (1) , 135-146
A Proactive Safety Architecture Based on Proximity Sensing for Enhanced Human-Robot Interaction in Tele-Homecare
With the advancement of robot technologies, robot-assisted tele-homecare systems enter human living environments to provide homecare services. In such a context, physical human-robot contact becomes inevitable. Ensuring safety during human-robot interaction in home environments becomes a critical challenge. However, conventional methods mainly rely on creating a collision-free environment, which is insufficient for inevitable or even desirable human-robot contact. This article proposes a proactive safety architecture that smoothly switches between collision avoidance and contact reaction. To enable proactive sensing for homecare robots, a proximity sensor is customized with the abilities of approach and contact awareness. Based on the proximity sensing data, a proactive safety architecture is proposed to ensure continuous task execution while avoiding potential collisions. For inevitable contact, a pretouch and contact reaction strategy is designed to enable a seamless transition from proximity-sensing-based collision avoidance to contact reaction. Comparative experiments are conducted to validate the proactive safety architecture on a telerobotic system prototype. Compared with the three state-of-the-art approaches, the proposed strategy reduces contact force (by at least 35.21% along the primary collision direction) while maximizing motion-tracking performance. A user study is conducted to investigate the user experience. Feedback from 10 participants highlights positive evaluations of this system’s usability, indicating the feasibility of the proposed strategy in enhancing safety during human-robot interaction for the tele-homecare system.
- 23 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. PP, 1-8
Safety-Aware Shared Control for Teleoperated Robotic Precision Tasks under Dynamic Interference
This study presents a safety-aware shared control strategy that combines proximity sensing and force guidance to achieve precise and stable teleoperation under dynamic interference. Based on the sensing information of the proximity sensor, a safety-aware controller is designed to enable the manipulator to anticipate and respond to potential collisions in its surroundings while ensuring precise task execution. By incorporating a prior-to-contact reaction strategy into the safety-aware controller, the manipulator can switch to a compliant mode, safely and effectively reacting to inevitable collisions, thereby mitigating the collision impact. Additionally, a force-constrained controller is integrated into the shared control strategy to enable stable movement by providing the operator with force guidance through haptic devices. The effectiveness and feasibility of this strategy are validated through experiments, which involve circular motion tracking and needle threading tasks under dynamic random interference. Compared to conventional teleoperation approaches that lack an active safety strategy or force guidance, the proposed strategy outperforms in reducing contact force by up to 78.81% while improving target tracking accuracy by up to 66.91%. The proposed strategy also enhances the stability of task execution under dynamic interference.
- 23 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. PP, 1-8
Safety-Aware Shared Control for Teleoperated Robotic Precision Tasks under Dynamic Interference
This study presents a safety-aware shared control strategy that combines proximity sensing and force guidance to achieve precise and stable teleoperation under dynamic interference. Based on the sensing information of the proximity sensor, a safety-aware controller is designed to enable the manipulator to anticipate and respond to potential collisions in its surroundings while ensuring precise task execution. By incorporating a prior-to-contact reaction strategy into the safety-aware controller, the manipulator can switch to a compliant mode, safely and effectively reacting to inevitable collisions, thereby mitigating the collision impact. Additionally, a force-constrained controller is integrated into the shared control strategy to enable stable movement by providing the operator with force guidance through haptic devices. The effectiveness and feasibility of this strategy are validated through experiments, which involve circular motion tracking and needle threading tasks under dynamic random interference. Compared to conventional teleoperation approaches that lack an active safety strategy or force guidance, the proposed strategy outperforms in reducing contact force by up to 78.81% while improving target tracking accuracy by up to 66.91%. The proposed strategy also enhances the stability of task execution under dynamic interference.
- 11 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 30 (6) , 7900-7911
Tactile-Based Prioritized Compliance for Mobile Manipulators: A Case Study of Human Walking Support
Large-scale tactile sensors facilitate mobile manipulators to perform close and safe physical human–robot interactions with the perception of physical contact. Particularly in applications such as elderly walking support, we introduce a dual-arm mobile manipulator equipped with tactile skin, enabling a more humanlike support mechanism. To make the robot support humans, we establish a task hierarchy; and to make it follow humans walking, we aim to strike a balance between tasks and the compliance to contact forces. If maintaining tasks with compliance in the null-space, the compliant motions will get constrained. In this article, we propose a method that shapes prioritized compliant behavior from null-space compliance (NS-C) to whole-body compliance for walking support. Our approach evaluates the feasibility of NS-C, and in barely feasible cases, hierarchically propagates infeasible force components into task space, allowing the robot to generate appropriate compliant motions. Using this method, we can shape various compliant behaviors through specific task settings. We validate the approach through experiments on the dual-arm mobile manipulator.
- 11 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 30 (6) , 7900-7911
Tactile-Based Prioritized Compliance for Mobile Manipulators: A Case Study of Human Walking Support
Large-scale tactile sensors facilitate mobile manipulators to perform close and safe physical human–robot interactions with the perception of physical contact. Particularly in applications such as elderly walking support, we introduce a dual-arm mobile manipulator equipped with tactile skin, enabling a more humanlike support mechanism. To make the robot support humans, we establish a task hierarchy; and to make it follow humans walking, we aim to strike a balance between tasks and the compliance to contact forces. If maintaining tasks with compliance in the null-space, the compliant motions will get constrained. In this article, we propose a method that shapes prioritized compliant behavior from null-space compliance (NS-C) to whole-body compliance for walking support. Our approach evaluates the feasibility of NS-C, and in barely feasible cases, hierarchically propagates infeasible force components into task space, allowing the robot to generate appropriate compliant motions. Using this method, we can shape various compliant behaviors through specific task settings. We validate the approach through experiments on the dual-arm mobile manipulator.
- 7 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 72 (5) , 5022-5031
Beyond End-Effector: Utilizing High-Resolution Tactile Signals for Physical Human–Robot Interaction
This article proposes a control framework for robots to apply their entire body as potential effectors in physical human–robot interaction (pHRI) tasks. This framework is implemented based on high-resolution electronic skin that covers the entire body of the robot. During pHRI, robots must respond appropriately to human intentions, necessitating advanced sensing capabilities to interpret tactile information effectively. However, both discerning human intention from such large-scale tactile data and accommodating interaction across the entire body’s surface present challenges. In this article, we propose a method to convert the large-area contact on a link into a contact center and estimate the corresponding wrench. Furthermore, we assign soft priorities and desired trajectories to each contact point and solve for the optimal joint velocities through quadratic programming (QP). By enabling a dual-arm mobile manipulator to dance the waltz with a human, our control framework has been validated for its effectiveness in handling multiple large-area contacts and time-varying pHRI tasks.
- 7 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 72 (5) , 5022-5031
Beyond End-Effector: Utilizing High-Resolution Tactile Signals for Physical Human–Robot Interaction
This article proposes a control framework for robots to apply their entire body as potential effectors in physical human–robot interaction (pHRI) tasks. This framework is implemented based on high-resolution electronic skin that covers the entire body of the robot. During pHRI, robots must respond appropriately to human intentions, necessitating advanced sensing capabilities to interpret tactile information effectively. However, both discerning human intention from such large-scale tactile data and accommodating interaction across the entire body’s surface present challenges. In this article, we propose a method to convert the large-area contact on a link into a contact center and estimate the corresponding wrench. Furthermore, we assign soft priorities and desired trajectories to each contact point and solve for the optimal joint velocities through quadratic programming (QP). By enabling a dual-arm mobile manipulator to dance the waltz with a human, our control framework has been validated for its effectiveness in handling multiple large-area contacts and time-varying pHRI tasks.
- 23 September 2024
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 44 (4) , 513-531
Advancing robots with greater dynamic dexterity: A large-scale multi-view and multi-modal dataset of human-human throw&catch of arbitrary objects
Learning and imitating behavioral intelligence from human demonstrations is a promising approach towards the intuitive programming of robots for enhanced dynamic dexterity. However, there has been no publicly available dataset in this domain. To address this gap, we introduce the first large-scale dataset and recording framework specifically designed for studying human collaborative dynamic dexterity in throw&catch tasks. The dataset, named H 2 TC, contains 15,000 multi-view and multi-modal synchronized recordings of diverse Human-Human Throw-and-Catch activities. It involves 34 human subjects with typical motor abilities and a variety of 52 objects frequently manipulated through throw&catch in domestic and/or industrial scenarios. The dataset is supplemented with a hierarchy of manually annotated semantic and dense labels, such as the ground truth human body, hand and object motions captured with specialized high-precision motion tracking systems. These rich annotations make the dataset well-suited for a wide range of robot studies, including both low-level motor skill learning and high-level cognitive planning and recognition. We envision that the proposed dataset and recording framework will facilitate learning pipelines to extract insights on how humans coordinate both intra- and interpersonally to throw and catch objects, ultimately leading to the development of more capable and collaborative robots. The dataset, along with a suite of utility tools, such as those for visualization and annotation, can be accessed from our project page at https://h2tc-roboticsx.github.io/ .
- 23 September 2024
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 44 (4) , 513-531
Advancing robots with greater dynamic dexterity: A large-scale multi-view and multi-modal dataset of human-human throw&catch of arbitrary objects
Learning and imitating behavioral intelligence from human demonstrations is a promising approach towards the intuitive programming of robots for enhanced dynamic dexterity. However, there has been no publicly available dataset in this domain. To address this gap, we introduce the first large-scale dataset and recording framework specifically designed for studying human collaborative dynamic dexterity in throw&catch tasks. The dataset, named H 2 TC, contains 15,000 multi-view and multi-modal synchronized recordings of diverse Human-Human Throw-and-Catch activities. It involves 34 human subjects with typical motor abilities and a variety of 52 objects frequently manipulated through throw&catch in domestic and/or industrial scenarios. The dataset is supplemented with a hierarchy of manually annotated semantic and dense labels, such as the ground truth human body, hand and object motions captured with specialized high-precision motion tracking systems. These rich annotations make the dataset well-suited for a wide range of robot studies, including both low-level motor skill learning and high-level cognitive planning and recognition. We envision that the proposed dataset and recording framework will facilitate learning pipelines to extract insights on how humans coordinate both intra- and interpersonally to throw and catch objects, ultimately leading to the development of more capable and collaborative robots. The dataset, along with a suite of utility tools, such as those for visualization and annotation, can be accessed from our project page at https://h2tc-roboticsx.github.io/ .
- 18 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 40 (15523098) , 3232-3251
TossNet: Learning to Accurately Measure and Predict Robot Throwing of Arbitrary Objects in Real Time With Proprioceptive Sensing
Accurate measuring and modeling of dynamic robot manipulation (e.g., tossing and catching) is particularly challenging, due to the inherent nonlinearity, complexity, and uncertainty in high-speed robot motions and highly dynamic robot-object interactions happening in very short distances and times. Most studies leverage extrinsic sensors such as visual and tactile feedback towards task or object-centric modeling of manipulation dynamics, which however may hit bottleneck due to the significant cost and complexity, e.g., the environmental restrictions. In this work, we investigate whether using solely the on-board proprioceptive sensory modalities can effectively capture and characterize dynamic manipulation processes. In particular, we present an object-agnostic strategy to learn the robot toss dynamics of arbitrary unknown objects from the spatio-temporal variations of robot toss movements and wrist-force/torque (F/T) observations. We then propose TossNet, an end-to-end formulation that jointly measures the robot toss dynamics and predicts the resulting flying trajectories of the tossed objects. Experimental results in both simulation and real-world scenarios demonstrate that our methods can accurately model the robot toss dynamics of both seen and unseen objects, and predict their flying trajectories with superior prediction accuracy in nearly real-time. Ablative results are also presented to demonstrate the effectiveness of each proprioceptive modality and their correlations in modeling the toss dynamics. Case studies show that TossNet can be applied on various real robot platforms for challenging tossing-centric robot applications, such as blind juggling and high-precise robot pitching. A video of the real robot experiments can be found at https://youtu.be/y_Bz9debQyg .
- 18 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 40 (15523098) , 3232-3251
TossNet: Learning to Accurately Measure and Predict Robot Throwing of Arbitrary Objects in Real Time With Proprioceptive Sensing
Accurate measuring and modeling of dynamic robot manipulation (e.g., tossing and catching) is particularly challenging, due to the inherent nonlinearity, complexity, and uncertainty in high-speed robot motions and highly dynamic robot-object interactions happening in very short distances and times. Most studies leverage extrinsic sensors such as visual and tactile feedback towards task or object-centric modeling of manipulation dynamics, which however may hit bottleneck due to the significant cost and complexity, e.g., the environmental restrictions. In this work, we investigate whether using solely the on-board proprioceptive sensory modalities can effectively capture and characterize dynamic manipulation processes. In particular, we present an object-agnostic strategy to learn the robot toss dynamics of arbitrary unknown objects from the spatio-temporal variations of robot toss movements and wrist-force/torque (F/T) observations. We then propose TossNet, an end-to-end formulation that jointly measures the robot toss dynamics and predicts the resulting flying trajectories of the tossed objects. Experimental results in both simulation and real-world scenarios demonstrate that our methods can accurately model the robot toss dynamics of both seen and unseen objects, and predict their flying trajectories with superior prediction accuracy in nearly real-time. Ablative results are also presented to demonstrate the effectiveness of each proprioceptive modality and their correlations in modeling the toss dynamics. Case studies show that TossNet can be applied on various real robot platforms for challenging tossing-centric robot applications, such as blind juggling and high-precise robot pitching. A video of the real robot experiments can be found at https://youtu.be/y_Bz9debQyg .
- 12 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
A Learning Framework for Human-Like Time Parameterization of Robot Manipulation Paths
This paper presents a transformer-based approach to time-parameterize robot manipulation paths in human-centred environments. We propose to build an implicit neural function, named Human Mimic Timing Function (HMTF) to model the spatio-temporal correlations in human motions, which in turn are used to generate human-like and path-consistent temporal features for robot manipulation motions. We train a multi-modal HMTF by imitating human demonstrations and integrate it into a sampling-based pipeline for human-like robot motion planning. Experimental results show that through training on large-scale human demonstrations, our HMTF allows generalization to different manipulation modalities, and can generate smooth and human-like temporal features for random robot paths. Compared with time-optimal path parameterization (TOPP), HMTF is more versatile and achieves significant improvement in the temporal human-likeness when applied to parameterize robot manipulation paths.
- 12 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
A Learning Framework for Human-Like Time Parameterization of Robot Manipulation Paths
This paper presents a transformer-based approach to time-parameterize robot manipulation paths in human-centred environments. We propose to build an implicit neural function, named Human Mimic Timing Function (HMTF) to model the spatio-temporal correlations in human motions, which in turn are used to generate human-like and path-consistent temporal features for robot manipulation motions. We train a multi-modal HMTF by imitating human demonstrations and integrate it into a sampling-based pipeline for human-like robot motion planning. Experimental results show that through training on large-scale human demonstrations, our HMTF allows generalization to different manipulation modalities, and can generate smooth and human-like temporal features for random robot paths. Compared with time-optimal path parameterization (TOPP), HMTF is more versatile and achieves significant improvement in the temporal human-likeness when applied to parameterize robot manipulation paths.
- Open Access
- 8 August 2023
- Published by Springer Nature in Complex & Intelligent Systems
- Vol. 10 (1) , 811-823
Learning high-level robotic manipulation actions with visual predictive model
Learning visual predictive models has great potential for real-world robot manipulations. Visual predictive models serve as a model of real-world dynamics to comprehend the interactions between the robot and objects. However, prior works in the literature have focused mainly on low-level elementary robot actions, which typically result in lengthy, inefficient, and highly complex robot manipulation. In contrast, humans usually employ top–down thinking of high-level actions rather than bottom–up stacking of low-level ones. To address this limitation, we present a novel formulation for robot manipulation that can be accomplished by pick-and-place, a commonly applied high-level robot action, through grasping. We propose a novel visual predictive model that combines an action decomposer and a video prediction network to learn the intrinsic semantic information of high-level actions. Experiments show that our model can accurately predict the object dynamics (i.e., the object movements under robot manipulation) while trained directly on observations of high-level pick-and-place actions. We also demonstrate that, together with a sampling-based planner, our model achieves a higher success rate using high-level actions on a variety of real robot manipulation tasks.
- 8 August 2023
- Published by Springer Nature in Complex & Intelligent Systems
- Vol. 10 (1) , 811-823
Open AccessLearning high-level robotic manipulation actions with visual predictive model
Learning visual predictive models has great potential for real-world robot manipulations. Visual predictive models serve as a model of real-world dynamics to comprehend the interactions between the robot and objects. However, prior works in the literature have focused mainly on low-level elementary robot actions, which typically result in lengthy, inefficient, and highly complex robot manipulation. In contrast, humans usually employ top–down thinking of high-level actions rather than bottom–up stacking of low-level ones. To address this limitation, we present a novel formulation for robot manipulation that can be accomplished by pick-and-place, a commonly applied high-level robot action, through grasping. We propose a novel visual predictive model that combines an action decomposer and a video prediction network to learn the intrinsic semantic information of high-level actions. Experiments show that our model can accurately predict the object dynamics (i.e., the object movements under robot manipulation) while trained directly on observations of high-level pick-and-place actions. We also demonstrate that, together with a sampling-based planner, our model achieves a higher success rate using high-level actions on a variety of real robot manipulation tasks.
- Open Access
- 12 April 2023
- Published by Springer Nature in Complex & Intelligent Systems
- Vol. 9 (5) , 5923-5936
Learning visual-based deformable object rearrangement with local graph neural networks
Goal-conditioned rearrangement of deformable objects (e.g. straightening a rope and folding a cloth) is one of the most common deformable manipulation tasks, where the robot needs to rearrange a deformable object into a prescribed goal configuration with only visual observations. These tasks are typically confronted with two main challenges: the high dimensionality of deformable configuration space and the underlying complexity, nonlinearity and uncertainty inherent in deformable dynamics. To address these challenges, we propose a novel representation strategy that can efficiently model the deformable object states with a set of keypoints and their interactions. We further propose local-graph neural network (GNN), a light local GNN learning to jointly model the deformable rearrangement dynamics and infer the optimal manipulation actions (e.g. pick and place) by constructing and updating two dynamic graphs. Both simulated and real experiments have been conducted to demonstrate that the proposed dynamic graph representation shows superior expressiveness in modeling deformable rearrangement dynamics. Our method reaches much higher success rates on a variety of deformable rearrangement tasks (96.3% on average) than state-of-the-art method in simulation experiments. Besides, our method is much more lighter and has a 60% shorter inference time than state-of-the-art methods. We also demonstrate that our method performs well in the multi-task learning scenario and can be transferred to real-world applications with an average success rate of 95% by solely fine tuning a keypoint detector. A supplementary video can be found at https://youtu.be/AhwTQo6fCM0.
- 12 April 2023
- Published by Springer Nature in Complex & Intelligent Systems
- Vol. 9 (5) , 5923-5936
Open AccessLearning visual-based deformable object rearrangement with local graph neural networks
Goal-conditioned rearrangement of deformable objects (e.g. straightening a rope and folding a cloth) is one of the most common deformable manipulation tasks, where the robot needs to rearrange a deformable object into a prescribed goal configuration with only visual observations. These tasks are typically confronted with two main challenges: the high dimensionality of deformable configuration space and the underlying complexity, nonlinearity and uncertainty inherent in deformable dynamics. To address these challenges, we propose a novel representation strategy that can efficiently model the deformable object states with a set of keypoints and their interactions. We further propose local-graph neural network (GNN), a light local GNN learning to jointly model the deformable rearrangement dynamics and infer the optimal manipulation actions (e.g. pick and place) by constructing and updating two dynamic graphs. Both simulated and real experiments have been conducted to demonstrate that the proposed dynamic graph representation shows superior expressiveness in modeling deformable rearrangement dynamics. Our method reaches much higher success rates on a variety of deformable rearrangement tasks (96.3% on average) than state-of-the-art method in simulation experiments. Besides, our method is much more lighter and has a 60% shorter inference time than state-of-the-art methods. We also demonstrate that our method performs well in the multi-task learning scenario and can be transferred to real-world applications with an average success rate of 95% by solely fine tuning a keypoint detector. A supplementary video can be found at https://youtu.be/AhwTQo6fCM0.