Jiaming Qi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Jiaming Qi has a total of 45 co-authors. Here are the top co-authors by number of shared publications:
- Peng Zhou
School of Advanced Engineering, Great Bay University, Dongguan, China
10 shared publications - David Navarro-Alarcon
The Hong Kong Polytechnic University
7 shared publications - Jia Pan
The University of Hong Kong
4 shared publications - Liang Lu
Department of Computer Science, The University of Hong Kong, Kowloon Tong, Hong Kong
3 shared publications - Chenguang Yang
Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China
2 shared publications - Guangtao Ran
Harbin Institute of Technology
2 shared publications
Latest Publications
- 16 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (11) , 11355-11362
BagIt! An Adaptive Dual-Arm Manipulation of Fabric Bags for Object Bagging
Bagging tasks, commonly found in industrial scenarios, are challenging considering deformable bags' complicated and unpredictable nature. This paper presents an automated bagging system from the proposed adaptive Structure-of-Interest (SOI) manipulation strategy for dual robot arms. The system dynamically adjusts its actions based on real-time visual feedback, removing the need for pre-existing knowledge of bag properties. Our framework incorporates Gaussian Mixture Models (GMM) for estimating SOI states, optimization techniques for SOI generation, motion planning via Constrained Bidirectional Rapidly exploring Random Tree (CBiRRT), and dual-arm coordination using Model Predictive Control (MPC). Extensive experiments validate the capability of our system to perform precise and robust bagging across various objects, showcasing its adaptability. This work offers a new solution for robotic deformable object manipulation (DOM), particularly in automated bagging tasks
- 16 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (11) , 11355-11362
BagIt! An Adaptive Dual-Arm Manipulation of Fabric Bags for Object Bagging
Bagging tasks, commonly found in industrial scenarios, are challenging considering deformable bags' complicated and unpredictable nature. This paper presents an automated bagging system from the proposed adaptive Structure-of-Interest (SOI) manipulation strategy for dual robot arms. The system dynamically adjusts its actions based on real-time visual feedback, removing the need for pre-existing knowledge of bag properties. Our framework incorporates Gaussian Mixture Models (GMM) for estimating SOI states, optimization techniques for SOI generation, motion planning via Constrained Bidirectional Rapidly exploring Random Tree (CBiRRT), and dual-arm coordination using Model Predictive Control (MPC). Extensive experiments validate the capability of our system to perform precise and robust bagging across various objects, showcasing its adaptability. This work offers a new solution for robotic deformable object manipulation (DOM), particularly in automated bagging tasks
- 3 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 17814-17827
A Coarse-to-Fine Robotic Fabric Alignment System Integrating Visual Servoing and Admittance Control
Fabric alignment is essential to key production processes such as cutting, sewing, and fusing in garment manufacturing. Traditionally, this task has relied heavily on the dexterity and expertise of skilled human workers. Although automated systems have been introduced, they often lack the flexibility required for complex alignment tasks. In this paper, we present a novel robotic fabric alignment framework that fully automates the process with high precision and adaptability. First, we propose a coarse-to-fine alignment strategy, where an initial imprecise target position is roughly computed based on a basic perception module and eye-to-hand calibration. This is followed by a sliding mode control (SMC)-based visual servoing approach (in an eye-in-hand configuration) to ensure a close-up view of feedback features for the fine alignment process. Additionally, we consider system disturbances estimated by a fuzzy logic system (FLS) and combine it with the controller to further enhance the system’s robustness. Finally, we developed an advanced end-effector equipped with force/torque (F/T) sensors and air-powered needle grippers for gentle fabric manipulation using admittance control. We validate our framework through a series of experiments that demonstrate its effectiveness in fabric alignment tasks.
- 3 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 17814-17827
A Coarse-to-Fine Robotic Fabric Alignment System Integrating Visual Servoing and Admittance Control
Fabric alignment is essential to key production processes such as cutting, sewing, and fusing in garment manufacturing. Traditionally, this task has relied heavily on the dexterity and expertise of skilled human workers. Although automated systems have been introduced, they often lack the flexibility required for complex alignment tasks. In this paper, we present a novel robotic fabric alignment framework that fully automates the process with high precision and adaptability. First, we propose a coarse-to-fine alignment strategy, where an initial imprecise target position is roughly computed based on a basic perception module and eye-to-hand calibration. This is followed by a sliding mode control (SMC)-based visual servoing approach (in an eye-in-hand configuration) to ensure a close-up view of feedback features for the fine alignment process. Additionally, we consider system disturbances estimated by a fuzzy logic system (FLS) and combine it with the controller to further enhance the system’s robustness. Finally, we developed an advanced end-effector equipped with force/torque (F/T) sensors and air-powered needle grippers for gentle fabric manipulation using admittance control. We validate our framework through a series of experiments that demonstrate its effectiveness in fabric alignment tasks.
- 19 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 30 (5) , 3254-3265
Bimanual Deformable Bag Manipulation Using a Structure-of-Interest Based Neural Dynamics Model
The manipulation of deformable objects by robotic systems presents significant challenges due to their complex dynamics and infinite-dimensional configuration spaces. This article introduces a novel approach to deformable object manipulation (DOM) by emphasizing the introduction and manipulation of structures of interest (SOIs) in deformable fabric bags. We propose a bimanual manipulation framework that leverages a graph neural network-based neural dynamics model to succinctly represent and predict the behavior of these SOIs. Our approach involves global particle sampling process to construct a particle representation from partial point clouds of the SOIs and learning the neural dynamics model that effectively captures the essential deformations of the SOIs for fabric bags. By integrating this neural dynamics model with model predictive control, we enable robotic manipulators to perform precise and stable manipulation tasks focused on the SOIs. We validate our new framework through various experiments that demonstrate its efficacy in manipulating deformable bags and T-shirts. Our contributions not only address the complexities inherent in DOM, but also provide new perspectives and methodologies for enhancing robotic interactions with deformable materials by concentrating on their critical structural elements.
- 19 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 30 (5) , 3254-3265
Bimanual Deformable Bag Manipulation Using a Structure-of-Interest Based Neural Dynamics Model
The manipulation of deformable objects by robotic systems presents significant challenges due to their complex dynamics and infinite-dimensional configuration spaces. This article introduces a novel approach to deformable object manipulation (DOM) by emphasizing the introduction and manipulation of structures of interest (SOIs) in deformable fabric bags. We propose a bimanual manipulation framework that leverages a graph neural network-based neural dynamics model to succinctly represent and predict the behavior of these SOIs. Our approach involves global particle sampling process to construct a particle representation from partial point clouds of the SOIs and learning the neural dynamics model that effectively captures the essential deformations of the SOIs for fabric bags. By integrating this neural dynamics model with model predictive control, we enable robotic manipulators to perform precise and stable manipulation tasks focused on the SOIs. We validate our new framework through various experiments that demonstrate its efficacy in manipulating deformable bags and T-shirts. Our contributions not only address the complexities inherent in DOM, but also provide new perspectives and methodologies for enhancing robotic interactions with deformable materials by concentrating on their critical structural elements.
- Open Access
- 21 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (12) , 11258-11265
Efficient Planar Fabric Repositioning: Deformation-Aware RRT* for Non-Prehensile Fabric Manipulation
Fabrics present significant challenges to robotic manipulation due to their complex dynamics and infinite degrees of freedom. This letter proposes a non-prehensile approach to aligning a fabric cut piece to a specified target pose, which is a common step for many garment manufacturing tasks. Compared to widely explored prehensile manipulation that grasps and lifts the fabric, the proposed approach uses pushing actions and allows for efficient fabric repositioning with a simpler end-effector. To handle the possible deformation caused by the pushing actions, we introduce the Deformation-aware Rapidly-exploring Random Tree Star (D-RRT*) algorithm that strategically plans the contact poses and actions to slide the fabric to desired configurations by leveraging the friction. Our D-RRT* algorithm enhances fabric manipulation by incorporating deformation-aware action sampling into the path planning process, enabling accurate predictions of non-prehensile actions and efficient navigation through the fabric's configuration space. Extensive simulations and real-world experiments on repositioning fabrics of various shapes and materials demonstrate the effectiveness of the proposed pipeline in achieving stable and efficient manipulation.
- 21 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (12) , 11258-11265
Open AccessEfficient Planar Fabric Repositioning: Deformation-Aware RRT* for Non-Prehensile Fabric Manipulation
Fabrics present significant challenges to robotic manipulation due to their complex dynamics and infinite degrees of freedom. This letter proposes a non-prehensile approach to aligning a fabric cut piece to a specified target pose, which is a common step for many garment manufacturing tasks. Compared to widely explored prehensile manipulation that grasps and lifts the fabric, the proposed approach uses pushing actions and allows for efficient fabric repositioning with a simpler end-effector. To handle the possible deformation caused by the pushing actions, we introduce the Deformation-aware Rapidly-exploring Random Tree Star (D-RRT*) algorithm that strategically plans the contact poses and actions to slide the fabric to desired configurations by leveraging the friction. Our D-RRT* algorithm enhances fabric manipulation by incorporating deformation-aware action sampling into the path planning process, enabling accurate predictions of non-prehensile actions and efficient navigation through the fabric's configuration space. Extensive simulations and real-world experiments on repositioning fabrics of various shapes and materials demonstrate the effectiveness of the proposed pipeline in achieving stable and efficient manipulation.
- 7 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (10) , 8651-8658
Non-Prehensile Object Transport by Nonholonomic Robots Connected by Linear Deformable Elements
This letter presents a new method to automatically transport objects with mobile robots via non-prehensile actions. Our proposed approach utilizes a pair of nonholonomic robots connected by a deformable tube to efficiently manipulate objects of irregular shapes toward target locations. To autonomously perform this task, we develop a local integrated planning and control strategy that solves the problem in two steps (viz. enveloping and transport) based on the model predictive control (MPC) framework. The deformable underactuated system is simplified by a linear kinematic model. The enveloping problem is formulated as the minimization of multiple criteria that represent the enclosing error of the object by the variable morphology system. The transport problem is tackled by formulating the non-prehensile dragging action as an inequality constraint specified by the body frame of the deformable system. Reactive obstacle avoidance is ensured by a maximum margin-based term that utilizes the system's geometry and the feedback proximity to the environment. To validate the performance of the proposed methodology, we report a detailed experimental study with vision-guided robotic prototypes conducting multiple autonomous object transport tasks.
- 7 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (10) , 8651-8658
Non-Prehensile Object Transport by Nonholonomic Robots Connected by Linear Deformable Elements
This letter presents a new method to automatically transport objects with mobile robots via non-prehensile actions. Our proposed approach utilizes a pair of nonholonomic robots connected by a deformable tube to efficiently manipulate objects of irregular shapes toward target locations. To autonomously perform this task, we develop a local integrated planning and control strategy that solves the problem in two steps (viz. enveloping and transport) based on the model predictive control (MPC) framework. The deformable underactuated system is simplified by a linear kinematic model. The enveloping problem is formulated as the minimization of multiple criteria that represent the enclosing error of the object by the variable morphology system. The transport problem is tackled by formulating the non-prehensile dragging action as an inequality constraint specified by the body frame of the deformable system. Reactive obstacle avoidance is ensured by a maximum margin-based term that utilizes the system's geometry and the feedback proximity to the environment. To validate the performance of the proposed methodology, we report a detailed experimental study with vision-guided robotic prototypes conducting multiple autonomous object transport tasks.
- 13 May 2024
- Published by Elsevier in ISA Transactions
- Vol. 150, 359-373
Model predictive manipulation of compliant objects with multi-objective optimizer and adversarial network for occlusion compensation
- 13 May 2024
- Published by Elsevier in ISA Transactions
- Vol. 150, 359-373
Model predictive manipulation of compliant objects with multi-objective optimizer and adversarial network for occlusion compensation
- 29 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (4) , 3838-3845
Semantics-Aware Receding Horizon Planner for Object-Centric Active Mapping
The escalating demands for real-time scene comprehension in modern industries underscore the growing significance of semantic information in the daily tasks of robots, particularly in areas like autonomous inspection and target searching. This letter introduces a semantics-aware receding horizon planner (SARHP) for efficiently building the object-centric volumetric map. It includes a multi-layer mapping strategy and a semantics-aware frontier detection and planning method. With the multi-layer map, the semantics-aware frontier detection is conducted in the local layer, and the route assessment is conducted in the Field-of-View layer, which can reduce the time cost of the planning stage. Moreover, kinematic cost, geometric cost, and semantic cost are considered in the planner to ensure high search performance for semantic objects without affecting the overall mapping efficiency. The effectiveness of the proposed mapping and planning algorithm is validated in simulation and real-world experiments.
- 29 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (4) , 3838-3845
Semantics-Aware Receding Horizon Planner for Object-Centric Active Mapping
The escalating demands for real-time scene comprehension in modern industries underscore the growing significance of semantic information in the daily tasks of robots, particularly in areas like autonomous inspection and target searching. This letter introduces a semantics-aware receding horizon planner (SARHP) for efficiently building the object-centric volumetric map. It includes a multi-layer mapping strategy and a semantics-aware frontier detection and planning method. With the multi-layer map, the semantics-aware frontier detection is conducted in the local layer, and the route assessment is conducted in the Field-of-View layer, which can reduce the time cost of the planning stage. Moreover, kinematic cost, geometric cost, and semantic cost are considered in the planner to ensure high search performance for semantic objects without affecting the overall mapping efficiency. The effectiveness of the proposed mapping and planning algorithm is validated in simulation and real-world experiments.
- 1 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 20 (4) , 6245-6256
Imitating Tool-Based Garment Folding From a Single Visual Observation Using Hand-Object Graph Dynamics
Garment folding is a ubiquitous domestic task that is difficult to automate due to the highly deformable nature of fabrics. In this article, we propose a novel method of learning from demonstrations that enables robots to autonomously manipulate an assistive tool to fold garments. In contrast to traditional methods (that rely on low-level pixel features), our proposed solution uses a dense visual descriptor to encode the demonstration into a high-level hand-object graph (HoG) that allows to efficiently represent the interactions between the manipulated tool and robots. With that, we leverage graph neural network to autonomously learn the forward dynamics model from HoGs, then, given only a single demonstration, the imitation policy is optimized with a model predictive controller to accomplish the folding task. To validate the proposed approach, we conducted a detailed experimental study on a robotic platform instrumented with vision sensors and a custom-made end-effector that interacts with the folding board.
- 1 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 20 (4) , 6245-6256
Imitating Tool-Based Garment Folding From a Single Visual Observation Using Hand-Object Graph Dynamics
Garment folding is a ubiquitous domestic task that is difficult to automate due to the highly deformable nature of fabrics. In this article, we propose a novel method of learning from demonstrations that enables robots to autonomously manipulate an assistive tool to fold garments. In contrast to traditional methods (that rely on low-level pixel features), our proposed solution uses a dense visual descriptor to encode the demonstration into a high-level hand-object graph (HoG) that allows to efficiently represent the interactions between the manipulated tool and robots. With that, we leverage graph neural network to autonomously learn the forward dynamics model from HoGs, then, given only a single demonstration, the imitation policy is optimized with a model predictive controller to accomplish the folding task. To validate the proposed approach, we conducted a detailed experimental study on a robotic platform instrumented with vision sensors and a custom-made end-effector that interacts with the folding board.
- 25 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (2) , 1428-1435
Adaptive Shape Servoing of Elastic Rods Using Parameterized Regression Features and Auto-Tuning Motion Controls
The robotic manipulation of deformable linear objects has shown great potential in a wide range of real-world applications. However, it presents many challenges due to the objects' non-linear properties and high-dimensional geometric configuration. In this letter, we propose an efficient shape servoing framework to manipulate elastic objects through real-time visual feedbackAuthor: Please check and confirm whether the authors affiliations in the first footnote are correct as set. automatically. The proposed parameterized regression features are used to construct a compact (low-dimensional) feature vector (Bézier and NURBS) that quantifies the object's shape, thus enabling the establishment of an explicit shape servo-loop. To automatically manipulate the object into a desired configuration, our adaptive controller can iteratively estimate the sensorimotor model that relates the robot's motion and shape changes. This valuable capability enables the effective deformation of objects with unknown mechanical models. An auto-tuning algorithm is introduced to adjust the controller's gain and, thus, modulate the shaping motions based on optimal performance criteria. To validate the proposed framework, a detailed experimental study with vision-guided robot manipulators is presented.
- 25 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (2) , 1428-1435
Adaptive Shape Servoing of Elastic Rods Using Parameterized Regression Features and Auto-Tuning Motion Controls
The robotic manipulation of deformable linear objects has shown great potential in a wide range of real-world applications. However, it presents many challenges due to the objects' non-linear properties and high-dimensional geometric configuration. In this letter, we propose an efficient shape servoing framework to manipulate elastic objects through real-time visual feedbackAuthor: Please check and confirm whether the authors affiliations in the first footnote are correct as set. automatically. The proposed parameterized regression features are used to construct a compact (low-dimensional) feature vector (Bézier and NURBS) that quantifies the object's shape, thus enabling the establishment of an explicit shape servo-loop. To automatically manipulate the object into a desired configuration, our adaptive controller can iteratively estimate the sensorimotor model that relates the robot's motion and shape changes. This valuable capability enables the effective deformation of objects with unknown mechanical models. An auto-tuning algorithm is introduced to adjust the controller's gain and, thus, modulate the shaping motions based on optimal performance criteria. To validate the proposed framework, a detailed experimental study with vision-guided robot manipulators is presented.
- 27 January 2023
- Published by Taylor & Francis in Advanced Robotics
- Vol. 37 (9) , 576-590
Adaptive finite-time model estimation and control for manipulator visual servoing using sliding mode control and neural networks
The image-based visual servoing without models of the system is challenging since it is hard to fetch an accurate estimation of hand-eye relationship via merely visual measurement. Whereas, the accuracy of the estimated hand-eye relationship expressed in local linear format with the Jacobian matrix is important to the whole system's performance. In this article, we proposed a finite-time controller as well as a Jacobian matrix estimator in a combination of online and offline ways. The local linear formulation is formulated first. Then, we use a combination of online and offline methods to boost the estimation of the highly coupled and nonlinear hand-eye relationship with data collected via depth camera. A neural network (NN) is pre-trained to give a relative reasonable initial estimation of the Jacobian matrix. Then, an online updating method is carried out to modify the offline trained NN for a more accurate estimation. Moreover, a sliding mode control algorithm is introduced to realize a finite-time controller. Compared with previous methods, our algorithm possesses better convergence speed. The proposed estimator possesses excellent performance in the accuracy of initial estimation and powerful tracking capabilities for time-varying estimation for the Jacobian matrix compared with other data-driven estimators. The proposed scheme acquires the combination of neural network and finite-time control effect which drives a faster convergence speed compared with the exponentially converge ones. Another main feature of our algorithm is that the state signals in the system are proven to be semi-global practical finite-time stable. Several experiments are carried out to validate the proposed algorithm's performance. GRAPHICAL ABSTRACT
- 27 January 2023
- Published by Taylor & Francis in Advanced Robotics
- Vol. 37 (9) , 576-590
Adaptive finite-time model estimation and control for manipulator visual servoing using sliding mode control and neural networks
The image-based visual servoing without models of the system is challenging since it is hard to fetch an accurate estimation of hand-eye relationship via merely visual measurement. Whereas, the accuracy of the estimated hand-eye relationship expressed in local linear format with the Jacobian matrix is important to the whole system's performance. In this article, we proposed a finite-time controller as well as a Jacobian matrix estimator in a combination of online and offline ways. The local linear formulation is formulated first. Then, we use a combination of online and offline methods to boost the estimation of the highly coupled and nonlinear hand-eye relationship with data collected via depth camera. A neural network (NN) is pre-trained to give a relative reasonable initial estimation of the Jacobian matrix. Then, an online updating method is carried out to modify the offline trained NN for a more accurate estimation. Moreover, a sliding mode control algorithm is introduced to realize a finite-time controller. Compared with previous methods, our algorithm possesses better convergence speed. The proposed estimator possesses excellent performance in the accuracy of initial estimation and powerful tracking capabilities for time-varying estimation for the Jacobian matrix compared with other data-driven estimators. The proposed scheme acquires the combination of neural network and finite-time control effect which drives a faster convergence speed compared with the exponentially converge ones. Another main feature of our algorithm is that the state signals in the system are proven to be semi-global practical finite-time stable. Several experiments are carried out to validate the proposed algorithm's performance. GRAPHICAL ABSTRACT