HLHoi-Yin Lee

Hoi-Yin Lee

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Hoi-Yin Lee has a total of 26 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Iterative Shaping of Multi-Particle Aggregates Based on Action Trees and VLM

    by Hoi-Yin Lee, Peng Zhou, Anqing Duan, Chenguang Yang, David Navarro-Alarcon

    In this letter, we address the problem of manipulating multi-particle aggregates using a bimanual robotic system. Our approach enables the autonomous transport of dispersed particles through a series of shaping and pushing actions using robotically controlled tools. Achieving this advanced manipulation capability presents two key challenges: high-level task planning and trajectory execution. For task planning, we leverage Vision Language Models (VLMs) to enable primitive actions such as tool affordance grasping and non-prehensile particle pushing. For trajectory execution, we represent the evolving particle aggregate's contour using truncated Fourier series, providing efficient parametrization of its closed shape. We adaptively compute trajectory waypoints based on group cohesion and the geometric centroid of the aggregate, accounting for its spatial distribution and collective motion. Through real-world experiments, we demonstrate the effectiveness of our methodology in actively shaping and manipulating multi-particle aggregates while maintaining high system cohesion.


  • PSO-Based Optimal Coverage Path Planning for Surface Defect Inspection of 3C Components With a Robotic Line Scanner

    by Hongpeng Chen, Shengzeng Huo, Muhammad Muddassir, Hoi-Yin Lee, Yuli Liu, Junxi Li, Anqing Duan, Pai Zheng, David Navarro-Alarcon

    The automatic inspection of surface defects is an essential task for quality control in the computers, communications, and consumer electronics (3C) industry. Traditional inspection mechanisms (viz. line-scan sensors) have a limited field of view, thus, prompting the necessity for a multifaceted robotic inspection system capable of comprehensive scanning. Optimally selecting the robot’s viewpoints and planning a path is regarded as coverage path planning (CPP), a problem that enables inspecting the object’s complete surface while reducing the scanning time and avoiding misdetection of defects. In this paper, we present a new approach for robotic line scanners to detect surface defects of 3C free-form objects automatically. A two-stage region segmentation method defines the local scanning based on the random sample consensus (RANSAC) and K-means clustering to improve the inspection coverage. The proposed method also consists of an adaptive region-of-interest (ROI) algorithm to define the local scanning paths. Besides, a Particle Swarm Optimization (PSO)-based method is used for global inspection path generation to minimize the inspection time. The developed method is validated by simulation-based and experimental studies on various free-form workpieces, and its performance is compared with that of two state-of-the-art solutions. The reported results demonstrate the feasibility and effectiveness of our proposed method.


  • Human-Aware Reactive Task Planning of Sequential Robotic Manipulation Tasks

    by Wanyu Ma, Anqing Duan, Hoi-Yin Lee, Pai Zheng, David Navarro-Alarcon

    The recent emergence of Industry 5.0 underscores the need for increased autonomy in human–robot interaction (HRI), presenting both motivation and challenges in achieving resilient and energy-efficient production systems. To address this, in this article, we introduce a strategy for seamless collaboration between humans and robots in manufacturing and maintenance tasks. Our method enables smooth switching between temporary HRI (human-aware mode) and long-horizon automated manufacturing (fully automatic mode), effectively solving the human–robot coexistence problem. We develop a task progress monitor that decomposes complex tasks into robot-centric action sequences, further divided into three-phase subtasks. A trigger signal orchestrates mode switches based on detected human actions and their contribution to the task. In addition, we introduce a human agent coefficient matrix, computed using selected environmental features, to determine cut-points for reactive execution by each robot. To validate our approach, we conducted extensive experiments involving robotic manipulators performing representative manufacturing tasks in collaboration with humans. The results show promise for advancing HRI, offering pathways to enhancing sustainability within Industry 5.0. Our work lays the foundation for intelligent manufacturing processes in future societies, marking a pivotal step toward realizing the full potential of human–robot collaboration.


  • Bimanual Deformable Bag Manipulation Using a Structure-of-Interest Based Neural Dynamics Model

    by Peng Zhou, Pai Zheng, Jiaming Qi, Chengxi Li, Hoi-Yin Lee, Yipeng Pan, Chenguang Yang, David Navarro-Alarcon, Jia Pan

    The manipulation of deformable objects by robotic systems presents significant challenges due to their complex dynamics and infinite-dimensional configuration spaces. This article introduces a novel approach to deformable object manipulation (DOM) by emphasizing the introduction and manipulation of structures of interest (SOIs) in deformable fabric bags. We propose a bimanual manipulation framework that leverages a graph neural network-based neural dynamics model to succinctly represent and predict the behavior of these SOIs. Our approach involves global particle sampling process to construct a particle representation from partial point clouds of the SOIs and learning the neural dynamics model that effectively captures the essential deformations of the SOIs for fabric bags. By integrating this neural dynamics model with model predictive control, we enable robotic manipulators to perform precise and stable manipulation tasks focused on the SOIs. We validate our new framework through various experiments that demonstrate its efficacy in manipulating deformable bags and T-shirts. Our contributions not only address the complexities inherent in DOM, but also provide new perspectives and methodologies for enhancing robotic interactions with deformable materials by concentrating on their critical structural elements.


  • Robust Grasping by Bimanual Robots With Stable Parameterization-Based Contact Servoing

    by Anqing Duan, Shengzeng Huo, Hoi-Yin Lee, Peng Zhou, Jose Guadalupe Romero, Chenguang Yang, David Navarro-Alarcon

    Robots with bimanual morphology usually possess higher flexibility, dexterity, and efficiency than those only equipped with a single arm. The dual-arm structure has enabled robots to perform various intricate tasks that are difficult or even impossible to achieve by unimanipulation. In this article, we aim to achieve robust bimanual grasping for object transportation. In particular, provided that stable contact is the key to the success of the transportation task, our focus lies on stabilizing the contact between the object and the robot end-effectors by employing the contact servoing strategy. To ensure that the contact is stable, the contact wrenches are required to evolve within the so-called friction cones all the time throughout the transportation task. To this end, we propose stabilizing the contact by leveraging a novel contact parameterization model. Parameterization expresses the contact stability manifold with a set of constraint-free exogenous parameters where the mapping is bijective. Notably, such parameterization can guarantee that the contact stability constraints can always be satisfied. We also show that many commonly used contact models can be parameterized out of a similar principle. Furthermore, to exploit the parameterized contact models in the control law, we devise a contact servoing strategy for the bimanual robotic system such that the force feedback signals from the force/torque sensors are incorporated into the control loop. The effectiveness of the proposed approach is well demonstrated with the experiments on several representative bimanual transportation tasks.


  • A Distributed Dynamic Framework to Allocate Collaborative Tasks Based on Capability Matching in Heterogeneous Multirobot Systems

    by Hoi-Yin Lee, Peng Zhou, Bin Zhang, Liuming Qiu, Bowen Fan, Anqing Duan, Jingtao Tang, Tin Lun Lam, David Navarro-Alarcon

    Collaboration among a group of robots with heterogeneous capabilities is an important research problem that enables to combine different robot functionalities, and thus, conducts complex tasks that may be difficult to achieve by a single robot with limited resources. In this paper, we propose a new distributed task allocation framework based on the capability matching of heterogeneous robots. The framework is composed of an ontological dynamic knowledge graph model and a hardware control scheme to model the capability and optimize resource utilization for collaborative tasks. We introduce an intuitive hardware control scheme based on a dynamic knowledge graph that resolves possible conflicts between the hardware control of different types of robots. Action sequences are produced by a task and motion planning algorithm to collaboratively perform the assigned task. The performance of the proposed methodology is evaluated by both simulations and hardware experiments.


  • A Multisensor Interface to Improve the Learning Experience in Arc Welding Training Tasks

    by Hoi-Yin Lee, Peng Zhou, Anqing Duan, Jiangliu Wang, Victor Wu, David Navarro-Alarcon

    This article presents the development of a multisensor user interface to facilitate the instruction of arc welding tasks. Traditional methods to acquire hand-eye coordination skills are typically conducted through one-to-one instruction, where trainees must wear protective helmets and conduct several tests. These approaches are inefficient as the harmful light emitted from the electric arc impedes the close monitoring of the process. Practitioners can only observe a small bright spot. To tackle these problems, recent training approaches have leveraged virtual reality to safely simulate the process and visualize the geometry of the workpieces. However, the synthetic nature of these types of simulation platforms reduces their effectiveness as they fail to comprise actual welding interactions with the environment, which hinders the trainees' learning process. To provide users with a real welding experience, we have developed a new multisensor extended reality platform for arc welding training. Our system is composed of: 1) An HDR camera, monitoring the real welding spot in real time. 2) A depth sensor, capturing the 3-D geometry of the scene; and 3) A head-mounted VR display, visualizing the process safely. Our innovative platform provides users with a “bot trainer,” virtual cues of the seam geometry, automatic spot tracking, and performance scores. To validate the platform's feasibility, we conduct extensive experiments with several welding training tasks. We show that compared with the traditional training practice and recent virtual reality approaches, our automated multisensor method achieves better performances in terms of accuracy, learning curve, and effectiveness.


  • Self-Reconfigurable Soft-Rigid Mobile Agent With Variable Stiffness and Adaptive Morphology

    by Luiza Labazanova, Shuang Peng, Liuming Qiu, Hoi-Yin Lee, Thrishantha Nanayakkara, David Navarro-Alarcon

    In this letter, we propose a novel design of a hybrid mobile robot with controllable stiffness and deformable shape. Compared to conventional mobile agents, our system can switch between rigid and compliant phases by solidifying or melting Field's metal in its structure and, thus, alter the shape through the motion of its active components. In the soft state, the robot's main body can bend into circular arcs, which enables it to conform to surrounding curved objects. This variable geometry of the robot creates new motion modes which cannot be described by standard (i.e., fixed geometry) models. To this end, we develop a unified mathematical model that captures the differential kinematics of both rigid and soft states. An optimised control strategy is further proposed to select the most appropriate phase states and motion modes needed to reach a target pose-shape configuration. The performance of our new method is validated with numerical simulations and experiments conducted on a prototype system.