Haegyeom Choi

In industrial environments, providing intuitive spatial information via 3D maps is essential for maximizing the efficiency of teleoperation. However, existing SLAM algorithms generating 3D maps predominantly focus on improving robot localization accuracy, often neglecting the optimization of viewability required for human operators to clearly perceive object depth and structure in virtual environments. To address this, this study proposes a methodology to optimize the viewability of RTAB-Map-based 3D maps using the Taguchi method, aiming to enhance VR teleoperation efficiency and reduce cognitive workload. We identified eight key parameters that critically affect visual quality and utilized an L18 orthogonal array to derive an optimal combination that controls point cloud density and noise levels. Experimental results from a target object picking task demonstrated that the optimized 3D map reduced task completion time by approximately 9 s compared to the RGB image condition, achieving efficiency levels approaching those of the physical-world baseline. Furthermore, evaluations using NASA-TLX confirmed that intuitive visual feedback minimized situational awareness errors and substantially alleviated cognitive workload. This study suggests a new direction for constructing high-efficiency teleoperation interfaces from a Human–Robot Interaction perspective by expanding SLAM optimization criteria from geometric precision to user-centric visual quality.

Telemanipulation-based object-side picking with a suction gripper often faces challenges such as occlusion of the target object or the gripper and the need for precise alignment between the suction cup and the object’s surface. These issues can significantly affect task success rates in logistics environments. To address these problems, this study proposes a multi-mode hand gesture-based virtual reality (VR) locomotion method to enable intuitive and precise viewpoint control. The system utilizes a head-mounted display (HMD) camera to capture hand skeleton data, which a multi-layer perceptron (MLP) model processes. The model classifies gestures into three modes: translation, rotation, and fixed, corresponding to fist, pointing, and unknown gestures, respectively. Translation mode moves the viewpoint based on the wrist’s displacement, rotation mode adjusts the viewpoint’s angle based on the wrist’s angular displacement, and fixed mode stabilizes the viewpoint when gestures are ambiguous. A dataset of 4312 frames was used for training and validation, with 666 frames for testing. The MLP model achieved a classification accuracy of 98.4%, with precision, recall, and F1-score exceeding 0.98. These results demonstrate the system’s ability to address the challenges of telemanipulation tasks by enabling accurate gesture recognition and seamless mode transitions.

This study proposes a method for time-efficient order picking based on a structural stability assessment (SSA) when target boxes inside box-stacking storage (BSS) on multi-layer racks are removed. This method performs optimal order picking by generating a path to directly pick the target box without first picking the upper boxes in the BBS, if it is possible to pick the target box directly. The SSA algorithm generates images of the complement structure by removing the target box within BBS and uses them as input data for the CNN model to evaluate the stability of the structure. To create the CNN model, we generated a dataset using CoppeliaSim simulation, considering the size and shape of the overall structure of the BBS, the size and number of each box, and the number of target boxes. The accuracy of the generated CNN model was 95.1% on test data, while it achieved 97% accuracy when using real-world data. This validation process confirmed that the algorithm can be effectively applied to real BBS logistics environments to perform optimal order picking.

This study presents a framework for enabling autonomous pick–place operations, addressing the need for efficiency in complex logistics environments using a direct multi-target teaching interface. First, tag and segmentation information were combined to recognize products in a complex warehouse, and a camera was installed on the rack to allow workers to remotely see the work environment, allowing workers to view the work environment in real time through a tablet. Workers can access the camera view showing the rack containing the target product through a swiping action and select the target product through direct teaching action. When the target product is finally selected, an optimal path is created through task planning, and an autonomous pick–place operation is performed based on the generated path. As a result of conducting a usability evaluation using the SUS (System Usability Scale) with six users on the interface that enables these tasks, it was confirmed that high user satisfaction was achieved with an average of 77.5 points. In conclusion, the proposed interface enhances operational efficiency and provides a user-friendly solution for complex warehouse tasks.

This study introduces a framework to improve upper extremity motion-based telemanipulation by component-wise rescaling (CWR) of spatial twist. This method allows for separate adjustments of linear and angular scaling parameters, significantly improving precision and dexterity even when the operator’s heading direction changes. By finely controlling both the linear and angular velocities independently, the CWR method enables more accurate telemanipulation in tasks requiring diverse speed and accuracy based on personal preferences or task-specific demands. The study conducted experiments confirming that operators could precisely control the robot gripper with a steady, controlled motion even in confined spaces, irrespective of changes in the subject’s body-heading direction. The performance evaluation of the proposed motion-scaling-based telemanipulation leveraged Optitrack’s motion-capture system, comparing the trajectories of the operator’s hand and the manipulator’s end effector (EEF). This verification process solidified the efficacy of the developed framework in enhancing telemanipulation performance.

This study proposes a telemanipulation framework with two wearable IMU sensors without human skeletal kinematics. First, the states (intensity and direction) of spatial hand-guiding gestures are separately estimated through the proposed state estimator, and the states are also combined with the gesture’s mode (linear, angular, and via) obtained with the bi-directional LSTM-based mode classifier. The spatial pose of the 6-DOF manipulator’s end-effector (EEF) can be controlled by combining the spatial linear and angular motions based on integrating the gesture’s mode and state. To validate the significance of the proposed method, the teleoperation of the EEF to the designated target poses was conducted in the motion-capture space. As a result, it was confirmed that the mode could be classified with 84.5% accuracy in real time, even during the operator’s dynamic movement; the direction could be estimated with an error of less than 1 degree; and the intensity could be successfully estimated with the gesture speed estimator and finely tuned with the scaling factor. Finally, it was confirmed that a subject could place the EEF within the average range of 83 mm and 2.56 degrees in the target pose with only less than ten consecutive hand-guiding gestures and visual inspection in the first trial.

The accuracy of the wearable inertia-measurement-unit (IMU)-sensor-based gesture recognition may be significantly affected by undesired changes in the body-fixed frame and the sensor-fixed frame according to the change in the subject and the sensor attachment. In this study, we proposed a novel wearable IMU-sensor-based hand-guiding gesture recognition method robust to significant changes in the subject’s body alignment based on the floating body-fixed frame method and the bi-directional long short-term memory (bi-LSTM). Through comparative experimental studies with the other two methods, it was confirmed that aligning the sensor-fixed frame with the reference frame of the human body and updating the reference frame according to the change in the subject’s body-heading direction helped improve the generalization performance of the gesture recognition model. As a result, the proposed floating body-fixed frame method showed a 91.7% test accuracy, confirming that it was appropriate for gesture recognition under significant changes in the subject’s body alignment during gestures.