Taeho Kim

This study introduces a framework to improve upper extremity motion-based telemanipulation by component-wise rescaling (CWR) of spatial twist. This method allows for separate adjustments of linear and angular scaling parameters, significantly improving precision and dexterity even when the operator’s heading direction changes. By finely controlling both the linear and angular velocities independently, the CWR method enables more accurate telemanipulation in tasks requiring diverse speed and accuracy based on personal preferences or task-specific demands. The study conducted experiments confirming that operators could precisely control the robot gripper with a steady, controlled motion even in confined spaces, irrespective of changes in the subject’s body-heading direction. The performance evaluation of the proposed motion-scaling-based telemanipulation leveraged Optitrack’s motion-capture system, comparing the trajectories of the operator’s hand and the manipulator’s end effector (EEF). This verification process solidified the efficacy of the developed framework in enhancing telemanipulation performance.

This study proposes a telemanipulation framework with two wearable IMU sensors without human skeletal kinematics. First, the states (intensity and direction) of spatial hand-guiding gestures are separately estimated through the proposed state estimator, and the states are also combined with the gesture’s mode (linear, angular, and via) obtained with the bi-directional LSTM-based mode classifier. The spatial pose of the 6-DOF manipulator’s end-effector (EEF) can be controlled by combining the spatial linear and angular motions based on integrating the gesture’s mode and state. To validate the significance of the proposed method, the teleoperation of the EEF to the designated target poses was conducted in the motion-capture space. As a result, it was confirmed that the mode could be classified with 84.5% accuracy in real time, even during the operator’s dynamic movement; the direction could be estimated with an error of less than 1 degree; and the intensity could be successfully estimated with the gesture speed estimator and finely tuned with the scaling factor. Finally, it was confirmed that a subject could place the EEF within the average range of 83 mm and 2.56 degrees in the target pose with only less than ten consecutive hand-guiding gestures and visual inspection in the first trial.

This paper proposed a multi-layered 3D NDT (normal distribution transform) scan-matching approach for robust localization even in the highly dynamic environment of warehouse logistics. Our approach partitioned a given 3D point-cloud map and the scan measurements into several layers regarding the degree of environmental changes in the height direction and computed the covariance estimates for each layer using 3D NDT scan-matching. Because the covariance determinant is the estimate’s uncertainty, we can determine which layers are better to use in the localization in the warehouse. If the layer gets close to the warehouse’s floor, the degree of environmental changes, such as the cluttered warehouse layout and position of boxes, would be significantly large, while it has many good features for scan-matching. If the observation at a specific layer is not explained well enough, then the layer for localization can be switched to other layers with lower uncertainties. Thus, the main novelty of this approach is that localization robustness can be improved even in very cluttered and dynamic environments. This study also provides the simulation-based validation using Nvidia’s Omniverse Isaac sim and detailed mathematical descriptions for the proposed method. Moreover, the evaluated results of this study can be a good starting point for further mitigating the effects of occlusion in warehouse navigation of mobile robots.

The accuracy of the wearable inertia-measurement-unit (IMU)-sensor-based gesture recognition may be significantly affected by undesired changes in the body-fixed frame and the sensor-fixed frame according to the change in the subject and the sensor attachment. In this study, we proposed a novel wearable IMU-sensor-based hand-guiding gesture recognition method robust to significant changes in the subject’s body alignment based on the floating body-fixed frame method and the bi-directional long short-term memory (bi-LSTM). Through comparative experimental studies with the other two methods, it was confirmed that aligning the sensor-fixed frame with the reference frame of the human body and updating the reference frame according to the change in the subject’s body-heading direction helped improve the generalization performance of the gesture recognition model. As a result, the proposed floating body-fixed frame method showed a 91.7% test accuracy, confirming that it was appropriate for gesture recognition under significant changes in the subject’s body alignment during gestures.