Haneul Jeon
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Haneul Jeon has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Donghun Lee
Department of Mechanical Engineering, Soongsil University, Seoul 06978, Republic of Korea
9 shared publications - Taeho Kim
Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea
4 shared publications - Donghyeon Noh
Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea
3 shared publications - Haegyeom Choi
Department of Mechanical Engineering, Soongsil University, Seoul 06978, Republic of Korea
3 shared publications - Hyungjin Jeong
Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea
1 shared publication - Jiwon Yu
Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea
1 shared publication
Latest Publications
- Open Access
Bi-Directional Long Short-Term Memory-Based Gait Phase Recognition Method Robust to Directional Variations in Subject’s Gait Progression Using Wearable Inertial Sensor
Inertial Measurement Unit (IMU) sensor-based gait phase recognition is widely used in medical and biomechanics fields requiring gait data analysis. However, there are several limitations due to the low reproducibility of IMU sensor attachment and the sensor outputs relative to a fixed reference frame. The prediction algorithm may malfunction when the user changes their walking direction. In this paper, we propose a gait phase recognition method robust to user body movements based on a floating body-fixed frame (FBF) and bi-directional long short-term memory (bi-LSTM). Data from four IMU sensors attached to the shanks and feet on both legs of three subjects, collected via the FBF method, are processed through preprocessing and the sliding window label overlapping method before inputting into the bi-LSTM for training. To improve the model’s recognition accuracy, we selected parameters that influence both training and test accuracy. We conducted a sensitivity analysis using a level average analysis of the Taguchi method to identify the optimal combination of parameters. The model, trained with optimal parameters, was validated on a new subject, achieving a high test accuracy of 86.43%.
Open AccessBi-Directional Long Short-Term Memory-Based Gait Phase Recognition Method Robust to Directional Variations in Subject’s Gait Progression Using Wearable Inertial Sensor
Inertial Measurement Unit (IMU) sensor-based gait phase recognition is widely used in medical and biomechanics fields requiring gait data analysis. However, there are several limitations due to the low reproducibility of IMU sensor attachment and the sensor outputs relative to a fixed reference frame. The prediction algorithm may malfunction when the user changes their walking direction. In this paper, we propose a gait phase recognition method robust to user body movements based on a floating body-fixed frame (FBF) and bi-directional long short-term memory (bi-LSTM). Data from four IMU sensors attached to the shanks and feet on both legs of three subjects, collected via the FBF method, are processed through preprocessing and the sliding window label overlapping method before inputting into the bi-LSTM for training. To improve the model’s recognition accuracy, we selected parameters that influence both training and test accuracy. We conducted a sensitivity analysis using a level average analysis of the Taguchi method to identify the optimal combination of parameters. The model, trained with optimal parameters, was validated on a new subject, achieving a high test accuracy of 86.43%.
- Open Access
- 22 January 2024
- Published by MDPI AG in Mathematics
- Vol. 12 (2) , 358
Upper Extremity Motion-Based Telemanipulation with Component-Wise Rescaling of Spatial Twist and Parameter-Invariant Skeletal Kinematics
This study introduces a framework to improve upper extremity motion-based telemanipulation by component-wise rescaling (CWR) of spatial twist. This method allows for separate adjustments of linear and angular scaling parameters, significantly improving precision and dexterity even when the operator’s heading direction changes. By finely controlling both the linear and angular velocities independently, the CWR method enables more accurate telemanipulation in tasks requiring diverse speed and accuracy based on personal preferences or task-specific demands. The study conducted experiments confirming that operators could precisely control the robot gripper with a steady, controlled motion even in confined spaces, irrespective of changes in the subject’s body-heading direction. The performance evaluation of the proposed motion-scaling-based telemanipulation leveraged Optitrack’s motion-capture system, comparing the trajectories of the operator’s hand and the manipulator’s end effector (EEF). This verification process solidified the efficacy of the developed framework in enhancing telemanipulation performance.
- 22 January 2024
- Published by MDPI AG in Mathematics
- Vol. 12 (2) , 358
Open AccessUpper Extremity Motion-Based Telemanipulation with Component-Wise Rescaling of Spatial Twist and Parameter-Invariant Skeletal Kinematics
This study introduces a framework to improve upper extremity motion-based telemanipulation by component-wise rescaling (CWR) of spatial twist. This method allows for separate adjustments of linear and angular scaling parameters, significantly improving precision and dexterity even when the operator’s heading direction changes. By finely controlling both the linear and angular velocities independently, the CWR method enables more accurate telemanipulation in tasks requiring diverse speed and accuracy based on personal preferences or task-specific demands. The study conducted experiments confirming that operators could precisely control the robot gripper with a steady, controlled motion even in confined spaces, irrespective of changes in the subject’s body-heading direction. The performance evaluation of the proposed motion-scaling-based telemanipulation leveraged Optitrack’s motion-capture system, comparing the trajectories of the operator’s hand and the manipulator’s end effector (EEF). This verification process solidified the efficacy of the developed framework in enhancing telemanipulation performance.
- Open Access
- 18 December 2023
- Published by MDPI AG in Mathematics
- Vol. 11 (24) , 4995
Optimal Joint Path Planning of a New Virtual-Linkage-Based Redundant Finishing Stage for Additive-Finishing Integrated Manufacturing
This paper describes the optimal path planning of a redundant finishing mechanism developed for joint space-based additive-finishing integrated manufacturing (AFM). The research motivation results from an inevitable one-sided layout of a finishing stage (FS) with regard to the additive stage (AS) in the AFM. These two stages share a 2-dof bed stage (BS), and the FS can lightly shave off the rough-surfaced 3D print on the bed. Since the FS located at the side of the AS cannot reach all the target points of the 3D print, the bed should be able to rotate the 3D print about the z-axis and translate it in the z-axis. As a result, the AS has 4-dof joints for 2P and 1P1R during the additive process with AS-BS, and FS has 4-dof and 2-dof integrated joints for 2P2R and 1P1R during the finishing process with FS-BS, respectively. For the kinematic modeling of the FS part and the BS, the virtual linkage connecting the bed frame origin and the FS’s joint frame for approaching the BS is considered to realize seamless kinematic redundancy. The minimum Euclidian norm of the joint velocity space is the objective function to find the optimal joint space solution for a given tool path. To confirm the feasibility of the developed joint path planning algorithm in the proposed FS-BS mechanism, layer-by-layer slicing of a given 3D print’s CAD model and tool path generation were performed. Then, the numerical simulations of the optimal joint path planning for some primitive 3D print geometries were conducted. As a result, we confirmed that the maximum and mean pose error in point-by-point only, with the developed optimal joint path planning algorithm, were less than 202 nm and 153 nm, respectively. Since precision and general machining accuracies in tool path generation are in the range of ±10 μm and 20 μm, the pose error in this study fully satisfies the industry requirements.
- 18 December 2023
- Published by MDPI AG in Mathematics
- Vol. 11 (24) , 4995
Open AccessOptimal Joint Path Planning of a New Virtual-Linkage-Based Redundant Finishing Stage for Additive-Finishing Integrated Manufacturing
This paper describes the optimal path planning of a redundant finishing mechanism developed for joint space-based additive-finishing integrated manufacturing (AFM). The research motivation results from an inevitable one-sided layout of a finishing stage (FS) with regard to the additive stage (AS) in the AFM. These two stages share a 2-dof bed stage (BS), and the FS can lightly shave off the rough-surfaced 3D print on the bed. Since the FS located at the side of the AS cannot reach all the target points of the 3D print, the bed should be able to rotate the 3D print about the z-axis and translate it in the z-axis. As a result, the AS has 4-dof joints for 2P and 1P1R during the additive process with AS-BS, and FS has 4-dof and 2-dof integrated joints for 2P2R and 1P1R during the finishing process with FS-BS, respectively. For the kinematic modeling of the FS part and the BS, the virtual linkage connecting the bed frame origin and the FS’s joint frame for approaching the BS is considered to realize seamless kinematic redundancy. The minimum Euclidian norm of the joint velocity space is the objective function to find the optimal joint space solution for a given tool path. To confirm the feasibility of the developed joint path planning algorithm in the proposed FS-BS mechanism, layer-by-layer slicing of a given 3D print’s CAD model and tool path generation were performed. Then, the numerical simulations of the optimal joint path planning for some primitive 3D print geometries were conducted. As a result, we confirmed that the maximum and mean pose error in point-by-point only, with the developed optimal joint path planning algorithm, were less than 202 nm and 153 nm, respectively. Since precision and general machining accuracies in tool path generation are in the range of ±10 μm and 20 μm, the pose error in this study fully satisfies the industry requirements.
- Open Access
- 14 August 2023
- Published by MDPI AG in Mathematics
- Vol. 11 (16) , 3514
Hand-Guiding Gesture-Based Telemanipulation with the Gesture Mode Classification and State Estimation Using Wearable IMU Sensors
This study proposes a telemanipulation framework with two wearable IMU sensors without human skeletal kinematics. First, the states (intensity and direction) of spatial hand-guiding gestures are separately estimated through the proposed state estimator, and the states are also combined with the gesture’s mode (linear, angular, and via) obtained with the bi-directional LSTM-based mode classifier. The spatial pose of the 6-DOF manipulator’s end-effector (EEF) can be controlled by combining the spatial linear and angular motions based on integrating the gesture’s mode and state. To validate the significance of the proposed method, the teleoperation of the EEF to the designated target poses was conducted in the motion-capture space. As a result, it was confirmed that the mode could be classified with 84.5% accuracy in real time, even during the operator’s dynamic movement; the direction could be estimated with an error of less than 1 degree; and the intensity could be successfully estimated with the gesture speed estimator and finely tuned with the scaling factor. Finally, it was confirmed that a subject could place the EEF within the average range of 83 mm and 2.56 degrees in the target pose with only less than ten consecutive hand-guiding gestures and visual inspection in the first trial.
- 14 August 2023
- Published by MDPI AG in Mathematics
- Vol. 11 (16) , 3514
Open AccessHand-Guiding Gesture-Based Telemanipulation with the Gesture Mode Classification and State Estimation Using Wearable IMU Sensors
This study proposes a telemanipulation framework with two wearable IMU sensors without human skeletal kinematics. First, the states (intensity and direction) of spatial hand-guiding gestures are separately estimated through the proposed state estimator, and the states are also combined with the gesture’s mode (linear, angular, and via) obtained with the bi-directional LSTM-based mode classifier. The spatial pose of the 6-DOF manipulator’s end-effector (EEF) can be controlled by combining the spatial linear and angular motions based on integrating the gesture’s mode and state. To validate the significance of the proposed method, the teleoperation of the EEF to the designated target poses was conducted in the motion-capture space. As a result, it was confirmed that the mode could be classified with 84.5% accuracy in real time, even during the operator’s dynamic movement; the direction could be estimated with an error of less than 1 degree; and the intensity could be successfully estimated with the gesture speed estimator and finely tuned with the scaling factor. Finally, it was confirmed that a subject could place the EEF within the average range of 83 mm and 2.56 degrees in the target pose with only less than ten consecutive hand-guiding gestures and visual inspection in the first trial.
- Open Access
A Multi-Layered 3D NDT Scan-Matching Method for Robust Localization in Logistics Warehouse Environments
This paper proposed a multi-layered 3D NDT (normal distribution transform) scan-matching approach for robust localization even in the highly dynamic environment of warehouse logistics. Our approach partitioned a given 3D point-cloud map and the scan measurements into several layers regarding the degree of environmental changes in the height direction and computed the covariance estimates for each layer using 3D NDT scan-matching. Because the covariance determinant is the estimate’s uncertainty, we can determine which layers are better to use in the localization in the warehouse. If the layer gets close to the warehouse’s floor, the degree of environmental changes, such as the cluttered warehouse layout and position of boxes, would be significantly large, while it has many good features for scan-matching. If the observation at a specific layer is not explained well enough, then the layer for localization can be switched to other layers with lower uncertainties. Thus, the main novelty of this approach is that localization robustness can be improved even in very cluttered and dynamic environments. This study also provides the simulation-based validation using Nvidia’s Omniverse Isaac sim and detailed mathematical descriptions for the proposed method. Moreover, the evaluated results of this study can be a good starting point for further mitigating the effects of occlusion in warehouse navigation of mobile robots.
Open AccessA Multi-Layered 3D NDT Scan-Matching Method for Robust Localization in Logistics Warehouse Environments
This paper proposed a multi-layered 3D NDT (normal distribution transform) scan-matching approach for robust localization even in the highly dynamic environment of warehouse logistics. Our approach partitioned a given 3D point-cloud map and the scan measurements into several layers regarding the degree of environmental changes in the height direction and computed the covariance estimates for each layer using 3D NDT scan-matching. Because the covariance determinant is the estimate’s uncertainty, we can determine which layers are better to use in the localization in the warehouse. If the layer gets close to the warehouse’s floor, the degree of environmental changes, such as the cluttered warehouse layout and position of boxes, would be significantly large, while it has many good features for scan-matching. If the observation at a specific layer is not explained well enough, then the layer for localization can be switched to other layers with lower uncertainties. Thus, the main novelty of this approach is that localization robustness can be improved even in very cluttered and dynamic environments. This study also provides the simulation-based validation using Nvidia’s Omniverse Isaac sim and detailed mathematical descriptions for the proposed method. Moreover, the evaluated results of this study can be a good starting point for further mitigating the effects of occlusion in warehouse navigation of mobile robots.
- Open Access
- 10 February 2023
- Published by MDPI AG in Mathematics
- Vol. 11 (4) , 905
Explicit Identification of Pointwise Terrain Gradients for Speed Compensation of Four Driving Tracks in Passively Articulated Tracked Mobile Robot
Tracked mobile robots can overcome the limitations of wheeled and legged robots in environments, such as construction and mining, but there are still significant challenges to be addressed in terms of trajectory tracking. This study proposes a kinematic strategy to improve the trajectory-tracking performance of a PASTRo (Passively Articulated Suspension based Track-typed mobile robot), which comprises four tracks, two rockers, a differential gear, and a main body. Due to the difficulties in explicitly identifying track-terrain contact angles, suspension kinematics is used to identify track-terrain contact angles (TTCA) in arbitrarily rough terrains. Thus, the TTCA-based driving velocity projection method is proposed in this study to improve the maneuverability of PASTRo in arbitrarily rough terrains. The RecurDyn-Simulink co-simulator is used to examine the improvement of PASTRo compared to a tracked mobile robot non-suspension version. The results indicate that PASTRo has a 33.3% lower RMS(Root Mean Square) distance error, 56.3% lower RMS directional error, and 43.2% lower RMS offset error than the four-track skid-steer mobile robot (SSMR), even with planar SSMR kinematics. To improve the maneuverability of PASTRo without any information on the rough terrain, the TTCA is calculated from the suspension kinematics, and the TTCA obtained is used for both TTCA-based driving velocity projection methods. The results show that PASTRo, with the TTCA-based driving velocity projection method, has a 39.2% lower RMS distance error, 57.9% lower RMS directional error, and 51.9% lower RMS offset error than the four-track SSMR.
- 10 February 2023
- Published by MDPI AG in Mathematics
- Vol. 11 (4) , 905
Open AccessExplicit Identification of Pointwise Terrain Gradients for Speed Compensation of Four Driving Tracks in Passively Articulated Tracked Mobile Robot
Tracked mobile robots can overcome the limitations of wheeled and legged robots in environments, such as construction and mining, but there are still significant challenges to be addressed in terms of trajectory tracking. This study proposes a kinematic strategy to improve the trajectory-tracking performance of a PASTRo (Passively Articulated Suspension based Track-typed mobile robot), which comprises four tracks, two rockers, a differential gear, and a main body. Due to the difficulties in explicitly identifying track-terrain contact angles, suspension kinematics is used to identify track-terrain contact angles (TTCA) in arbitrarily rough terrains. Thus, the TTCA-based driving velocity projection method is proposed in this study to improve the maneuverability of PASTRo in arbitrarily rough terrains. The RecurDyn-Simulink co-simulator is used to examine the improvement of PASTRo compared to a tracked mobile robot non-suspension version. The results indicate that PASTRo has a 33.3% lower RMS(Root Mean Square) distance error, 56.3% lower RMS directional error, and 43.2% lower RMS offset error than the four-track skid-steer mobile robot (SSMR), even with planar SSMR kinematics. To improve the maneuverability of PASTRo without any information on the rough terrain, the TTCA is calculated from the suspension kinematics, and the TTCA obtained is used for both TTCA-based driving velocity projection methods. The results show that PASTRo, with the TTCA-based driving velocity projection method, has a 39.2% lower RMS distance error, 57.9% lower RMS directional error, and 51.9% lower RMS offset error than the four-track SSMR.
- Open Access
- 14 December 2022
- Published by MDPI AG in Mathematics
- Vol. 10 (24) , 4753
Wearable Inertial Sensor-Based Hand-Guiding Gestures Recognition Method Robust to Significant Changes in the Body-Alignment of Subject
The accuracy of the wearable inertia-measurement-unit (IMU)-sensor-based gesture recognition may be significantly affected by undesired changes in the body-fixed frame and the sensor-fixed frame according to the change in the subject and the sensor attachment. In this study, we proposed a novel wearable IMU-sensor-based hand-guiding gesture recognition method robust to significant changes in the subject’s body alignment based on the floating body-fixed frame method and the bi-directional long short-term memory (bi-LSTM). Through comparative experimental studies with the other two methods, it was confirmed that aligning the sensor-fixed frame with the reference frame of the human body and updating the reference frame according to the change in the subject’s body-heading direction helped improve the generalization performance of the gesture recognition model. As a result, the proposed floating body-fixed frame method showed a 91.7% test accuracy, confirming that it was appropriate for gesture recognition under significant changes in the subject’s body alignment during gestures.
- 14 December 2022
- Published by MDPI AG in Mathematics
- Vol. 10 (24) , 4753
Open AccessWearable Inertial Sensor-Based Hand-Guiding Gestures Recognition Method Robust to Significant Changes in the Body-Alignment of Subject
The accuracy of the wearable inertia-measurement-unit (IMU)-sensor-based gesture recognition may be significantly affected by undesired changes in the body-fixed frame and the sensor-fixed frame according to the change in the subject and the sensor attachment. In this study, we proposed a novel wearable IMU-sensor-based hand-guiding gesture recognition method robust to significant changes in the subject’s body alignment based on the floating body-fixed frame method and the bi-directional long short-term memory (bi-LSTM). Through comparative experimental studies with the other two methods, it was confirmed that aligning the sensor-fixed frame with the reference frame of the human body and updating the reference frame according to the change in the subject’s body-heading direction helped improve the generalization performance of the gesture recognition model. As a result, the proposed floating body-fixed frame method showed a 91.7% test accuracy, confirming that it was appropriate for gesture recognition under significant changes in the subject’s body alignment during gestures.
- 10 August 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (4) , 8671-8677
A New Data Augmentation Method for Time Series Wearable Sensor Data Using a Learning Mode Switching-Based DCGAN
This paper describes a new image augmentation method based on a DCGAN considering mode switching in terms of transient learning accuracy threshold on the problem of human gesture recognition through imaging of wearable sensor time-series data and a deep CNN structure. Because the discriminator in GANs learns faster than the generator, it is known that mode collapse occurs, in which only image modes biased to a specific image type are augmented among various image forms. In this study, to solve the mode collapse caused by the learning difficulty mismatch between networks, we add a learning mode switching layer between the generator and discriminator and receive feedback from both networks transient learning accuracies to switch the learning mode if predefined thresholds are exceeded. We confirm that the proposed approach balanced the learning rate between the generator and discriminator networks, resolved the mode collapse problem, and increased the test accuracy of a deep CNN trained with an augmented image set by approximately 20.35% compared to a conventional DCGAN. In addition, it showed better accuracy on a performance comparison with other improved DCGAN methods.
- 10 August 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (4) , 8671-8677
A New Data Augmentation Method for Time Series Wearable Sensor Data Using a Learning Mode Switching-Based DCGAN
This paper describes a new image augmentation method based on a DCGAN considering mode switching in terms of transient learning accuracy threshold on the problem of human gesture recognition through imaging of wearable sensor time-series data and a deep CNN structure. Because the discriminator in GANs learns faster than the generator, it is known that mode collapse occurs, in which only image modes biased to a specific image type are augmented among various image forms. In this study, to solve the mode collapse caused by the learning difficulty mismatch between networks, we add a learning mode switching layer between the generator and discriminator and receive feedback from both networks transient learning accuracies to switch the learning mode if predefined thresholds are exceeded. We confirm that the proposed approach balanced the learning rate between the generator and discriminator networks, resolved the mode collapse problem, and increased the test accuracy of a deep CNN trained with an augmented image set by approximately 20.35% compared to a conventional DCGAN. In addition, it showed better accuracy on a performance comparison with other improved DCGAN methods.
- Open Access
Fast Wearable Sensor–Based Foot–Ground Contact Phase Classification Using a Convolutional Neural Network with Sliding-Window Label Overlapping
Classification of foot–ground contact phases, as well as the swing phase is essential in biomechanics domains where lower-limb motion analysis is required; this analysis is used for lower-limb rehabilitation, walking gait analysis and improvement, and exoskeleton motion capture. In this study, sliding-window label overlapping of time-series wearable motion data in training dataset acquisition is proposed to accurately detect foot–ground contact phases, which are composed of 3 sub-phases as well as the swing phase, at a frequency of 100 Hz with a convolutional neural network (CNN) architecture. We not only succeeded in developing a real-time CNN model for learning and obtaining a test accuracy of 99.8% or higher, but also confirmed that its validation accuracy was close to 85%.
Open AccessFast Wearable Sensor–Based Foot–Ground Contact Phase Classification Using a Convolutional Neural Network with Sliding-Window Label Overlapping
Classification of foot–ground contact phases, as well as the swing phase is essential in biomechanics domains where lower-limb motion analysis is required; this analysis is used for lower-limb rehabilitation, walking gait analysis and improvement, and exoskeleton motion capture. In this study, sliding-window label overlapping of time-series wearable motion data in training dataset acquisition is proposed to accurately detect foot–ground contact phases, which are composed of 3 sub-phases as well as the swing phase, at a frequency of 100 Hz with a convolutional neural network (CNN) architecture. We not only succeeded in developing a real-time CNN model for learning and obtaining a test accuracy of 99.8% or higher, but also confirmed that its validation accuracy was close to 85%.