Xingyu Li

Personalization in the myoelectric control of robotic exoskeletons is crucial to ensuring accurate interpretation and adaptation to the unique muscle activity patterns and movement intentions of each user. This approach minimizes the risk of incorrect or excessive force application, significantly reducing the likelihood of user discomfort or injury during operation. This study introduces a model-agnostic meta-learning approach for personalizing a soft upper-limb exoskeleton in industrial settings. The framework incorporates an attention-based CNN-LSTM model that predicts future angular positions of the robot using EMG and IMU signals. The MAML framework demonstrates significant adaptability and personalization, efficiently predicting future angular positions with minimal data, approximately 20-25 seconds per task. This approach effectively reduces the necessity for extensive retraining with new users or in new environments by 50%, showcasing real-time task adaptation capabilities. Our findings confirmed a reduced human effort of nearly 13% in load-bearing tasks. Also, the results show that the exerted torque from the exoskeleton was 24% higher while maintaining higher accuracy. A comparison with other deep learning models further emphasizes the enhanced adaptability and accuracy offered by the meta-learning approach.

The softmax function is a fundamental component in deep learning. This study delves into the often-overlooked parameter within the softmax function, known as "temperature," providing novel insights into the practical and theoretical aspects of temperature scaling for image classification. Our empirical studies, adopting convolutional neural networks and transformers on multiple benchmark datasets, reveal that moderate temperatures generally introduce better overall performance. Through extensive experiments and rigorous theoretical analysis, we explore the role of temperature scaling in model training and unveil that temperature not only influences learning step size but also shapes the model's optimization direction. Moreover, for the first time, we discover a surprising benefit of elevated temperatures: enhanced model robustness against common corruption, natural perturbation, and non-targeted adversarial attacks like Projected Gradient Descent. We extend our discoveries to adversarial training, demonstrating that, compared to the standard softmax function with the default temperature value, higher temperatures have the potential to enhance adversarial training. The insights of this work open new avenues for improving model performance and security in deep learning applications.

Bimanual tasks, where the brain must simultaneously control and plan the movements of both hands, such as needle passing and tissue cutting, commonly exist in surgeries, e.g., robot-assisted minimally invasive surgery. In this study, we present a novel approach for quantifying the quality of hands coordination and correspondence in bimanual tasks by utilizing information theory concepts to build a mathematical framework for measuring the collaboration strength between the two hands. The introduced method makes no assumption about the dynamics and couplings within the robotic platform, executive task, or human motor control. We implemented the proposed approach on MEELS and JIGSAWS datasets, corresponding to conventional minimally invasive surgery (MIS) and robot-assisted MIS, respectively. We analyzed the advantages of hands collaboration features in the skills assessment and style recognition of robotic surgery tasks. Furthermore, we demonstrated that incorporating intuitive domain knowledge of bimanual tasks potentially paves the way for other complex applications, including, but not limited to, autonomous surgery with a high level of model explainability and interpretability. Finally, we presented preliminary results to argue that incorporating hands collaboration features in deep learning-based classifiers reduces uncertainty, improves accuracy, and enhances the out-of-distribution robustness of the final model.

Cross-View Geo-Localization (CVGL) estimates the location of a ground image by matching it to a geo-tagged aerial image in a database. Recent works achieve outstanding progress on CVGL benchmarks. However, existing methods still suffer from poor performance in cross-area evaluation, in which the training and testing data are captured from completely distinct areas. We attribute this deficiency to the lack of ability to extract the geometric layout of visual features and models’ overfitting to low-level details. Our preliminary work (Zhang et al. 2022) introduced a Geometric Layout Extractor (GLE) to capture the geometric layout from input features. However, the previous GLE does not fully exploit information in the input feature. In this work, we propose GeoDTR+ with an enhanced GLE module that better models the correlations among visual features. To fully explore the LS techniques from our preliminary work, we further propose Contrastive Hard Samples Generation (CHSG) to facilitate model training. Extensive experiments show that GeoDTR+ achieves state-of-the-art (SOTA) results in cross-area evaluation on CVUSA (Workman et al. 2015), CVACT (Liu and Li, 2019), and VIGOR (Zhu et al. 2021) by a large margin (16.44%, 22.71%, and 13.66% without polar transformation) while keeping the same-area performance comparable to existing SOTA. Moreover, we provide detailed analyses of GeoDTR+.

Recent applications of deep reinforcement learning (DRL) in surgical autonomy have shown promising results in automating various surgical sub-tasks. While most of these studies consider the rigid and soft body dynamics in the surgery such as tissue deformation, only a few have investigated the situation where fluid is present. However, the presence of fluids, particularly blood, is common in surgeries, and interacting with them adds additional challenges to task automation. In this work, we investigate the use of DRL in automating blood suction, a common surgical sub-task where blood is removed from the surgical field. We build a blood suction simulation environment based on position-based fluids (PBF), train an agent with domain-randomized environment parameters through curriculum learning, and obtain a generalizable policy that can be applied to various shapes of tissue and types of liquid. Real-world experiments show that the agent can perform autonomous suction in different tissue models with different amounts and types of liquid, and only one of the 50 trials resulted in more than 3 ml of blood remaining.