Keshi He
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Keshi He has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Bryan J. Ranger
Department of Engineering, Boston College, Chestnut Hill, MA, USA
2 shared publications - Donglai Wei
Department of Computer Science, Boston College, Chestnut Hill, MA, USA
2 shared publications - Honghai Liu
Harbin Institute of Technology (Shenzhen)
2 shared publications - Dalin Zhou
University of Portsmouth
1 shared publication - Emily M. Nagel
Department of Pediatrics, University of Minnesota Medical School, Minneapolis, MN, USA
1 shared publication - Hayoung Cho
Department of Engineering, Boston College, Chestnut Hill, MA, USA
1 shared publication
Latest Publications
- 19 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 26 (5) , 7178-7185
Online Performance Evaluation and Robustness Analysis of Ultrasound Images of the Forearm Used as Human–Machine Interface
Ultrasound imaging (US imaging) is increasingly used to build a human machine interface (HMI). However, few studies investigate online performance of such HMI and its robustness against fatigue. This study evaluates the feasibility of using B-mode ultrasound as a real-time HMI and explore effects of muscle fatigue and force level variation on the classification performance of the ultrasound-based HMI. To achieve this, an online finger motion experiment and a muscle fatigue experiment were conducted. The real-time performance was evaluated with motion selection time (ST), completion time (CT), completion rate (CR), and real-time accuracy (RA). Results demonstrate that the achieved ST and CT meet requirements for online prosthetic control. Then, after an algorithm incorporating bag-of-features (BoF) model and speeded-up robust features (SURF) was introduced as well as gray-level co-occurrence matrix (GLCM), robustness against muscle fatigue and force level variation was evaluated with features derived from edge points based keypoints (BoF-EP) and dense grid based keypoints (BoF-DG). Results reveal that GLCM and BoF-EP achieve high classification accuracy (CA), indicating that they have a good robustness against muscle fatigue and force level variation. Additionally, muscle fatigue has a minor influence on the HMI’s classification performance, whereas force level variation affects it significantly.
- 19 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 26 (5) , 7178-7185
Online Performance Evaluation and Robustness Analysis of Ultrasound Images of the Forearm Used as Human–Machine Interface
Ultrasound imaging (US imaging) is increasingly used to build a human machine interface (HMI). However, few studies investigate online performance of such HMI and its robustness against fatigue. This study evaluates the feasibility of using B-mode ultrasound as a real-time HMI and explore effects of muscle fatigue and force level variation on the classification performance of the ultrasound-based HMI. To achieve this, an online finger motion experiment and a muscle fatigue experiment were conducted. The real-time performance was evaluated with motion selection time (ST), completion time (CT), completion rate (CR), and real-time accuracy (RA). Results demonstrate that the achieved ST and CT meet requirements for online prosthetic control. Then, after an algorithm incorporating bag-of-features (BoF) model and speeded-up robust features (SURF) was introduced as well as gray-level co-occurrence matrix (GLCM), robustness against muscle fatigue and force level variation was evaluated with features derived from edge points based keypoints (BoF-EP) and dense grid based keypoints (BoF-DG). Results reveal that GLCM and BoF-EP achieve high classification accuracy (CA), indicating that they have a good robustness against muscle fatigue and force level variation. Additionally, muscle fatigue has a minor influence on the HMI’s classification performance, whereas force level variation affects it significantly.
- Open Access
- 3 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 206355-206366
Developing a Deep Learning Approach for Automated Body Composition Prediction in Newborns Using Ultrasound Images
Objective: Measurements of human body composition such as fat mass (FM) and fat-free mass (FFM) are critical for studying malnutrition and the effects of nutritional interventions. This study introduces research toward a novel ultrasound scanning protocol combined with a deep learning analysis pipeline for predicting body composition. Methods: We analyzed a clinical dataset of 65 premature infants, consisting of ultrasound images from three anatomical locations (biceps, abdomen, and quadriceps), and ground truth FM and FFM from air displacement plethysmography (ADP). Our investigation focused on determining: (1) the optimal data processing methods for this application; (2) suitable baseline deep learning models for prediction to guide our learning strategy; and (3) the anatomical locations and image regions most predictive of FM and FFM. Results: We demonstrate that: (1) pre-processing techniques such as denoising, median filtering, and data augmentation enhance performance; (2) by employing a modified EfficientNet-B1 architecture, we achieve fully automatic body composition predictions from ultrasound images; (3) images obtained from combinations of biceps and quadriceps, as well as biceps, quadriceps, and abdomen scanning locations, resulted in mean absolute percent error (MAPE) values of 26.1% and 25.32%, respectively. Finally, sensitivity analysis shows that FM and FFM prediction are influenced by different body parts, as well as adipose and muscle tissue thickness. Conclusion: This study represents the first demonstration of deep learning for automated human body composition prediction from ultrasound images and lays a critical foundation for a novel ultrasound scanning and interpretation protocol to assess malnutrition.
- 3 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 206355-206366
Open AccessDeveloping a Deep Learning Approach for Automated Body Composition Prediction in Newborns Using Ultrasound Images
Objective: Measurements of human body composition such as fat mass (FM) and fat-free mass (FFM) are critical for studying malnutrition and the effects of nutritional interventions. This study introduces research toward a novel ultrasound scanning protocol combined with a deep learning analysis pipeline for predicting body composition. Methods: We analyzed a clinical dataset of 65 premature infants, consisting of ultrasound images from three anatomical locations (biceps, abdomen, and quadriceps), and ground truth FM and FFM from air displacement plethysmography (ADP). Our investigation focused on determining: (1) the optimal data processing methods for this application; (2) suitable baseline deep learning models for prediction to guide our learning strategy; and (3) the anatomical locations and image regions most predictive of FM and FFM. Results: We demonstrate that: (1) pre-processing techniques such as denoising, median filtering, and data augmentation enhance performance; (2) by employing a modified EfficientNet-B1 architecture, we achieve fully automatic body composition predictions from ultrasound images; (3) images obtained from combinations of biceps and quadriceps, as well as biceps, quadriceps, and abdomen scanning locations, resulted in mean absolute percent error (MAPE) values of 26.1% and 25.32%, respectively. Finally, sensitivity analysis shows that FM and FFM prediction are influenced by different body parts, as well as adipose and muscle tissue thickness. Conclusion: This study represents the first demonstration of deep learning for automated human body composition prediction from ultrasound images and lays a critical foundation for a novel ultrasound scanning and interpretation protocol to assess malnutrition.
- 26 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 30 (6) , 5296-5310
Deep Learning for Clinical Ultrasound Imaging: From Supervised Approaches to Foundation Models
Deep learning models have traditionally been developed and trained to perform specific tasks, which limits their generalizability across different domains. Recently, the field has experienced a paradigm shift toward foundation models, wherein AI systems are pre-trained on extensive datasets to support a broad range of tasks across multiple domains. This shift is particularly promising for ultrasound imaging, a relatively low-cost and versatile modality in clinical practice. However, their adoption remains limited due to challenges such as scarce labeled data, domain-specific variability, and the need for user expertise. This review first outlines the workflow and design principles of deep learning based ultrasound clinical applications, with emphasis on advancements in supervised and self-supervised learning, discriminative models, generative AI models, foundation models, and other miscellaneous models. Recent literature on ultrasound image analysis is then reviewed in detail. Furthermore, deep learning-based ultrasound clinical applications are summarized, and the potential of foundation models to enhance diagnostic performance and expand the scope of ultrasound in clinical settings is highlighted. Finally, key challenges are discussed, and future directions for deep learning in clinical ultrasound are proposed.
- 26 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 30 (6) , 5296-5310
Deep Learning for Clinical Ultrasound Imaging: From Supervised Approaches to Foundation Models
Deep learning models have traditionally been developed and trained to perform specific tasks, which limits their generalizability across different domains. Recently, the field has experienced a paradigm shift toward foundation models, wherein AI systems are pre-trained on extensive datasets to support a broad range of tasks across multiple domains. This shift is particularly promising for ultrasound imaging, a relatively low-cost and versatile modality in clinical practice. However, their adoption remains limited due to challenges such as scarce labeled data, domain-specific variability, and the need for user expertise. This review first outlines the workflow and design principles of deep learning based ultrasound clinical applications, with emphasis on advancements in supervised and self-supervised learning, discriminative models, generative AI models, foundation models, and other miscellaneous models. Recent literature on ultrasound image analysis is then reviewed in detail. Furthermore, deep learning-based ultrasound clinical applications are summarized, and the potential of foundation models to enhance diagnostic performance and expand the scope of ultrasound in clinical settings is highlighted. Finally, key challenges are discussed, and future directions for deep learning in clinical ultrasound are proposed.
- 2 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 74 (00189456) , 1-21
A Survey of Vision-Based Tactile Sensors: Hardware, Algorithm, Application, and Future Direction
Since vision-based tactile sensors (VBTSs) are firstly proposed and developed, they have attracted more and more attentions in recent years. This survey aims to provide a comprehensive review of VBTSs, with emphasis on their algorithms and applications. First, I briefly introduce the concept, principle, and prototype of VBTSs. Then, I illustrate the deployment of VBTSs into robotic systems and discuss the state-of-the-art progress of VBTSs-based robotic grasping and manipulation, then outline the main functions of VBTSs and provide a systematic survey on their algorithms, with a focus on deep learning and computer vision. Furthermore, I summarize the applications of VBTSs in practical scenarios. Finally, I discuss key research challenges and envision future directions for VBTSs.
- 2 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 74 (00189456) , 1-21
A Survey of Vision-Based Tactile Sensors: Hardware, Algorithm, Application, and Future Direction
Since vision-based tactile sensors (VBTSs) are firstly proposed and developed, they have attracted more and more attentions in recent years. This survey aims to provide a comprehensive review of VBTSs, with emphasis on their algorithms and applications. First, I briefly introduce the concept, principle, and prototype of VBTSs. Then, I illustrate the deployment of VBTSs into robotic systems and discuss the state-of-the-art progress of VBTSs-based robotic grasping and manipulation, then outline the main functions of VBTSs and provide a systematic survey on their algorithms, with a focus on deep learning and computer vision. Furthermore, I summarize the applications of VBTSs in practical scenarios. Finally, I discuss key research challenges and envision future directions for VBTSs.
- 25 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Robotics and Bionics
- Vol. 7 (1) , 200-212
Ultrasound-Based Human Machine Interfaces for Hand Gesture Recognition: A Scoping Review and Future Direction
Since ultrasound signal is firstly used to build a human machine interface (HMI) for prosthetic control in 2006, ultrasound-based HMIs have received the great attention in the past 18 years. In this paper, I provide a comprehensive overview of every aspect of an ultrasound-based HMI for hand gesture recognition (HGR). Firstly, I introduce the principle of ultrasound-based HGR and then outline a workflow for an ultrasound-based HMI for HGR and detail each step involved in the workflow, followed by an introduction of performance evaluation and robustness of this type of HMI. Then, I review the research progress of ultrasound-based HMIs for HGR. After that, I introduce the state-of-the-art wearable ultrasound systems for HMIs. Furthermore, I summarize the miscellaneous application of ultrasound-based HMIs for HGR. Finally, I discuss the main research challenges and further envision future research directions for ultrasound-based HMIs for HGR.
- 25 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Robotics and Bionics
- Vol. 7 (1) , 200-212
Ultrasound-Based Human Machine Interfaces for Hand Gesture Recognition: A Scoping Review and Future Direction
Since ultrasound signal is firstly used to build a human machine interface (HMI) for prosthetic control in 2006, ultrasound-based HMIs have received the great attention in the past 18 years. In this paper, I provide a comprehensive overview of every aspect of an ultrasound-based HMI for hand gesture recognition (HGR). Firstly, I introduce the principle of ultrasound-based HGR and then outline a workflow for an ultrasound-based HMI for HGR and detail each step involved in the workflow, followed by an introduction of performance evaluation and robustness of this type of HMI. Then, I review the research progress of ultrasound-based HMIs for HGR. After that, I introduce the state-of-the-art wearable ultrasound systems for HMIs. Furthermore, I summarize the miscellaneous application of ultrasound-based HMIs for HGR. Finally, I discuss the main research challenges and further envision future research directions for ultrasound-based HMIs for HGR.
- 9 December 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 17 (1) , 587-595
Screening Early Children With Autism Spectrum Disorder via Response-to-Name Protocol
Incidence of children with autism spectrum disorder (ASD) has increased with an average rate of 1% worldwide. Clinical ASD screening, especially for children screening is a laborious and skilled task, there is however no objective and effective method automating ASD children screening. Analysing children ASD characteristics in predefined motion behaviour protocols is attempted to provide automatic solutions to children ASD screening. A novel protocol, Response to Name (RTN), is proposed in this paper for ASD clinical validation and diagnosis. The RTN method is jointly designed with clinical partners, novel gaze estimation is developed for validating ASD characteristic behaviour. Seventeen subjects including 10 adults and 7 children (5 ASD subjects and 2 healthy subjects) participated the experiment. The experiment results showed that the proposed RTN system achieves an average classification score of 92.7% fully demonstrating that the principle of motion protocol based ASD screening has the potential to have early ASD screening automated.
- 9 December 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 17 (1) , 587-595
Screening Early Children With Autism Spectrum Disorder via Response-to-Name Protocol
Incidence of children with autism spectrum disorder (ASD) has increased with an average rate of 1% worldwide. Clinical ASD screening, especially for children screening is a laborious and skilled task, there is however no objective and effective method automating ASD children screening. Analysing children ASD characteristics in predefined motion behaviour protocols is attempted to provide automatic solutions to children ASD screening. A novel protocol, Response to Name (RTN), is proposed in this paper for ASD clinical validation and diagnosis. The RTN method is jointly designed with clinical partners, novel gaze estimation is developed for validating ASD characteristic behaviour. Seventeen subjects including 10 adults and 7 children (5 ASD subjects and 2 healthy subjects) participated the experiment. The experiment results showed that the proposed RTN system achieves an average classification score of 92.7% fully demonstrating that the principle of motion protocol based ASD screening has the potential to have early ASD screening automated.
- 25 October 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 22 (5) , 1395-1405
Ultrasound-Based Sensing Models for Finger Motion Classification
Motions of the fingers are complex since hand grasping and manipulation are conducted by spatial and temporal coordination of forearm muscles and tendons. The dominant methods based on surface electromyography (sEMG) could not offer satisfactory solutions for finger motion classification due to its inherent nature of measuring the electrical activity of motor units at the skin's surface. In order to recognize morphological changes of forearm muscles for accurate hand motion prediction, ultrasound imaging is employed to investigate the feasibility of detecting mechanical deformation of deep muscle compartments in potential clinical applications. In this study, finger motion classification has been represented as subproblems: recognizing the discrete finger motions and predicting the continuous finger angles. Predefined 14 finger motions are presented in both sEMG signals and ultrasound images and captured simultaneously. Linear discriminant analysis classifier shows the ultrasound has better average accuracy (95.88%) than the sEMG (90.14%). On the other hand, the study of predicting the metacarpophalangeal (MCP) joint angle of each finger in nonperiod movements also confirms that classification method based on ultrasound achieves better results (average correlation 0.89 ± 0.07 and NRMSE 0.15 ± 0.05) than sEMG (0.81 ± 0.09 and 0.19 ± 0.05). The research outcomes evidently demonstrate that the ultrasound can be a feasible solution for muscle-driven machine interface, such as accurate finger motion control of prostheses and wearable robotic devices.
- 25 October 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 22 (5) , 1395-1405
Ultrasound-Based Sensing Models for Finger Motion Classification
Motions of the fingers are complex since hand grasping and manipulation are conducted by spatial and temporal coordination of forearm muscles and tendons. The dominant methods based on surface electromyography (sEMG) could not offer satisfactory solutions for finger motion classification due to its inherent nature of measuring the electrical activity of motor units at the skin's surface. In order to recognize morphological changes of forearm muscles for accurate hand motion prediction, ultrasound imaging is employed to investigate the feasibility of detecting mechanical deformation of deep muscle compartments in potential clinical applications. In this study, finger motion classification has been represented as subproblems: recognizing the discrete finger motions and predicting the continuous finger angles. Predefined 14 finger motions are presented in both sEMG signals and ultrasound images and captured simultaneously. Linear discriminant analysis classifier shows the ultrasound has better average accuracy (95.88%) than the sEMG (90.14%). On the other hand, the study of predicting the metacarpophalangeal (MCP) joint angle of each finger in nonperiod movements also confirms that classification method based on ultrasound achieves better results (average correlation 0.89 ± 0.07 and NRMSE 0.15 ± 0.05) than sEMG (0.81 ± 0.09 and 0.19 ± 0.05). The research outcomes evidently demonstrate that the ultrasound can be a feasible solution for muscle-driven machine interface, such as accurate finger motion control of prostheses and wearable robotic devices.