Fan Wei
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Fan Wei has a total of 39 co-authors. Here are the top co-authors by number of shared publications:
- Guanghua Xu
School of Mechanical Engineering, Xi’an Jiaotong University
11 shared publications - Qingqiang Wu
School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710049, China
10 shared publications - Jiachen Kuang
School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710049, China
7 shared publications - Chengcheng Han
School of Mechanical Engineering Xi'an Jiaotong University Xi'an China
5 shared publications - Peiyuan Tian
School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710049, China
3 shared publications - Tangfei Tao
School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China
3 shared publications
Latest Publications
- Open Access
Whole-Body 3D Pose Estimation Based on Body Mass Distribution and Center of Gravity Constraints
Estimating the 3D pose of a human body from monocular images is crucial for computer vision applications, but the technique remains challenging due to depth ambiguity and self-occlusion. Traditional methods often suffer from insufficient prior knowledge and weak constraints, resulting in inaccurate 3D keypoint estimation. In this paper, we propose a method for whole-body 3D pose estimation based on a Transformer architecture, integrating body mass distribution and center of gravity constraints. The method maps the pose to the center of gravity position using the anatomical mass ratio of the human body and computes the segment-level center of gravity using the moment synthesis method. A combined loss function is designed to enforce consistency between the predicted keypoints and the center of gravity position, as well as the invariance of limb length. Extensive experiments on the Human 3.6M WholeBody dataset demonstrate that the proposed method achieves state-of-the-art performance, with a whole-body mean joint position error (MPJPE) of 44.49 mm, which is 60.4% lower than the previous Large Simple Baseline method. Notably, it reduces the body part keypoints’ MPJPE from 112.6 to 40.41, showcasing the enhanced robustness and effectiveness to occluded scenes. This study highlights the effectiveness of integrating physical constraints into deep learning frameworks for accurate 3D pose estimation.
Open AccessWhole-Body 3D Pose Estimation Based on Body Mass Distribution and Center of Gravity Constraints
Estimating the 3D pose of a human body from monocular images is crucial for computer vision applications, but the technique remains challenging due to depth ambiguity and self-occlusion. Traditional methods often suffer from insufficient prior knowledge and weak constraints, resulting in inaccurate 3D keypoint estimation. In this paper, we propose a method for whole-body 3D pose estimation based on a Transformer architecture, integrating body mass distribution and center of gravity constraints. The method maps the pose to the center of gravity position using the anatomical mass ratio of the human body and computes the segment-level center of gravity using the moment synthesis method. A combined loss function is designed to enforce consistency between the predicted keypoints and the center of gravity position, as well as the invariance of limb length. Extensive experiments on the Human 3.6M WholeBody dataset demonstrate that the proposed method achieves state-of-the-art performance, with a whole-body mean joint position error (MPJPE) of 44.49 mm, which is 60.4% lower than the previous Large Simple Baseline method. Notably, it reduces the body part keypoints’ MPJPE from 112.6 to 40.41, showcasing the enhanced robustness and effectiveness to occluded scenes. This study highlights the effectiveness of integrating physical constraints into deep learning frameworks for accurate 3D pose estimation.
- Open Access
- 27 August 2024
- Published by MDPI AG in Electronics
- Vol. 13 (17) , 3411
Research on an Indoor Light Environment Comfort Evaluation Index Based on Electroencephalogram and Pupil Signals
With the development of modern technology, many people work for a long time around various artificial light sources and electronic equipment, causing them to feel discomfort in their eyes and even eye diseases. The industry currently lacks an objective quantitative environmental–visual comfort index that combines subjective and objective indicators. For this experiment, objective eye movement and electroencephalogram (EEG) signals were collected in combination with a subjective questionnaire survey and a preference inquiry for comprehensive data mining. Finally, the results on a Likert scale show that high screen brightness can reduce the visual fatigue of subjects under high illuminance and high correlated color temperature (CCT). Pupil data show that, under medium and high ambient illuminance, visual perception sensitivity is more likely to be stimulated, and visual fatigue is more likely to deepen. EEG data show that visual fatigue is related to illuminance and screen brightness. On this basis, this study proposes a new evaluation index, the visual comfort level (0.6404 average at a low screen brightness, 0.4218 average at a medium screen brightness, and 0.5139 average at a high screen brightness), where a higher score for the visual comfort level represents a better visual experience. The visual comfort level provides a useful reference for enhancing the processing of multi-dimensional and biomedical signals and protecting the eyes.
- 27 August 2024
- Published by MDPI AG in Electronics
- Vol. 13 (17) , 3411
Open AccessResearch on an Indoor Light Environment Comfort Evaluation Index Based on Electroencephalogram and Pupil Signals
With the development of modern technology, many people work for a long time around various artificial light sources and electronic equipment, causing them to feel discomfort in their eyes and even eye diseases. The industry currently lacks an objective quantitative environmental–visual comfort index that combines subjective and objective indicators. For this experiment, objective eye movement and electroencephalogram (EEG) signals were collected in combination with a subjective questionnaire survey and a preference inquiry for comprehensive data mining. Finally, the results on a Likert scale show that high screen brightness can reduce the visual fatigue of subjects under high illuminance and high correlated color temperature (CCT). Pupil data show that, under medium and high ambient illuminance, visual perception sensitivity is more likely to be stimulated, and visual fatigue is more likely to deepen. EEG data show that visual fatigue is related to illuminance and screen brightness. On this basis, this study proposes a new evaluation index, the visual comfort level (0.6404 average at a low screen brightness, 0.4218 average at a medium screen brightness, and 0.5139 average at a high screen brightness), where a higher score for the visual comfort level represents a better visual experience. The visual comfort level provides a useful reference for enhancing the processing of multi-dimensional and biomedical signals and protecting the eyes.
- Open Access
- 10 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Systems and Rehabilitation Engineering
- Vol. 31 (15344320) , 1670-1679
A Training-Free Infant Spontaneous Movement Assessment Method for Cerebral Palsy Prediction Based on Videos
objective: Early diagnosis of infant cerebral palsy (CP) is very important for infant health. In this paper, we present a novel training-free method to quantify infant spontaneous movements for predicting CP. Methods: Unlike other classification methods, our method turns the assessment into a clustering task. First, the joints of the infant are extracted by the current pose estimation algorithm, and the skeleton sequence is segmented into multiple clips through a sliding window. Then we cluster the clips and quantify infant CP by the number of cluster classes. Results: The proposed method was tested on two datasets, and achieved state-of-the-arts (SOTAs) on both datasets using the same parameters. What’s more, our method is interpretable with visualized results. Conclusion: The proposed method can quantify abnormal brain development in infants effectively and be used in different datasets without training. Significance: Limited by small samples, we propose a training-free method for quantifying infant spontaneous movements. Unlike other binary classification methods, our work not only enables continuous quantification of infant brain development, but also provides interpretable conclusions by visualizing the results. The proposed spontaneous movement assessment method significantly advances SOTAs in automatically measuring infant health.
- 10 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Systems and Rehabilitation Engineering
- Vol. 31 (15344320) , 1670-1679
Open AccessA Training-Free Infant Spontaneous Movement Assessment Method for Cerebral Palsy Prediction Based on Videos
objective: Early diagnosis of infant cerebral palsy (CP) is very important for infant health. In this paper, we present a novel training-free method to quantify infant spontaneous movements for predicting CP. Methods: Unlike other classification methods, our method turns the assessment into a clustering task. First, the joints of the infant are extracted by the current pose estimation algorithm, and the skeleton sequence is segmented into multiple clips through a sliding window. Then we cluster the clips and quantify infant CP by the number of cluster classes. Results: The proposed method was tested on two datasets, and achieved state-of-the-arts (SOTAs) on both datasets using the same parameters. What’s more, our method is interpretable with visualized results. Conclusion: The proposed method can quantify abnormal brain development in infants effectively and be used in different datasets without training. Significance: Limited by small samples, we propose a training-free method for quantifying infant spontaneous movements. Unlike other binary classification methods, our work not only enables continuous quantification of infant brain development, but also provides interpretable conclusions by visualizing the results. The proposed spontaneous movement assessment method significantly advances SOTAs in automatically measuring infant health.
- 13 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 23 (4) , 4000-4010
Domain Conditioned Joint Adaptation Network for Intelligent Bearing Fault Diagnosis Across Different Positions and Machines
In recent years, unsupervised domain adaptation-based methods have been widely developed for intelligent bearing fault diagnosis across various working conditions. However, a considerably more challenging and practical fault diagnosis scenario in which the source and target domains are respectively collected from bearings across different positions and machines is urgent to be addressed. To solve this issue, an innovative end-to-end domain conditioned joint adaptation network (DCJAN), which is composed of a domain conditioned feature extractor, two classifiers, and a domain discriminator is presented. On the one hand, the domain conditioned feature extraction structure is designed to relax totally-shared network assumptions in feature extraction and learn more domain-specialized features for cross-domain fault diagnosis of bearings. On the other hand, a joint adaptation strategy is implemented for diagnostic knowledge transfer across domains, in which domain-level and class-level adaptations are respectively achieved by domain-adversarial training and bi-classifier adversarial training. Extensive experiments including cross-position fault diagnosis and cross-machine fault diagnosis of bearings indicate the validity and superiority of the proposed method.
- 13 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 23 (4) , 4000-4010
Domain Conditioned Joint Adaptation Network for Intelligent Bearing Fault Diagnosis Across Different Positions and Machines
In recent years, unsupervised domain adaptation-based methods have been widely developed for intelligent bearing fault diagnosis across various working conditions. However, a considerably more challenging and practical fault diagnosis scenario in which the source and target domains are respectively collected from bearings across different positions and machines is urgent to be addressed. To solve this issue, an innovative end-to-end domain conditioned joint adaptation network (DCJAN), which is composed of a domain conditioned feature extractor, two classifiers, and a domain discriminator is presented. On the one hand, the domain conditioned feature extraction structure is designed to relax totally-shared network assumptions in feature extraction and learn more domain-specialized features for cross-domain fault diagnosis of bearings. On the other hand, a joint adaptation strategy is implemented for diagnostic knowledge transfer across domains, in which domain-level and class-level adaptations are respectively achieved by domain-adversarial training and bi-classifier adversarial training. Extensive experiments including cross-position fault diagnosis and cross-machine fault diagnosis of bearings indicate the validity and superiority of the proposed method.
- Open Access
Domain-Adaptive Prototype-Recalibrated Network with Transductive Learning Paradigm for Intelligent Fault Diagnosis under Various Limited Data Conditions
In real industrial scenarios, intelligent fault diagnosis based on data-driven methods has been widely researched in the past decade. However, data scarcity is widespread in fault diagnosis tasks owning to the difficulties in collecting adequate data. As a result, there is an increasing demand for both researchers and engineers for fault identification with scarce data. To address this issue, an innovative domain-adaptive prototype-recalibrated network (DAPRN) based on a transductive learning paradigm and prototype recalibration strategy (PRS) is proposed, which has the potential to promote the generalization ability from the source domain to target domain in a few-shot fault diagnosis. Within this scheme, the DAPRN is composed of a feature extractor, a domain discriminator, and a label predictor. Concretely, the feature extractor is jointly optimized by the minimization of few-shot classification loss and the maximization of domain-discriminative loss. The cosine similarity-based label predictor, which is promoted by the PRS, is exploited to avoid the bias of naïve prototypes in the metric space and recognize the health conditions of machinery in the meta-testing process. The efficacy and advantage of DAPRN are validated by extensive experiments on bearing and gearbox datasets compared with seven popular and well-established few-shot fault diagnosis methods. In practical application, the proposed DAPRN is expected to solve more challenging few-shot fault diagnosis scenarios and facilitate practical fault identification problems in modern manufacturing.
Open AccessDomain-Adaptive Prototype-Recalibrated Network with Transductive Learning Paradigm for Intelligent Fault Diagnosis under Various Limited Data Conditions
In real industrial scenarios, intelligent fault diagnosis based on data-driven methods has been widely researched in the past decade. However, data scarcity is widespread in fault diagnosis tasks owning to the difficulties in collecting adequate data. As a result, there is an increasing demand for both researchers and engineers for fault identification with scarce data. To address this issue, an innovative domain-adaptive prototype-recalibrated network (DAPRN) based on a transductive learning paradigm and prototype recalibration strategy (PRS) is proposed, which has the potential to promote the generalization ability from the source domain to target domain in a few-shot fault diagnosis. Within this scheme, the DAPRN is composed of a feature extractor, a domain discriminator, and a label predictor. Concretely, the feature extractor is jointly optimized by the minimization of few-shot classification loss and the maximization of domain-discriminative loss. The cosine similarity-based label predictor, which is promoted by the PRS, is exploited to avoid the bias of naïve prototypes in the metric space and recognize the health conditions of machinery in the meta-testing process. The efficacy and advantage of DAPRN are validated by extensive experiments on bearing and gearbox datasets compared with seven popular and well-established few-shot fault diagnosis methods. In practical application, the proposed DAPRN is expected to solve more challenging few-shot fault diagnosis scenarios and facilitate practical fault identification problems in modern manufacturing.
- 5 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 71 (00189456) , 1-12
Dual-Weight Consistency-Induced Partial Domain Adaptation Network for Intelligent Fault Diagnosis of Machinery
Domain adaptation (DA)-based methods have been broadly developed for cross-domain fault diagnosis of machinery, in which the target and source label spaces are identical. However, a more challenging and practical diagnostic scenario in which the source label space subsumes target one is remaining to be solved. To address this issue, an innovative two-stage dual-weight consistency-induced partial DA (DCPDA) network, consisting of two feature extractors, a classifier, a domain discriminator, a dual-weight consistency-induced reweighting (DCR) module, and a Wasserstein distance-based-DA (WDA) module, is proposed. In the pretrained stage, a well-trained model on the source domain is acquired. In the fine-tuned stage, the DCR, which conducts a dual-weight consistency-induced weighting strategy, obtains a bi-level compound weight from both class- and sample-level weights. Embedded with the bi-level weight, the WDA is encouraged to transfer the source knowledge with mutual agreement from double-level weights across domains. Accordingly, the DCPDA is capable of alleviating negative transfer dramatically and mapping the source diagnostic knowledge into the target domain selectively. Extensive experiments indicate the validity and advantage of the proposed DCPDA.
- 5 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 71 (00189456) , 1-12
Dual-Weight Consistency-Induced Partial Domain Adaptation Network for Intelligent Fault Diagnosis of Machinery
Domain adaptation (DA)-based methods have been broadly developed for cross-domain fault diagnosis of machinery, in which the target and source label spaces are identical. However, a more challenging and practical diagnostic scenario in which the source label space subsumes target one is remaining to be solved. To address this issue, an innovative two-stage dual-weight consistency-induced partial DA (DCPDA) network, consisting of two feature extractors, a classifier, a domain discriminator, a dual-weight consistency-induced reweighting (DCR) module, and a Wasserstein distance-based-DA (WDA) module, is proposed. In the pretrained stage, a well-trained model on the source domain is acquired. In the fine-tuned stage, the DCR, which conducts a dual-weight consistency-induced weighting strategy, obtains a bi-level compound weight from both class- and sample-level weights. Embedded with the bi-level weight, the WDA is encouraged to transfer the source knowledge with mutual agreement from double-level weights across domains. Accordingly, the DCPDA is capable of alleviating negative transfer dramatically and mapping the source diagnostic knowledge into the target domain selectively. Extensive experiments indicate the validity and advantage of the proposed DCPDA.
- 6 June 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 71 (00189456) , 1-11
Supine Infant Pose Estimation via Single Depth Image
Objective: The infant pose estimation is very important for the evaluation of infants’ brain development and motor quality. Here, we propose a new method to quickly build a large-scale depth image pose dataset and, on this basis, establish the Xi’an Jiaotong University Infant Depth Pose (XJTU-IDP) dataset, and then, we also propose a new method based on joint feature coding (JFC) to estimate the infants’ posture from depth image. Methods: First, we use Kinect V2 to collect depth and color images synchronously and use the proposed framework to build a large dataset. Then, we encode the offset vector in the confidence region of each joint of the infant and export the joint positioning and connection encoding through our convolutional neural network (CNN) model. The proposed method is tested from two datasets (including 16 and 27 clinical actual infants, respectively). Results: When applied to pose estimation, the median root mean square distance (RMSD) calculated among all limbs is 4.12 pixels, which is much lower than the baseline (9.06 pixels) on babyPose dataset and RMSD in XJTU-IDP dataset is 3.93 pixels, which achieves the best in comparison with the state-of-the-arts. Conclusion: The proposed framework of the building dataset in our work is effective, and the JFC can enrich the network information. Significance: Our work not only provides a larger infant pose dataset to make up for the deficiencies of babyPose but also proposes a new framework for quickly establishing high-quality dataset. The proposed pose estimation method significantly improves the state-of-the-art in automatic measurement of infants’ health status.
- 6 June 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 71 (00189456) , 1-11
Supine Infant Pose Estimation via Single Depth Image
Objective: The infant pose estimation is very important for the evaluation of infants’ brain development and motor quality. Here, we propose a new method to quickly build a large-scale depth image pose dataset and, on this basis, establish the Xi’an Jiaotong University Infant Depth Pose (XJTU-IDP) dataset, and then, we also propose a new method based on joint feature coding (JFC) to estimate the infants’ posture from depth image. Methods: First, we use Kinect V2 to collect depth and color images synchronously and use the proposed framework to build a large dataset. Then, we encode the offset vector in the confidence region of each joint of the infant and export the joint positioning and connection encoding through our convolutional neural network (CNN) model. The proposed method is tested from two datasets (including 16 and 27 clinical actual infants, respectively). Results: When applied to pose estimation, the median root mean square distance (RMSD) calculated among all limbs is 4.12 pixels, which is much lower than the baseline (9.06 pixels) on babyPose dataset and RMSD in XJTU-IDP dataset is 3.93 pixels, which achieves the best in comparison with the state-of-the-arts. Conclusion: The proposed framework of the building dataset in our work is effective, and the JFC can enrich the network information. Significance: Our work not only provides a larger infant pose dataset to make up for the deficiencies of babyPose but also proposes a new framework for quickly establishing high-quality dataset. The proposed pose estimation method significantly improves the state-of-the-art in automatic measurement of infants’ health status.
- Open Access
Effects of Paradigm Color and Screen Brightness on Visual Fatigue in Light Environment of Night Based on Eye Tracker and EEG Acquisition Equipment
Nowadays, more people tend to go to bed late and spend their sleep time with various electronic devices. At the same time, the BCI (brain–computer interface) rehabilitation equipment uses a visual display, thus it is necessary to evaluate the problem of visual fatigue to avoid the impact on the training effect. Therefore, it is very important to understand the impact of using electronic devices in a dark environment at night on human visual fatigue. This paper uses Matlab to write different color paradigm stimulations, uses a 4K display with an adjustable screen brightness to jointly design the experiment, uses eye tracker and g.tec Electroencephalogram (EEG) equipment to collect the signal, and then carries out data processing and analysis, finally obtaining the influence of the combination of different colors and different screen brightness on human visual fatigue in a dark environment. In this study, subjects were asked to evaluate their subjective (Likert scale) perception, and objective signals (pupil diameter, θ + α frequency band data) were collected in a dark environment (<3 lx). The Likert scale showed that a low screen brightness in the dark environment could reduce the visual fatigue of the subjects, and participants preferred blue to red. The pupil data revealed that visual perception sensitivity was more vulnerable to stimulation at a medium and high screen brightness, which is easier to deepen visual fatigue. EEG frequency band data concluded that there was no significant difference between paradigm colors and screen brightness on visual fatigue. On this basis, this paper puts forward a new index—the visual anti-fatigue index, which provides a valuable reference for the optimization of the indoor living environment, the improvement of satisfaction with the use of electronic equipment and BCI rehabilitation equipment, and the protection of human eyes.
Open AccessEffects of Paradigm Color and Screen Brightness on Visual Fatigue in Light Environment of Night Based on Eye Tracker and EEG Acquisition Equipment
Nowadays, more people tend to go to bed late and spend their sleep time with various electronic devices. At the same time, the BCI (brain–computer interface) rehabilitation equipment uses a visual display, thus it is necessary to evaluate the problem of visual fatigue to avoid the impact on the training effect. Therefore, it is very important to understand the impact of using electronic devices in a dark environment at night on human visual fatigue. This paper uses Matlab to write different color paradigm stimulations, uses a 4K display with an adjustable screen brightness to jointly design the experiment, uses eye tracker and g.tec Electroencephalogram (EEG) equipment to collect the signal, and then carries out data processing and analysis, finally obtaining the influence of the combination of different colors and different screen brightness on human visual fatigue in a dark environment. In this study, subjects were asked to evaluate their subjective (Likert scale) perception, and objective signals (pupil diameter, θ + α frequency band data) were collected in a dark environment (<3 lx). The Likert scale showed that a low screen brightness in the dark environment could reduce the visual fatigue of the subjects, and participants preferred blue to red. The pupil data revealed that visual perception sensitivity was more vulnerable to stimulation at a medium and high screen brightness, which is easier to deepen visual fatigue. EEG frequency band data concluded that there was no significant difference between paradigm colors and screen brightness on visual fatigue. On this basis, this paper puts forward a new index—the visual anti-fatigue index, which provides a valuable reference for the optimization of the indoor living environment, the improvement of satisfaction with the use of electronic equipment and BCI rehabilitation equipment, and the protection of human eyes.
- 11 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 22 (10) , 9733-9742
Azure Kinect Calibration and Parameter Recommendation in Different Scenarios
The series of Kinect, which has the third-generation device now, is a low-cost scene information collection sensor to capture the color information and depth information from the scene at the same time. Due to the hardware structure of the sensor, the color image and the depth image shot by Kinect is not homologous. In this article, we compared the hardware parameters of different Kinect three generations of products, and we have calibrated the RGB camera and IR (infrared) camera of Azure Kinect, so as to the internal parameters and external parameters between the RGB camera and IR camera, and align the color image and depth image taken with Azure Kinect. At the same time, we evaluated the shooting accuracy and stability of Azure Kinect’s depth camera, and conducted experiments on the relationship between shooting error and measurement distance on three generations of products, also the parameter optimization suggestions are provided. In addition, according to different usage scenarios, we recommend the shooting parameters and the image effective range of the sensor simultaneously.
- 11 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 22 (10) , 9733-9742
Azure Kinect Calibration and Parameter Recommendation in Different Scenarios
The series of Kinect, which has the third-generation device now, is a low-cost scene information collection sensor to capture the color information and depth information from the scene at the same time. Due to the hardware structure of the sensor, the color image and the depth image shot by Kinect is not homologous. In this article, we compared the hardware parameters of different Kinect three generations of products, and we have calibrated the RGB camera and IR (infrared) camera of Azure Kinect, so as to the internal parameters and external parameters between the RGB camera and IR camera, and align the color image and depth image taken with Azure Kinect. At the same time, we evaluated the shooting accuracy and stability of Azure Kinect’s depth camera, and conducted experiments on the relationship between shooting error and measurement distance on three generations of products, also the parameter optimization suggestions are provided. In addition, according to different usage scenarios, we recommend the shooting parameters and the image effective range of the sensor simultaneously.
- 8 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 70 (00189456) , 1-11
Automatically Measure the Quality of Infants’ Spontaneous Movement via Videos to Predict the Risk of Cerebral Palsy
Early brain development assessment of infants is very important for infant growth and development. Timely diagnosis of cerebral palsy (CP) can reduce its negative effects. In this article, a novel automatic measurement method was proposed for the early prediction of infant CP with videos toward Prechtl’s general movement assessment (GMA). First, we adopted state-of-the-art human pose estimation method to obtain the 2-D or 3-D posture of the lying infant. After a series of interpolation and filtering, continuous infant’s limb movement curves were obtained. Through appropriate feature extraction, the infant’s limb movement complexity and the movement correlation between the limbs were integrated, and finally a comprehensive evaluation score for evaluating the complexity of the infant’s spontaneous movements was obtained. To verify our score, we did some experiments on a public dataset consisting of 12 real infants’ movements. Our method achieved the state-of-the-art with sensitivity of 100%, specificity of 87.5%, and accuracy of 91.67% on the public dataset whether in 2-D or 3-D posture input. We also collected another 47 cases of infant movement color videos ourselves. And achieved an accuracy of 91.5%, sensitivity of 100%, and specificity of 87.8% on all 59 (12 + 47) subjects. The results show that our score method has a great prospect in assessing the infant’s early brain development. The proposed method can be widely used in the screening of babies in homes and hospitals.
- 8 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 70 (00189456) , 1-11
Automatically Measure the Quality of Infants’ Spontaneous Movement via Videos to Predict the Risk of Cerebral Palsy
Early brain development assessment of infants is very important for infant growth and development. Timely diagnosis of cerebral palsy (CP) can reduce its negative effects. In this article, a novel automatic measurement method was proposed for the early prediction of infant CP with videos toward Prechtl’s general movement assessment (GMA). First, we adopted state-of-the-art human pose estimation method to obtain the 2-D or 3-D posture of the lying infant. After a series of interpolation and filtering, continuous infant’s limb movement curves were obtained. Through appropriate feature extraction, the infant’s limb movement complexity and the movement correlation between the limbs were integrated, and finally a comprehensive evaluation score for evaluating the complexity of the infant’s spontaneous movements was obtained. To verify our score, we did some experiments on a public dataset consisting of 12 real infants’ movements. Our method achieved the state-of-the-art with sensitivity of 100%, specificity of 87.5%, and accuracy of 91.67% on the public dataset whether in 2-D or 3-D posture input. We also collected another 47 cases of infant movement color videos ourselves. And achieved an accuracy of 91.5%, sensitivity of 100%, and specificity of 87.8% on all 59 (12 + 47) subjects. The results show that our score method has a great prospect in assessing the infant’s early brain development. The proposed method can be widely used in the screening of babies in homes and hospitals.