Rex Liu
Rex Liu has a total of 22 co-authors. Here are the top co-authors by number of shared publications:
- Xin Liu
Department of Computer Science, University of California, Davis, CA 95616, USA
5 shared publications - Albara Ah Ramli
Department of Physical Medicine and Rehabilitation, School of Medicine, University of California, Davis, 1 Shields Ave., Davis, CA 95616, USA
3 shared publications - Erik K. Henricson
Department of Physical Medicine and Rehabilitation, School of Medicine, University of California, Davis, 1 Shields Ave., Davis, CA 95616, USA
2 shared publications - Huanle Zhang
School of Computer Science and Technology, Shandong University, Qingdao, China
2 shared publications - Alina Nicorici
Department of Physical Medicine and Rehabilitation School of Medicine University of California 1 Shields Ave 95616 Davis CA USA
1 shared publication - Amanda Lopez
Department of Physical Medicine and Rehabilitation School of Medicine University of California 1 Shields Ave 95616 Davis CA USA
1 shared publication
Latest Publications
- Open Access
MU-MAE: Multimodal Masked Autoencoders-Based One-Shot Learning
With the exponential growth of multimedia data, leveraging multimodal sensors presents a promising approach for improving accuracy in human activity recognition. Nevertheless, accurately identifying these activities using both video data and wearable sensor data presents challenges due to the labor-intensive data annotation, and reliance on external pretrained models or additional data. To address these challenges, we introduce Multimodal Masked Autoencoders-Based One-Shot Learning (Mu-MAE). Mu-MAE integrates a multimodal masked autoencoder with a synchronized masking strategy tailored for wearable sensors. This masking strategy compels the networks to capture more meaningful spatiotemporal features, which enables effective self-supervised pretraining without the need for external data. Furthermore, Mu-MAE leverages the representation extracted from multimodal masked autoencoders as prior information input to a cross-attention multimodal fusion layer. This fusion layer emphasizes spatiotemporal features requiring attention across different modalities while highlighting differences from other classes, aiding in the classification of various classes in metric-based one-shot learning. Comprehensive evaluations on MMAct one-shot classification show that Mu-MAE outperforms all the evaluated approaches, achieving up to an 80.17% accuracy for five-way one-shot multimodal classification, without the use of additional data.
Open AccessMU-MAE: Multimodal Masked Autoencoders-Based One-Shot Learning
With the exponential growth of multimedia data, leveraging multimodal sensors presents a promising approach for improving accuracy in human activity recognition. Nevertheless, accurately identifying these activities using both video data and wearable sensor data presents challenges due to the labor-intensive data annotation, and reliance on external pretrained models or additional data. To address these challenges, we introduce Multimodal Masked Autoencoders-Based One-Shot Learning (Mu-MAE). Mu-MAE integrates a multimodal masked autoencoder with a synchronized masking strategy tailored for wearable sensors. This masking strategy compels the networks to capture more meaningful spatiotemporal features, which enables effective self-supervised pretraining without the need for external data. Furthermore, Mu-MAE leverages the representation extracted from multimodal masked autoencoders as prior information input to a cross-attention multimodal fusion layer. This fusion layer emphasizes spatiotemporal features requiring attention across different modalities while highlighting differences from other classes, aiding in the classification of various classes in metric-based one-shot learning. Comprehensive evaluations on MMAct one-shot classification show that Mu-MAE outperforms all the evaluated approaches, achieving up to an 80.17% accuracy for five-way one-shot multimodal classification, without the use of additional data.
- Open Access
Gait Characterization in Duchenne Muscular Dystrophy (DMD) Using a Single-Sensor Accelerometer: Classical Machine Learning and Deep Learning Approaches
Differences in gait patterns of children with Duchenne muscular dystrophy (DMD) and typically developing (TD) peers are visible to the eye, but quantifications of those differences outside of the gait laboratory have been elusive. In this work, we measured vertical, mediolateral, and anteroposterior acceleration using a waist-worn iPhone accelerometer during ambulation across a typical range of velocities. Fifteen TD and fifteen DMD children from 3 to 16 years of age underwent eight walking/running activities, including five 25 m walk/run speed-calibration tests at a slow walk to running speeds (SC-L1 to SC-L5), a 6-min walk test (6MWT), a 100 m fast walk/jog/run (100MRW), and a free walk (FW). For clinical anchoring purposes, participants completed a Northstar Ambulatory Assessment (NSAA). We extracted temporospatial gait clinical features (CFs) and applied multiple machine learning (ML) approaches to differentiate between DMD and TD children using extracted temporospatial gait CFs and raw data. Extracted temporospatial gait CFs showed reduced step length and a greater mediolateral component of total power (TP) consistent with shorter strides and Trendelenberg-like gait commonly observed in DMD. ML approaches using temporospatial gait CFs and raw data varied in effectiveness at differentiating between DMD and TD controls at different speeds, with an accuracy of up to 100%. We demonstrate that by using ML with accelerometer data from a consumer-grade smartphone, we can capture DMD-associated gait characteristics in toddlers to teens.
Open AccessGait Characterization in Duchenne Muscular Dystrophy (DMD) Using a Single-Sensor Accelerometer: Classical Machine Learning and Deep Learning Approaches
Differences in gait patterns of children with Duchenne muscular dystrophy (DMD) and typically developing (TD) peers are visible to the eye, but quantifications of those differences outside of the gait laboratory have been elusive. In this work, we measured vertical, mediolateral, and anteroposterior acceleration using a waist-worn iPhone accelerometer during ambulation across a typical range of velocities. Fifteen TD and fifteen DMD children from 3 to 16 years of age underwent eight walking/running activities, including five 25 m walk/run speed-calibration tests at a slow walk to running speeds (SC-L1 to SC-L5), a 6-min walk test (6MWT), a 100 m fast walk/jog/run (100MRW), and a free walk (FW). For clinical anchoring purposes, participants completed a Northstar Ambulatory Assessment (NSAA). We extracted temporospatial gait clinical features (CFs) and applied multiple machine learning (ML) approaches to differentiate between DMD and TD children using extracted temporospatial gait CFs and raw data. Extracted temporospatial gait CFs showed reduced step length and a greater mediolateral component of total power (TP) consistent with shorter strides and Trendelenberg-like gait commonly observed in DMD. ML approaches using temporospatial gait CFs and raw data varied in effectiveness at differentiating between DMD and TD controls at different speeds, with an accuracy of up to 100%. We demonstrate that by using ML with accelerometer data from a consumer-grade smartphone, we can capture DMD-associated gait characteristics in toddlers to teens.
- Open Access
Uncovering the Gut–Liver Axis Biomarkers for Predicting Metabolic Burden in Mice
Western diet (WD) intake, aging, and inactivation of farnesoid X receptor (FXR) are risk factors for metabolic and chronic inflammation-related health issues ranging from metabolic dysfunction-associated steatotic liver disease (MASLD) to dementia. The progression of MASLD can be escalated when those risks are combined. Inactivation of FXR, the receptor for bile acid (BA), is cancer prone in both humans and mice. The current study used multi-omics including hepatic transcripts, liver, serum, and urine metabolites, hepatic BAs, as well as gut microbiota from mouse models to classify those risks using machine learning. A linear support vector machine with K-fold cross-validation was used for classification and feature selection. We have identified that increased urine sucrose alone achieved 91% accuracy in predicting WD intake. Hepatic lithocholic acid and serum pyruvate had 100% and 95% accuracy, respectively, to classify age. Urine metabolites (decreased creatinine and taurine as well as increased succinate) or increased gut bacteria (Dorea, Dehalobacterium, and Oscillospira) could predict FXR deactivation with greater than 90% accuracy. Human disease relevance is partly revealed using the metabolite–disease interaction network. Transcriptomics data were also compared with the human liver disease datasets. WD-reduced hepatic Cyp39a1 (cytochrome P450 family 39 subfamily a member 1) and increased Gramd1b (GRAM domain containing 1B) were also changed in human liver cancer and metabolic liver disease, respectively. Together, our data contribute to the identification of noninvasive biomarkers within the gut–liver axis to predict metabolic status.
Open AccessUncovering the Gut–Liver Axis Biomarkers for Predicting Metabolic Burden in Mice
Western diet (WD) intake, aging, and inactivation of farnesoid X receptor (FXR) are risk factors for metabolic and chronic inflammation-related health issues ranging from metabolic dysfunction-associated steatotic liver disease (MASLD) to dementia. The progression of MASLD can be escalated when those risks are combined. Inactivation of FXR, the receptor for bile acid (BA), is cancer prone in both humans and mice. The current study used multi-omics including hepatic transcripts, liver, serum, and urine metabolites, hepatic BAs, as well as gut microbiota from mouse models to classify those risks using machine learning. A linear support vector machine with K-fold cross-validation was used for classification and feature selection. We have identified that increased urine sucrose alone achieved 91% accuracy in predicting WD intake. Hepatic lithocholic acid and serum pyruvate had 100% and 95% accuracy, respectively, to classify age. Urine metabolites (decreased creatinine and taurine as well as increased succinate) or increased gut bacteria (Dorea, Dehalobacterium, and Oscillospira) could predict FXR deactivation with greater than 90% accuracy. Human disease relevance is partly revealed using the metabolite–disease interaction network. Transcriptomics data were also compared with the human liver disease datasets. WD-reduced hepatic Cyp39a1 (cytochrome P450 family 39 subfamily a member 1) and increased Gramd1b (GRAM domain containing 1B) were also changed in human liver cancer and metabolic liver disease, respectively. Together, our data contribute to the identification of noninvasive biomarkers within the gut–liver axis to predict metabolic status.
- Open Access
An Overview of Human Activity Recognition Using Wearable Sensors: Healthcare and Artificial Intelligence
With the rapid development of the internet of things (IoT) and artificial intelligence (AI) technologies, human activity recognition (HAR) has been applied in a variety of domains such as security and surveillance, human-robot interaction, and entertainment. Even though a number of surveys and review papers have been published, there is a lack of HAR overview papers focusing on healthcare applications that use wearable sensors. Therefore, we fill in the gap by presenting this overview paper. In particular, we present our projects to illustrate the system design of HAR applications for healthcare. Our projects include early mobility identification of human activities for intensive care unit (ICU) patients and gait analysis of Duchenne muscular dystrophy (DMD) patients. We cover essential components of designing HAR systems including sensor factors (e.g., type, number, and placement location), AI model selection (e.g., classical machine learning models versus deep learning models), and feature engineering. In addition, we highlight the challenges of such healthcare-oriented HAR systems and propose several research opportunities for both the medical and the computer science community.
Open AccessAn Overview of Human Activity Recognition Using Wearable Sensors: Healthcare and Artificial Intelligence
With the rapid development of the internet of things (IoT) and artificial intelligence (AI) technologies, human activity recognition (HAR) has been applied in a variety of domains such as security and surveillance, human-robot interaction, and entertainment. Even though a number of surveys and review papers have been published, there is a lack of HAR overview papers focusing on healthcare applications that use wearable sensors. Therefore, we fill in the gap by presenting this overview paper. In particular, we present our projects to illustrate the system design of HAR applications for healthcare. Our projects include early mobility identification of human activities for intensive care unit (ICU) patients and gait analysis of Duchenne muscular dystrophy (DMD) patients. We cover essential components of designing HAR systems including sensor factors (e.g., type, number, and placement location), AI model selection (e.g., classical machine learning models versus deep learning models), and feature engineering. In addition, we highlight the challenges of such healthcare-oriented HAR systems and propose several research opportunities for both the medical and the computer science community.
- Open Access
BWCNN: Blink to Word, a Real-Time Convolutional Neural Network Approach
Amyotrophic lateral sclerosis (ALS) is a progressive neurodegenerative disease of the brain and the spinal cord, which leads to paralysis of motor functions. Patients retain their ability to blink, which can be used for communication. Here, We present an Artificial Intelligence (AI) system that uses eye-blinks to communicate with the outside world, running on real-time Internet-of-Things (IoT) devices. The system uses a Convolutional Neural Network (CNN) to find the blinking pattern, which is defined as a series of Open and Closed states. Each pattern is mapped to a collection of words that manifest the patient's intent. To investigate the best trade-off between accuracy and latency, we investigated several Convolutional Network architectures, such as ResNet, SqueezeNet, DenseNet, and InceptionV3, and evaluated their performance. We found that the InceptionV3 architecture, after hyper-parameter fine-tuning on the specific task led to the best performance with an accuracy of 99.20% and 94ms latency. This work demonstrates how the latest advances in deep learning architectures can be adapted for clinical systems that ameliorate the patient's quality of life regardless of the point-of-care.
Open AccessBWCNN: Blink to Word, a Real-Time Convolutional Neural Network Approach
Amyotrophic lateral sclerosis (ALS) is a progressive neurodegenerative disease of the brain and the spinal cord, which leads to paralysis of motor functions. Patients retain their ability to blink, which can be used for communication. Here, We present an Artificial Intelligence (AI) system that uses eye-blinks to communicate with the outside world, running on real-time Internet-of-Things (IoT) devices. The system uses a Convolutional Neural Network (CNN) to find the blinking pattern, which is defined as a series of Open and Closed states. Each pattern is mapped to a collection of words that manifest the patient's intent. To investigate the best trade-off between accuracy and latency, we investigated several Convolutional Network architectures, such as ResNet, SqueezeNet, DenseNet, and InceptionV3, and evaluated their performance. We found that the InceptionV3 architecture, after hyper-parameter fine-tuning on the specific task led to the best performance with an accuracy of 99.20% and 94ms latency. This work demonstrates how the latest advances in deep learning architectures can be adapted for clinical systems that ameliorate the patient's quality of life regardless of the point-of-care.