Alexander Nelson
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Alexander Nelson has a total of 53 co-authors. Here are the top co-authors by number of shared publications:
Nilanjan Banerjee
Department of Physical Therapy and Rehabilitation Science, University of Maryland, School of Medicine, Baltimore, MD, USA
6 shared publicationsHaoyan Liu
Department of Computer Science and Computer Engineering, University of Arkansas, Fayetteville, USA
4 shared publications- Haoyan Liu
State Key Lab of Software Development Environment Beihang University China
4 shared publications Ryan Robucci
Department of Physical Therapy and Rehabilitation Science, University of Maryland, School of Medicine, Baltimore, MD, USA
4 shared publicationsChintan Patel
Department of Physical Therapy and Rehabilitation Science, University of Maryland, School of Medicine, Baltimore, MD, USA
3 shared publicationsGurashish Singh
Department of Computer Science and Electrical Engineering, University of Maryland at Baltimore County, Baltimore, MD, USA
3 shared publications
Latest Publications
- Open Access
Effects of Light Wavelength on Broiler Performance, Blood Cell Profiles, Stress Levels, and Tibiotarsi Morphology
Lighting influences broiler production, health, and behavior. The objective of this study was to examine the effects of three light wavelengths (White [350–780 nm], Blue [450 nm], and Green [560 nm]) on broiler production, activity, fear, stress, blood cell profiles, and tibiotarsi (tibia) morphology. Day-of-hatch male broiler chicks (N = 600) were housed in pens (N = 12) with one lighting treatment for 42 days. Body weight and feed consumption were recorded on day (D) 0, 14, 28, and 42, and the feed conversion ratio was calculated. The Tonic Immobility test was used to assess the latency (seconds) to right from the testing cradle (D12 and D33). Blood was drawn for leukocyte and plasma corticosterone concentrations (D21 and D41). Accelerometers were attached at 2 and 5 weeks of age to measure activity. On D41, thermal images of the head were taken to measure surface temperatures (eye and beak), the bursa of Fabricius (bursa) was extracted for relative bursa weight, and the right and left tibias were extracted for tibia morphology. After sampling, the remaining broilers were processed. Activity was greatest in Green light at Week 2 (261.17 ± 8.52 m/s2, p < 0.0001), and activity was lowest in White light at Week 5 (98.99 ± 8.52 m/s2, p < 0.0001). Broilers under Blue light had lower concentrations of lymphocytes (8.62 ± 0.40 × 103 cells/µL, p = 0.01) and T cells (7.16 ± 0.33 × 103 cells/µL, p = 0.008) compared to White light. Breast yields were greatest (26.89 ± 0.13%, p = 0.01) in the White treatments compared to Green and Blue treatments. These results suggest that blue light may negatively impact immune health, while green light increases activity, without decreasing production performance, and white light can improve carcass traits. Therefore, lighting color can be strategically used to target welfare or production goals.
Open AccessEffects of Light Wavelength on Broiler Performance, Blood Cell Profiles, Stress Levels, and Tibiotarsi Morphology
Lighting influences broiler production, health, and behavior. The objective of this study was to examine the effects of three light wavelengths (White [350–780 nm], Blue [450 nm], and Green [560 nm]) on broiler production, activity, fear, stress, blood cell profiles, and tibiotarsi (tibia) morphology. Day-of-hatch male broiler chicks (N = 600) were housed in pens (N = 12) with one lighting treatment for 42 days. Body weight and feed consumption were recorded on day (D) 0, 14, 28, and 42, and the feed conversion ratio was calculated. The Tonic Immobility test was used to assess the latency (seconds) to right from the testing cradle (D12 and D33). Blood was drawn for leukocyte and plasma corticosterone concentrations (D21 and D41). Accelerometers were attached at 2 and 5 weeks of age to measure activity. On D41, thermal images of the head were taken to measure surface temperatures (eye and beak), the bursa of Fabricius (bursa) was extracted for relative bursa weight, and the right and left tibias were extracted for tibia morphology. After sampling, the remaining broilers were processed. Activity was greatest in Green light at Week 2 (261.17 ± 8.52 m/s2, p < 0.0001), and activity was lowest in White light at Week 5 (98.99 ± 8.52 m/s2, p < 0.0001). Broilers under Blue light had lower concentrations of lymphocytes (8.62 ± 0.40 × 103 cells/µL, p = 0.01) and T cells (7.16 ± 0.33 × 103 cells/µL, p = 0.008) compared to White light. Breast yields were greatest (26.89 ± 0.13%, p = 0.01) in the White treatments compared to Green and Blue treatments. These results suggest that blue light may negatively impact immune health, while green light increases activity, without decreasing production performance, and white light can improve carcass traits. Therefore, lighting color can be strategically used to target welfare or production goals.
- Open Access
- 13 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 33631-33642
SoGAR: Self-Supervised Spatiotemporal Attention-Based Social Group Activity Recognition
Social group activity recognition is crucial for various applications including surveillance, human-robot interaction, and behavioral analysis. Current approaches often require extensive manual annotations and rely heavily on pre-trained detectors, limiting their practical applications. Additionally, existing methods struggle to effectively model long-term spatiotemporal relationships in group activities. This paper introduces a novel approach to Social Group Activity Recognition (SoGAR) using Self-supervised Transformers network that can effectively utilize unlabeled video data. To extract spatio-temporal information, we create local and global views with varying frame rates. Our self-supervised objective ensures that features extracted from contrasting views of the same video are consistent across spatio-temporal domains. Our proposed approach efficiently uses transformer-based encoders to alleviate the weakly supervised setting of group activity recognition. By leveraging the benefits of transformer models, our approach can model long-term relationships along spatio-temporal dimensions. Our proposed SoGAR method achieves state-of-the-art results on three group activity recognition benchmarks, namely JRDB-PAR, NBA, and Volleyball datasets, surpassing the current state-of-the-art in terms of F1-score, MCA, and MPCA metrics.
- 13 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 33631-33642
Open AccessSoGAR: Self-Supervised Spatiotemporal Attention-Based Social Group Activity Recognition
Social group activity recognition is crucial for various applications including surveillance, human-robot interaction, and behavioral analysis. Current approaches often require extensive manual annotations and rely heavily on pre-trained detectors, limiting their practical applications. Additionally, existing methods struggle to effectively model long-term spatiotemporal relationships in group activities. This paper introduces a novel approach to Social Group Activity Recognition (SoGAR) using Self-supervised Transformers network that can effectively utilize unlabeled video data. To extract spatio-temporal information, we create local and global views with varying frame rates. Our self-supervised objective ensures that features extracted from contrasting views of the same video are consistent across spatio-temporal domains. Our proposed approach efficiently uses transformer-based encoders to alleviate the weakly supervised setting of group activity recognition. By leveraging the benefits of transformer models, our approach can model long-term relationships along spatio-temporal dimensions. Our proposed SoGAR method achieves state-of-the-art results on three group activity recognition benchmarks, namely JRDB-PAR, NBA, and Volleyball datasets, surpassing the current state-of-the-art in terms of F1-score, MCA, and MPCA metrics.
- 24 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 3, 1-6
Towards Cloud-based Infrastructure for Post-Quantum Cryptography Side-channel Attack Analysis
Post-quantum cryptography (PQC) refers to cryptographic algorithms that are thought to be secure against a cryptanalytic attack by a quantum computer. Before PQC algorithms can be widely deployed to replace the current standards such as the RSA algorithm, they need to be rigorously evaluated theoretically and practically. In this work, we present a cloud-based infrastructure being developed for performing side-channel analysis on PQC algorithms for the research community. Multiple types of side-channel attacks, such as timing attacks, power attacks, and electromagnetic attacks can be applied on different types of devices, such as FPGA devices and microcontrollers. An automated tool flow is being developed that can run executables on the target devices, collect traces (e.g., power consumption waveforms and electromagnetic radiation signals), perform leakage assessment (using Test Vector Leakage Assessment), and generate analysis reports. Remote users access the infrastructure through a web portal by uploading the hardware or software implementations of cryptographic algorithms. Side-channel attack and leakage analysis are performed on the given implementation. Finally, the user is informed for downloading the analysis report from the portal.
- 24 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 3, 1-6
Towards Cloud-based Infrastructure for Post-Quantum Cryptography Side-channel Attack Analysis
Post-quantum cryptography (PQC) refers to cryptographic algorithms that are thought to be secure against a cryptanalytic attack by a quantum computer. Before PQC algorithms can be widely deployed to replace the current standards such as the RSA algorithm, they need to be rigorously evaluated theoretically and practically. In this work, we present a cloud-based infrastructure being developed for performing side-channel analysis on PQC algorithms for the research community. Multiple types of side-channel attacks, such as timing attacks, power attacks, and electromagnetic attacks can be applied on different types of devices, such as FPGA devices and microcontrollers. An automated tool flow is being developed that can run executables on the target devices, collect traces (e.g., power consumption waveforms and electromagnetic radiation signals), perform leakage assessment (using Test Vector Leakage Assessment), and generate analysis reports. Remote users access the infrastructure through a web portal by uploading the hardware or software implementations of cryptographic algorithms. Side-channel attack and leakage analysis are performed on the given implementation. Finally, the user is informed for downloading the analysis report from the portal.
- Open Access
SPARTAN: Self-supervised Spatiotemporal Transformers Approach to Group Activity Recognition
In this paper, we propose a new, simple, and effective Self-supervised Spatio-temporal Transformers (SPARTAN) approach to Group Activity Recognition (GAR) using unlabeled video data. Given a video, we create local and global Spatio-temporal views with varying spatial patch sizes and frame rates. The proposed self-supervised objective aims to match the features of these contrasting views representing the same video to be consistent with the variations in spatiotemporal domains. To the best of our knowledge, the proposed mechanism is one of the first works to alleviate the weakly supervised setting of GAR using the encoders in video transformers. Furthermore, using the advantage of transformer models, our proposed approach supports long-term relationship modeling along spatio-temporal dimensions. The proposed SPARTAN approach performs well on two group activity recognition benchmarks, including NBA and Volleyball datasets, by surpassing the state-of-the-art results by a significant margin in terms of MCA and MPCA metrics.
Open AccessSPARTAN: Self-supervised Spatiotemporal Transformers Approach to Group Activity Recognition
In this paper, we propose a new, simple, and effective Self-supervised Spatio-temporal Transformers (SPARTAN) approach to Group Activity Recognition (GAR) using unlabeled video data. Given a video, we create local and global Spatio-temporal views with varying spatial patch sizes and frame rates. The proposed self-supervised objective aims to match the features of these contrasting views representing the same video to be consistent with the variations in spatiotemporal domains. To the best of our knowledge, the proposed mechanism is one of the first works to alleviate the weakly supervised setting of GAR using the encoders in video transformers. Furthermore, using the advantage of transformer models, our proposed approach supports long-term relationship modeling along spatio-temporal dimensions. The proposed SPARTAN approach performs well on two group activity recognition benchmarks, including NBA and Volleyball datasets, by surpassing the state-of-the-art results by a significant margin in terms of MCA and MPCA metrics.
- 1 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 2023, 80-87
Visual Enhancement and Semantic Segmentation of Murine Tissue Scans with Pulsed THz Spectroscopy
Semantic Artificial Intelligence has certain qualities that are advantageous for deep learning-based medical imaging tasks. Medical images can be augmented by injecting semantic context into the underlying classification mechanism, increasing the information density of the scan and ultimately can provide more trust in the result. This work considers an application of semantic AI to segment tissue types from excised breast tumors imaged with pulsed terahertz (THz)–an emerging imaging technology. Prior work has demonstrated traditional data driven methodology for deep learning on THz has two key weaknesses: namely 1) low image resolution compared to other state-of-the-art imaging techniques and 2) a lack of expertly-labeled images due to domain transformation and tissue changes during histopathology. This work seeks to address these limitations through two semantic AI mechanisms. Specifically, we introduce a two stage pipeline using an unsupervised image-to-image translation network and a supervised segmentation network. The combination of these contributions enables enhanced near-real-time visualization of excised tissue through THz scans and a supervised segmentation and classification training strategy using only synthetic THz scans generated by our bi-directional image-to-image translation network.
- 1 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 2023, 80-87
Visual Enhancement and Semantic Segmentation of Murine Tissue Scans with Pulsed THz Spectroscopy
Semantic Artificial Intelligence has certain qualities that are advantageous for deep learning-based medical imaging tasks. Medical images can be augmented by injecting semantic context into the underlying classification mechanism, increasing the information density of the scan and ultimately can provide more trust in the result. This work considers an application of semantic AI to segment tissue types from excised breast tumors imaged with pulsed terahertz (THz)–an emerging imaging technology. Prior work has demonstrated traditional data driven methodology for deep learning on THz has two key weaknesses: namely 1) low image resolution compared to other state-of-the-art imaging techniques and 2) a lack of expertly-labeled images due to domain transformation and tissue changes during histopathology. This work seeks to address these limitations through two semantic AI mechanisms. Specifically, we introduce a two stage pipeline using an unsupervised image-to-image translation network and a supervised segmentation network. The combination of these contributions enables enhanced near-real-time visualization of excised tissue through THz scans and a supervised segmentation and classification training strategy using only synthetic THz scans generated by our bi-directional image-to-image translation network.
- 12 January 2022
- Published by SPIE-Intl Soc Optical Eng in Journal of Medical Imaging
- Vol. 9 (01) , 014002-20
Hyperspectral terahertz imaging and optical clearance for cancer classification in breast tumor surgical specimen
Purpose: We investigate the enhancement in terahertz (THz) images of freshly excised breast tumors upon treatment with an optical clearance agent. The hyperspectral imaging and spectral classifications are used to quantitatively demonstrate the image enhancement. Glycerol solution with 60% concentration is applied to excised breast tumor specimens for various time durations to investigate the effectiveness on image enhancement. Approach: THz reflection spectroscopy is utilized to obtain the absorption coefficient and the index of refraction of untreated and glycerol-treated tissues at each frequency up to 3 THz. Two classifiers, spectral angular mapping (SAM) based on several kernels and Euclidean minimum distance (EMD) are implemented to evaluate the effectiveness of the treatment. The testing raw data is obtained from five breast cancer specimens: two untreated specimens and three specimens treated with glycerol solution for 20, 40, or 60 min. All tumors used in the testing data have healthy tissues adjacent to cancerous ones consistent with the challenge faced in lumpectomy surgeries. Results: The glycerol-treated tissues showed a decrease in the absorption coefficients compared with untreated tissues, especially as the period of treatment increased. Although the sensitivity metric of the classifier presented higher values in the untreated tissues compared with the treated ones, the specificity and accuracy metrics demonstrated higher values for the treated tissues compared with the untreated ones. Conclusions: The biocompatible glycerol solution is a potential optical clearance agent in THz imaging while keeping the histopathology imaging intact. The SAM technique provided a good classification of cancerous tissues despite the small amount of cancer in the training data (only 7%). The SAM exponential kernel and EMD presented classification accuracy of ∼80 % to 85% compared with linear and polynomial kernels that provided accuracy ranging from 70% to 80%. Overall, glycerol treatment provides a potential improvement in cancer classification in freshly excised breast tumors.
- 12 January 2022
- Published by SPIE-Intl Soc Optical Eng in Journal of Medical Imaging
- Vol. 9 (01) , 014002-20
Hyperspectral terahertz imaging and optical clearance for cancer classification in breast tumor surgical specimen
Purpose: We investigate the enhancement in terahertz (THz) images of freshly excised breast tumors upon treatment with an optical clearance agent. The hyperspectral imaging and spectral classifications are used to quantitatively demonstrate the image enhancement. Glycerol solution with 60% concentration is applied to excised breast tumor specimens for various time durations to investigate the effectiveness on image enhancement. Approach: THz reflection spectroscopy is utilized to obtain the absorption coefficient and the index of refraction of untreated and glycerol-treated tissues at each frequency up to 3 THz. Two classifiers, spectral angular mapping (SAM) based on several kernels and Euclidean minimum distance (EMD) are implemented to evaluate the effectiveness of the treatment. The testing raw data is obtained from five breast cancer specimens: two untreated specimens and three specimens treated with glycerol solution for 20, 40, or 60 min. All tumors used in the testing data have healthy tissues adjacent to cancerous ones consistent with the challenge faced in lumpectomy surgeries. Results: The glycerol-treated tissues showed a decrease in the absorption coefficients compared with untreated tissues, especially as the period of treatment increased. Although the sensitivity metric of the classifier presented higher values in the untreated tissues compared with the treated ones, the specificity and accuracy metrics demonstrated higher values for the treated tissues compared with the untreated ones. Conclusions: The biocompatible glycerol solution is a potential optical clearance agent in THz imaging while keeping the histopathology imaging intact. The SAM technique provided a good classification of cancerous tissues despite the small amount of cancer in the training data (only 7%). The SAM exponential kernel and EMD presented classification accuracy of ∼80 % to 85% compared with linear and polynomial kernels that provided accuracy ranging from 70% to 80%. Overall, glycerol treatment provides a potential improvement in cancer classification in freshly excised breast tumors.
- 7 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 22 (4) , 3605-3615
An FPGA-Based Upper-Limb Rehabilitation Device for Gesture Recognition and Motion Evaluation Using Multi-Task Recurrent Neural Networks
Upper-Extremity motor impairment affects millions of Americans due to cerebrovascular incidents, spinal cord injuries, or brain trauma. Current therapy practices used to assist these individuals in regaining motor functionality often require extensive time at rehabilitation facilities with potentially prohibitive travel or financial costs. This work presents a mobile low-cost field programmable gate array (FPGA)-smart rehabilitation system that can be used in home environments. The prototype is a rehabilitation table instrumented with a capacitive sensor array (CSA) to track upper-extremity motions of the user through proximity or touch. In addition, inertial measurement units (IMUs) are placed on the affected upper limb and combined with the CSA data with our sensor fusion signal processing architecture. Motions are classified and evaluated using multi-task convolutional recurrent neural networks with three additional motion quality output classes to personalize recognition based on the particular motor skills of each patient. The prototype achieves above 99% accuracy with 32-bit fixed-point format implementation for recognizing dynamic motions and identifying unnatural characteristics (i.e., tremor or limited flexion and extension) in upper limb motions based on sensor values. The convolutional recurrent neural network (C-RNN) fusion classification network is implemented on a 200 MHz Zynq ZCU104 FPGA using an HLS-based design optimized with pipelining and parallelism techniques and achieves 5.4x speedup compared to ARM® Cortex-A53 implementation running at an operating frequency of 1.3 GHz. The prototype is also demonstrated to perform the machine learning classification in real-time.
- 7 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 22 (4) , 3605-3615
An FPGA-Based Upper-Limb Rehabilitation Device for Gesture Recognition and Motion Evaluation Using Multi-Task Recurrent Neural Networks
Upper-Extremity motor impairment affects millions of Americans due to cerebrovascular incidents, spinal cord injuries, or brain trauma. Current therapy practices used to assist these individuals in regaining motor functionality often require extensive time at rehabilitation facilities with potentially prohibitive travel or financial costs. This work presents a mobile low-cost field programmable gate array (FPGA)-smart rehabilitation system that can be used in home environments. The prototype is a rehabilitation table instrumented with a capacitive sensor array (CSA) to track upper-extremity motions of the user through proximity or touch. In addition, inertial measurement units (IMUs) are placed on the affected upper limb and combined with the CSA data with our sensor fusion signal processing architecture. Motions are classified and evaluated using multi-task convolutional recurrent neural networks with three additional motion quality output classes to personalize recognition based on the particular motor skills of each patient. The prototype achieves above 99% accuracy with 32-bit fixed-point format implementation for recognizing dynamic motions and identifying unnatural characteristics (i.e., tremor or limited flexion and extension) in upper limb motions based on sensor values. The convolutional recurrent neural network (C-RNN) fusion classification network is implemented on a 200 MHz Zynq ZCU104 FPGA using an HLS-based design optimized with pipelining and parallelism techniques and achieves 5.4x speedup compared to ARM® Cortex-A53 implementation running at an operating frequency of 1.3 GHz. The prototype is also demonstrated to perform the machine learning classification in real-time.
- 1 January 2022
- Published by Springer Nature in Journal of Infrared, Millimeter, and Terahertz Waves
- Vol. 43 (1-2) , 48-70
Deep Learning Classification of Breast Cancer Tissue from Terahertz Imaging Through Wavelet Synchro-Squeezed Transformation and Transfer Learning
Terahertz imaging and spectroscopy is an exciting technology that has the potential to provide insights in medical imaging. Prior research has leveraged statistical inference to classify tissue regions from terahertz images. To date, these approaches have shown that the segmentation problem is challenging for images of fresh tissue and for tumors that have invaded muscular regions. Artificial intelligence, particularly machine learning and deep learning, has been shown to improve performance in some medical imaging challenges. This paper builds on that literature by modifying a set of deep learning approaches to the challenge of classifying tissue regions of images captured by terahertz imaging and spectroscopy of freshly excised murine xenograft tissue. Our approach is to preprocess the images through a wavelet synchronous-squeezed transformation (WSST) to convert time-sequential terahertz data of each THz pixel to a spectrogram. Spectrograms are used as input tensors to a deep convolution neural network for pixel-wise classification. Based on the classification result of each pixel, a cancer tissue segmentation map is achieved. In experimentation, we adopt leave-one-sample-out cross-validation strategy, and evaluate our chosen networks and results using multiple metrics such as accuracy, precision, intersection, and size. The results from this experimentation demonstrate improvement in classification accuracy compared to statistical methods, an improvement to segmentation between muscle and cancerous regions in xenograft tumors, and identify areas to improve the imaging and classification methodology.
- 1 January 2022
- Published by Springer Nature in Journal of Infrared, Millimeter, and Terahertz Waves
- Vol. 43 (1-2) , 48-70
Deep Learning Classification of Breast Cancer Tissue from Terahertz Imaging Through Wavelet Synchro-Squeezed Transformation and Transfer Learning
Terahertz imaging and spectroscopy is an exciting technology that has the potential to provide insights in medical imaging. Prior research has leveraged statistical inference to classify tissue regions from terahertz images. To date, these approaches have shown that the segmentation problem is challenging for images of fresh tissue and for tumors that have invaded muscular regions. Artificial intelligence, particularly machine learning and deep learning, has been shown to improve performance in some medical imaging challenges. This paper builds on that literature by modifying a set of deep learning approaches to the challenge of classifying tissue regions of images captured by terahertz imaging and spectroscopy of freshly excised murine xenograft tissue. Our approach is to preprocess the images through a wavelet synchronous-squeezed transformation (WSST) to convert time-sequential terahertz data of each THz pixel to a spectrogram. Spectrograms are used as input tensors to a deep convolution neural network for pixel-wise classification. Based on the classification result of each pixel, a cancer tissue segmentation map is achieved. In experimentation, we adopt leave-one-sample-out cross-validation strategy, and evaluate our chosen networks and results using multiple metrics such as accuracy, precision, intersection, and size. The results from this experimentation demonstrate improvement in classification accuracy compared to statistical methods, an improvement to segmentation between muscle and cancerous regions in xenograft tumors, and identify areas to improve the imaging and classification methodology.
- 1 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 225
FPGA-Based Gesture Recognition with Capacitive Sensor Array using Recurrent Neural Networks
This work presents a prototype of an FPGA-based hand motion recognition system using a capacitive sensor array (CSA). The prototype system is being developed as a tool to evaluate upper-limb motor skills for assistive or rehabilitative applications. A light-weight gesture segmentation algorithm was developed that uses summation and moving average filtering of quantized capacitive sensing data to segment motions. The time-series hand motions are then recognized through a recurrent classifier based on long short-term memory (LSTM) neural networks. The classifier model is trained on uni-stroke hand written digit ('0'-'9') samples obtained from four volunteers. A total of 12,000 hand motion samples are collected. The accuracy of 10-fold and leave-one-user-out cross-validation accuracy is respectively 97.5% and 91.3% using a two-layer LSTM network. The LSTM classifier is implemented on a Zynq FPGA device. The experiment demonstrated that the FPGA implementation of the LSTM-based classifier can achieve real-time gesture classification with capacitive sensor data.
- 1 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 225
FPGA-Based Gesture Recognition with Capacitive Sensor Array using Recurrent Neural Networks
This work presents a prototype of an FPGA-based hand motion recognition system using a capacitive sensor array (CSA). The prototype system is being developed as a tool to evaluate upper-limb motor skills for assistive or rehabilitative applications. A light-weight gesture segmentation algorithm was developed that uses summation and moving average filtering of quantized capacitive sensing data to segment motions. The time-series hand motions are then recognized through a recurrent classifier based on long short-term memory (LSTM) neural networks. The classifier model is trained on uni-stroke hand written digit ('0'-'9') samples obtained from four volunteers. A total of 12,000 hand motion samples are collected. The accuracy of 10-fold and leave-one-user-out cross-validation accuracy is respectively 97.5% and 91.3% using a two-layer LSTM network. The LSTM classifier is implemented on a Zynq FPGA device. The experiment demonstrated that the FPGA implementation of the LSTM-based classifier can achieve real-time gesture classification with capacitive sensor data.
- 1 October 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-4
Gesture Classification with Low-Cost Capacitive Sensor Array for Upper Extremity Rehabilitation
Machine Learning and artificial intelligence play major roles in understanding human activity through various classification and regression tasks. However, for many lowresource devices, high computation cost resulting from the construction of AI models may limit their applications. To that end, this work explores gesture recognition through a low-cost capacitive sensor matrix overlayed on a rehabilitation activity table. For gesture recognition, a convolutional long short-term memory (C-LSTM) neural network structure is applied and hyper-parameters are varied to determine what resources are necessary to perform classification tasks. The 8 X 8 mutual capacitive sensor array (CSA) is constructed with low-cost copper adhesive. The designed capacitive sensors capture hand motions performed by patients during rehabilitative exercise. The motions cause changes in the electric field that is quantified through sampling the changing capacitance between the copper tape electrodes. An MSP430 MCU computes the capacitance-todigital conversion at a 50 Hz sampling rate. To identify low computation cost models for the C-LSTM neural network, we evaluate different numbers of capacitor sensors, kernels, convolutional layers, and hidden nodes. Six subjects performed 1200 gestures, and the accuracy metrics are calculated using fivefold cross-validation.
- 1 October 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-4
Gesture Classification with Low-Cost Capacitive Sensor Array for Upper Extremity Rehabilitation
Machine Learning and artificial intelligence play major roles in understanding human activity through various classification and regression tasks. However, for many lowresource devices, high computation cost resulting from the construction of AI models may limit their applications. To that end, this work explores gesture recognition through a low-cost capacitive sensor matrix overlayed on a rehabilitation activity table. For gesture recognition, a convolutional long short-term memory (C-LSTM) neural network structure is applied and hyper-parameters are varied to determine what resources are necessary to perform classification tasks. The 8 X 8 mutual capacitive sensor array (CSA) is constructed with low-cost copper adhesive. The designed capacitive sensors capture hand motions performed by patients during rehabilitative exercise. The motions cause changes in the electric field that is quantified through sampling the changing capacitance between the copper tape electrodes. An MSP430 MCU computes the capacitance-todigital conversion at a 50 Hz sampling rate. To identify low computation cost models for the C-LSTM neural network, we evaluate different numbers of capacitor sensors, kernels, convolutional layers, and hidden nodes. Six subjects performed 1200 gestures, and the accuracy metrics are calculated using fivefold cross-validation.