Multimodal Gait Recognition for Neurodegenerative Diseases
- 11 March 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 52 (9) , 9439-9453
Abstract
In recent years, single modality-based gait recognition has been extensively explored in the analysis of medical images or other sensory data, and it is recognized that each of the established approaches has different strengths and weaknesses. As an important motor symptom, gait disturbance is usually used for diagnosis and evaluation of diseases; moreover, the use of multimodality analysis of the patient’s walking pattern compensates for the one-sidedness of single modality gait recognition methods that only learn gait changes in a single measurement dimension. The fusion of multiple measurement resources has demonstrated promising performance in the identification of gait patterns associated with individual diseases. In this article, as a useful tool, we propose a novel hybrid model to learn the gait differences between three neurodegenerative diseases, between patients with different severity levels of Parkinson’s disease, and between healthy individuals and patients, by fusing and aggregating data from multiple sensors. A spatial feature extractor (SFE) is applied to generating representative features of images or signals. In order to capture temporal information from the two modality data, a new correlative memory neural network (CorrMNN) architecture is designed for extracting temporal features. Afterward, we embed a multiswitch discriminator to associate the observations with individual state estimations. Compared with several state-of-the-art techniques, our proposed framework shows more accurate classification results.Keywords
All Related Versions
Funding Information
- U.K. Royal Society-Newton Advanced Fellowship (NA160342)
This publication has 40 references indexed in Scilit:
- A Q-backpropagated time delay neural network for diagnosing severity of gait disturbances in Parkinson’s diseaseJournal of Biomedical Informatics, 2016
- Gait Rhythm Fluctuation Analysis for Neurodegenerative Diseases by Empirical Mode DecompositionIEEE Transactions on Biomedical Engineering, 2016
- Symmetry Analysis of Gait between Left and Right Limb Using Cross-Fuzzy EntropyComputational and Mathematical Methods in Medicine, 2016
- A novel approach for analysis of altered gait variability in amyotrophic lateral sclerosisMedical & Biological Engineering & Computing, 2015
- Classification of neurodegenerative diseases using gait dynamics via deterministic learningInformation Sciences, 2015
- HMM for Classification of Parkinson’s Disease Based on the Raw Gait DataJournal of Medical Systems, 2014
- Gait Recognition in the Classification of Neurodegenerative Diseases2014
- Long Short-Term MemoryNeural Computation, 1997
- Footswitch system for measurement of the temporal parameters of gaitJournal of Biomechanics, 1995
- The gait abnormality of Huntington's diseaseNeurology, 1985