Gait Activity Classification Using Multi-Modality Sensor Fusion: A Deep Learning Approach
Open Access
- 5 May 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 21 (15) , 16870-16879
Abstract
Floor Sensors (FS) are used to capture information from the force induced on the contact surface by feet during gait. On the other hand, the Ambulatory Inertial Sensors (AIS) are used to capture the velocity, acceleration and orientation of the body during different activities. In this paper, fusion of the stated modalities is performed to overcome the challenge of gait classification from wearable sensors on the lower portion of human body not in contact with ground as in FS. Deep learning models are utilized for the automatic feature extraction of the ground reaction force obtained from a set of 116 FS and body movements from AIS attached at 3 different locations of lower body, which is novel. Spatio-temporal information of disproportionate inputs obtained from the two modalities is balanced and fused within deep learning network layers whilst reserving the categorical content for each gait activity. Our approach of fusion compensates the degradation in spatio-temporal accuracies in individual modalities and makes the overall classification outcomes more accurate. Further assessment of multi-modality based results show significant improvements in f-scores using different deep learning models i.e., LSTM (99.90%), 2D-CNN (88.73%), 1D-CNN (94.97%) and ANN (89.33%) respectively.Keywords
Funding Information
- Engineering and Physical Sciences Research Council (EPSRC) for the Award Sensor Fusion and Data Processing (1925970)
This publication has 57 references indexed in Scilit:
- The use of the dual-task paradigm in detecting gait performance deficits following a sports-related concussion: A systematic review and meta-analysisJournal of Science and Medicine in Sport, 2013
- Age-Related Deficits of Dual-Task Walking: A ReviewNeural Plasticity, 2012
- Gait Analysis Using Wearable SensorsSensors, 2012
- Dual-Task Complexity Affects Gait in People With Mild Cognitive Impairment: The Interplay Between Gait Variability, Dual Tasking, and Risk of FallsArchives of Physical Medicine and Rehabilitation, 2012
- Age-related differences in dual task walking: a cross sectional studyJournal of NeuroEngineering and Rehabilitation, 2008
- The role of executive function and attention in gaitMovement Disorders, 2007
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Long Short-Term MemoryNeural Computation, 1997
- A scaled conjugate gradient algorithm for fast supervised learningNeural Networks, 1993
- Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropyIEEE Transactions on Information Theory, 1980