Automatic Assessment of Upper Extremity Function and Mobile Application for Self-Administered Stroke Rehabilitation
Open Access
- 25 January 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Systems and Rehabilitation Engineering
- Vol. 32 (15344320) , 652-661
Abstract
Rehabilitation training is essential for a successful recovery of upper extremity function after stroke. Training programs are typically conducted in hospitals or rehabilitation centers, supervised by specialized medical professionals. However, frequent visits to hospitals can be burdensome for stroke patients with limited mobility. We consider a self-administered rehabilitation system based on a mobile application in which patients can periodically upload videos of themselves performing reach-to-grasp tasks to receive recommendations for self-managed exercises or progress reports. Sensing equipment aside from cameras is typically unavailable in the home environment. A key contribution of our work is to propose a deep learning-based assessment model trained only with video data. As all patients carry out identical tasks, a fine-grained assessment of task execution is required. Our model addresses this difficulty by learning RGB and optical flow data in a complementary manner. The correlation between the RGB and optical flow data is captured by a novel module for modality fusion using cross-attention with Transformers. Experiments showed that our model achieved higher accuracy in movement assessment than existing methods for action recognition. Based on the assessment model, we developed a patient-centered, solution-based mobile application for upper extremity exercises for hemiplegia, which can recommend 57 exercises with three levels of difficulty. A prototype of our application was evaluated by potential end-users and achieved a good quality score on the Mobile Application Rating Scale (MARS).Keywords
Funding Information
- ICT Creative Consilience program
- Institute for Information and Communications Technology Planning and Evaluation (IITP-2020-0-01819)
- National Research Foundation of Korea
- Ministry of Science and ICT (2021R1A2C1007215, 2022R1A5A1027646)
- Korea Medical Device Development Fund Grant by MSIT (1711195279, RS-2021-KD000009)
- Ministry of Trade, Industry and Energy
- Ministry of Health and Welfare
- Ministry of Food and Drug Safety of the Korean Government
- Basic Science Research Program through NRF
- Ministry of Science, ICT and Future Planning (NRF-2019R1G1A1009646)
This publication has 52 references indexed in Scilit:
- HDL-PSR: Modelling Spatio-Temporal Features Using Hybrid Deep Learning Approach for Post-Stroke RehabilitationNeural Processing Letters, 2022
- An optimized hybrid deep learning model using ensemble learning approach for human walking activities recognitionThe Journal of Supercomputing, 2021
- Pattern identification of different human joints for different human walking styles using inertial measurement unit (IMU) sensorArtificial Intelligence Review, 2021
- Reaching performance scale for stroke – Test‐retest reliability, measurement error, concurrent and discriminant validityPM&R, 2021
- mHealth Intervention Applications for Adults Living With the Effects of Stroke: A Scoping ReviewArchives of Rehabilitation Research and Clinical Translation, 2021
- Machine Learning for 3D Kinematic Analysis of Movements in NeurorehabilitationCurrent Neurology and Neuroscience Reports, 2020
- Medical Mobile Applications for Stroke Survivors and CaregiversJournal of Stroke and Cerebrovascular Diseases, 2019
- Systematic Review of Mobile Health Applications in RehabilitationArchives of Physical Medicine and Rehabilitation, 2019
- Kinect V2 implementation and testing of the reaching performance scale for motor evaluation of patients with neurological impairmentMedical Engineering & Physics, 2018
- Increasing the amount of usual rehabilitation improves activity after stroke: a systematic reviewJournal of Physiotherapy, 2016