Unobtrusive and Automatic Classification of Multiple People’s Abnormal Respiratory Patterns in Real Time Using Deep Neural Network and Depth Camera
- 30 April 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 7 (9) , 8559-8571
Abstract
Respiratory pattern is a representation of human breathing activity, which can reflect people’s physical and psychological condition. Capturing the unexpected abnormal respiratory pattern unobtrusively of the patient or the potential patient has great significance. In the current work, we attempt to capitalize on depth camera and deep learning architecture to achieve the accurate and unobtrusive measurement of abnormal respiratory patterns, and the whole system can classify multiple people’s respiratory patterns in a real-time manner. The challenges in this task are threefold: the real-time online system means that the ROI (region of interest) needs to be located and tracked automatically; the amount of real-world data is not enough for training to obtain the robust deep neural network; and the intra-class variation is large and the outer-class variation is small. Consequently, human joints tracking is applied to determine the location of subjects shoulder and chest. Based on the characteristics of actual respiratory signals, a novel and efficient Respiratory Simulation Model (RSM) is proposed to generate abundant and high-quality training data. Finally, we apply a GRU neural network with bidirectional and attentional mechanisms (BI-AT-GRU) to classify 6 clinically significant respiratory patterns (Eupnea, Tachypnea, Bradypnea, Biots, Cheyne-Stokes and Central-Apnea). The performance of the obtained BI-AT-GRU is tested by the data that is actually measured by depth camera. Experimental results demonstrate that the proposed model can classify 6 different respiratory patterns with the accuracy, precision, recall and F1 of 94.5%, 94.4%, 95.1% and 94.8% respectively. In comparative experiments, the obtained BI-AT-GRU specific to respiratory pattern classification outperforms the existing state-of-the-art viz. BI-AT-LSTM, GRU and LSTM, BI-AT-GRU. Moreover, other experimental results indicate that the proposed online measuring system, deep neural network and the modeling ideas have the potential to be extended to the large scale applications such as public places, sleep scenario, and office environment. The demo videos of the proposed system are available at.Keywords
Funding Information
- Shanghai Sailing Program (19YF1414100)
- National Natural Science Foundation of China (61831015, 61901172, 61975056)
- Science and Technology Commission of Shanghai Municipality (18DZ2270700, 18511102500, 19511120100)
- Foundation of Key Laboratory of Artificial Intelligence, Ministry of Education (AI2019002)
This publication has 45 references indexed in Scilit:
- Respiration rate monitoring methods: A reviewPediatric Pulmonology, 2011
- Pathophysiology of Sleep ApneaPhysiological Reviews, 2010
- The objective medical emergency team activation criteria: A case–control studyResuscitation, 2007
- Defining eupneaRespiratory Physiology & Neurobiology, 2003
- Automated processing of the single-lead electrocardiogram for the detection of obstructive sleep apnoeaIEEE Transactions on Biomedical Engineering, 2003
- Generation of the respiratory rhythm: state-dependency and switchingNeurocomputing, 2002
- Sleep-Related Breathing Disorders and Risk of StrokeStroke, 2001
- Long Short-Term MemoryNeural Computation, 1997
- Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatientsJournal of General Internal Medicine, 1993
- Finding structure in timeCognitive Science, 1990