Abstract
Respiratory pattern is a representation of human breathing activity, which can reflect people’s physical and psychological condition. Capturing the unexpected abnormal respiratory pattern unobtrusively of the patient or the potential patient has great significance. In the current work, we attempt to capitalize on depth camera and deep learning architecture to achieve the accurate and unobtrusive measurement of abnormal respiratory patterns, and the whole system can classify multiple people’s respiratory patterns in a real-time manner. The challenges in this task are threefold: the real-time online system means that the ROI (region of interest) needs to be located and tracked automatically; the amount of real-world data is not enough for training to obtain the robust deep neural network; and the intra-class variation is large and the outer-class variation is small. Consequently, human joints tracking is applied to determine the location of subjects shoulder and chest. Based on the characteristics of actual respiratory signals, a novel and efficient Respiratory Simulation Model (RSM) is proposed to generate abundant and high-quality training data. Finally, we apply a GRU neural network with bidirectional and attentional mechanisms (BI-AT-GRU) to classify 6 clinically significant respiratory patterns (Eupnea, Tachypnea, Bradypnea, Biots, Cheyne-Stokes and Central-Apnea). The performance of the obtained BI-AT-GRU is tested by the data that is actually measured by depth camera. Experimental results demonstrate that the proposed model can classify 6 different respiratory patterns with the accuracy, precision, recall and F1 of 94.5%, 94.4%, 95.1% and 94.8% respectively. In comparative experiments, the obtained BI-AT-GRU specific to respiratory pattern classification outperforms the existing state-of-the-art viz. BI-AT-LSTM, GRU and LSTM, BI-AT-GRU. Moreover, other experimental results indicate that the proposed online measuring system, deep neural network and the modeling ideas have the potential to be extended to the large scale applications such as public places, sleep scenario, and office environment. The demo videos of the proposed system are available at.
Funding Information
  • Shanghai Sailing Program (19YF1414100)
  • National Natural Science Foundation of China (61831015, 61901172, 61975056)
  • Science and Technology Commission of Shanghai Municipality (18DZ2270700, 18511102500, 19511120100)
  • Foundation of Key Laboratory of Artificial Intelligence, Ministry of Education (AI2019002)

This publication has 45 references indexed in Scilit: