In-Bed Pose Estimation: Deep Learning With Shallow Dataset
Open Access
- 14 January 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Translational Engineering in Health and Medicine
- Vol. 7, 1-12
Abstract
This paper presents a robust human posture and body parts detection method under a specific application scenario known as in-bed pose estimation. Although the human pose estimation for various computer vision (CV) applications has been studied extensively in the last few decades, the in-bed pose estimation using camera-based vision methods has been ignored by the CV community because it is assumed to be identical to the general purpose pose estimation problems. However, the in-bed pose estimation has its own specialized aspects and comes with specific challenges, including the notable differences in lighting conditions throughout the day and having pose distribution different from the common human surveillance viewpoint. In this paper, we demonstrate that these challenges significantly reduce the effectiveness of the existing general purpose pose estimation models. In order to address the lighting variation challenge, the infrared selective (IRS) image acquisition technique is proposed to provide uniform quality data under various lighting conditions. In addition, to deal with the unconventional pose perspective, a 2- end histogram of oriented gradient (HOG) rectification method is presented. The deep learning framework proves to be the most effective model in human pose estimation; however, the lack of large public dataset for in-bed poses prevents us from using a large network from scratch. In this paper, we explored the idea of employing a pre-trained convolutional neural network (CNN) model trained on large public datasets of general human poses and fine-tuning the model using our own shallow (limited in size and different in perspective and color) in-bed IRS dataset. We developed an IRS imaging system and collected IRS image data from several realistic life-size mannequins in a simulated hospital room environment. A pre-trained CNN called convolutional pose machine (CPM) was fine-tuned for in-bed pose estimation by re-training its specific intermediate layers. Using the HOG rectification method, the pose estimation performance of CPM improved significantly by 26.4% in the probability of correct key-point (PCK) criteria at PCK0.1 compared to the model without such rectification. Even testing with only well aligned in-bed pose images, our fine-tuned model still surpassed the traditionally tuned CNN by another 16.6% increase in pose estimation accuracy.Keywords
All Related Versions
Funding Information
- National Science Foundation (1755695)
This publication has 26 references indexed in Scilit:
- Changes in site of obstruction in obstructive sleep apnea patients according to sleep position: A DISE studyThe Laryngoscope, 2014
- Human Pose Estimation Using Body Parts Dependent Joint Regressors2013
- Using Linking Features in Learning Non-parametric Part Models2012
- Exploring the Spatial Hierarchy of Mixture Models for Human Pose Estimation2012
- Preferred Sleep Position on the Side is Associated with Carpal Tunnel SyndromeHAND, 2010
- THE ROLE OF POSTOPERATIVE PATIENT POSTURE IN THE RECURRENCE OF TRAUMATIC CHRONIC SUBDURAL HEMATOMA AFTER BURR-HOLE SURGERYNeurosurgery, 2007
- National Pressure Ulcer Advisory Panel's Updated Pressure Ulcer Staging SystemAdvances in Skin & Wound Care, 2007
- Histograms of Oriented Gradients for Human Detection2005
- Pictorial Structures for Object RecognitionInternational Journal of Computer Vision, 2005
- Determining the minimum-area encasing rectangle for an arbitrary closed curveCommunications of the ACM, 1975