Deep-Learning-Based Remote Photoplethysmography Measurement in Driving Scenarios With Color and Near-Infrared Images

Authors:
Publisher website
Google Scholar
Abstract
Driver monitoring is a popular topic of investigation. Contactless vital-sign measurement via remote photoplethysmography (rPPG) is an easy and effective alternative to cumbersome wearable devices during driving. However, driving is a noise-heavy situation for remote vital-sign sensing, owing to the disturbances caused by jolted cars or varying illumination from time and weather variations. In this study, we propose deep-learning models for rPPG construction and heart-rate estimation that work for both red-green-blue (RGB) and monochrome near-infrared (NIR) images. The proposed method overcomes the adaptation issues commonly seen in deep-learning based heart-rate estimation approaches and achieves a significant improvement in robustness for heart-rate measurement. To evaluate the performance thoroughly, in addition to a public dataset, we construct four testing driving sets for various practical conditions, including driving at night, on rainy days, and on long-term journeys. It is also worth noting that we conducted the assessment with a cross-dataset protocol to further verify the efficiency of our method. The experimental results on the five datasets show that our proposed method can perform better than the state-of-the-art method for both RGB and NIR images, especially in some challenging cases. The root-mean-square error (RMSE) for the situation with additional head motion conditions can be improved by 28.6% with RGB images and by 21.91% with NIR images on the open dataset. Moreover, RMSE can be improved by up to 42.6% at night, 14.41% on rainy days, and 18.78% on long-term journeys.
Funding Information
  • National Science and Technology Council (MOST 111-2221-E-A49-166-MY3)

This publication has 27 references indexed in Scilit: