Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces
Preprint
Abstract
Photoplethysmography (PPG) signals have become a key technology in many fields such as medicine, well-being, or sports. Our work proposes a set of pipelines to extract remote PPG signals (rPPG) from the face, robustly, reliably, and in a configurable manner. We identify and evaluate the possible choices in the critical steps of unsupervised rPPG methodologies. We evaluate a state-of-the-art processing pipeline in six different datasets, incorporating important corrections in the methodology that ensure reproducible and fair comparisons. In addition, we extend the pipeline by proposing three novel ideas; 1) a new method to stabilize the detected face based on a rigid mesh normalization; 2) a new method to dynamically select the different regions in the face that provide the best raw signals, and 3) a new RGB to rPPG transformation method called Orthogonal Matrix Image Transformation (OMIT) based on QR decomposition, that increases robustness against compression artifacts. We show that all three changes introduce noticeable improvements in retrieving rPPG signals from faces, obtaining state-of-the-art results compared with unsupervised, non-learning-based methodologies, and in some databases, very close to supervised, learning-based methods. We perform a comparative study to quantify the contribution of each proposed idea. In addition, we depict a series of observations that could help in future implementations.Keywords
All Related Versions
This publication has 53 references indexed in Scilit:
- Real-time face alignment: evaluation methods, training strategies and implementation optimizationJournal of Real-Time Image Processing, 2021
- Research Techniques Made Simple: Cutaneous Colorimetry: A Reliable Technique for Objective Skin Color MeasurementJournal of Investigative Dermatology, 2020
- Wearable sensors for monitoring the internal and external workload of the athletenpj Digital Medicine, 2019
- Unsupervised skin tissue segmentation for remote photoplethysmographyPattern Recognition Letters, 2019
- Current progress of photoplethysmography and SPO2 for health monitoringBiomedical Engineering Letters, 2019
- DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks2018
- Principal component analysis using QR decompositionInternational Journal of Machine Learning and Cybernetics, 2012
- Multi-PIEImage and Vision Computing, 2010
- Photoplethysmography and its application in clinical physiological measurementPhysiological Measurement, 2007
- Quantification of scaling exponents and crossover phenomena in nonstationary heartbeat time seriesChaos: An Interdisciplinary Journal of Nonlinear Science, 1995