PulseFormer: Continuous Remote Heart Rate Measurement Through Zoomed Time-Spectral Attention
Open Access
- 21 February 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biometrics, Behavior, and Identity Science
- Vol. 7 (4) , 876-889
Abstract
Despite the recent advances in remote heart rate measurement, most improvements primarily focus on recovering the rPPG signal, often overlooking the inherent challenges of estimating heart rate (HR) from the derived signal. Furthermore, most existing methods adopt the average HR per video to assess model performance, thus relying on rather large temporal windows to produce a single estimate; this hampers their applicability to scenarios in which the continuous monitoring of a patient’s physiological status is crucial. Besides, this evaluation approach can also lead to biased performance assessments due to low continuous precision, as it considers only the mean value of the entire video. In this paper, we present the PulseFormer, a novel continuous deep estimator for remote HR. Our proposed method utilizes a time-frequency attention block that leverages the enhanced resolution properties of the Chirp-Z Transform (CZT) to accurately estimate HR from the recovered low-resolution signal using a reduced temporal window size. We validate the effectiveness of our model on the large-scale Vision-for-Vitals (V4V) benchmark, designed for continuous physiological signals estimation from facial videos. The results reveal outstanding frame-to-frame HR estimation capabilities, establishing the proposed approach as a robust and versatile estimator that could be used with any rPPG method.Keywords
Funding Information
- eSCANFace Project funded by the Spanish Ministry of Science and Innovation (PID2020-114083GB-I00)
This publication has 64 references indexed in Scilit:
- PhysFormer++: Facial Video-Based Physiological Measurement with SlowFast Temporal Difference TransformerInternational Journal of Computer Vision, 2023
- Instantaneous Physiological Estimation Using Video Transformers2022
- Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution2022
- Augmentation of rPPG Benchmark Datasets: Learning to Remove and Embed rPPG Signals via Double Cycle Consistent Learning from Unpaired Facial Videos2022
- AND-rPPG: A novel denoising-rPPG network for improving remote heart rate estimationComputers in Biology and Medicine, 2022
- Spatial Temporal Transformer Network for Skeleton-Based Action Recognition2021
- Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-learner2020
- Learning Joint Spatial-Temporal Transformations for Video Inpainting2020
- DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks2018
- 3D Mask Face Anti-spoofing with Remote Photoplethysmography2016