BeatFormer: Efficient motion-robust remote heart rate estimation through unsupervised spectral zoomed attention filters
Preprint
Abstract
Remote photoplethysmography (rPPG) captures cardiac signals from facial videos and is gaining attention for its diverse applications. While deep learning has advanced rPPG estimation, it relies on large, diverse datasets for effective generalization. In contrast, handcrafted methods utilize physiological priors for better generalization in unseen scenarios like motion while maintaining computational efficiency. However, their linear assumptions limit performance in complex conditions, where deep learning provides superior pulsatile information extraction. This highlights the need for hybrid approaches that combine the strengths of both methods. To address this, we present BeatFormer, a lightweight spectral attention model for rPPG estimation, which integrates zoomed orthonormal complex attention and frequency-domain energy measurement, enabling a highly efficient model. Additionally, we introduce Spectral Contrastive Learning (SCL), which allows BeatFormer to be trained without any PPG or HR labels. We validate BeatFormer on the PURE, UBFC-rPPG, and MMPD datasets, demonstrating its robustness and performance, particularly in cross-dataset evaluations under motion scenarios.Keywords
All Related Versions
This publication has 56 references indexed in Scilit:
- PhysMamba: Efficient Remote Physiological Measurement with SlowFast Temporal Difference Mamba2025
- Implicit Neural Models to Extract Heart Rate from Video2024
- TransPhys: Transformer-based unsupervised contrastive learning for remote heart rate measurementBiomedical Signal Processing and Control, 2023
- PhysFormer++: Facial Video-Based Physiological Measurement with SlowFast Temporal Difference TransformerInternational Journal of Computer Vision, 2023
- Contrast-Phys: Unsupervised Video-Based Remote Physiological Measurement via Spatiotemporal Contrast2022
- Augmentation of rPPG Benchmark Datasets: Learning to Remove and Embed rPPG Signals via Double Cycle Consistent Learning from Unpaired Facial Videos2022
- A neonatal dataset and benchmark for non-contact neonatal heart rate monitoring based on spatio-temporal neural networksEngineering Applications of Artificial Intelligence, 2021
- Spatial Temporal Transformer Network for Skeleton-Based Action Recognition2021
- Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-learner2020
- DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks2018