Xiaoxin Guo

Facial video-based remote physiological measurement (rPPG) has gained prominence for its ability to non-invasively estimate vital signs such as heart rate (HR). rPPG measures HR by detecting variations in the diffuse reflection of light caused by blood volume changes in the skin, which is influenced by light conditions. Inspired by this property, we identify a new task, that is, to embed malicious information into facial video by subtly altering the light conditions. With this task, we can mislead state-of-the-art rPPG HR methods through natural and imperceptible lighting changes, aiming for two objectives: testing the resilience of rPPG methods against light variations and safeguarding heart rate data, which is crucial for individual privacy. However, such a task is non-trivial and should be able to adapt to different input videos automatically and generate natural and imperceptible spatial-temporal lighting perturbations. To address these challenges, we propose the AdversariaL vidEo Relighting aTtack (ALERT) method, which involves three modules: video lighting estimation (VLightE), adversarial temporal lighting prediction (AdvTLight), and adversarial temporal lighting injection (AdvTLightInj). VLightE is to estimate the spatial-temporal lighting conditions of the original video. AdvTLight predicts the adversarial spatial-temporal lighting conditions that are imperceptible but can mislead the HR detectors according to the original lighting conditions automatically. The final module (i.e., AdvTLightInj) is to inject the predicted adversarial lighting conditions into the input video and render a new one. Extensive experiments on UBFC-rPPG and PURE datasets demonstrate that ALERT generates realistic, imperceptible adversarial videos, effectively misleading 11 rPPG-based HR methods and outperforming all baseline methods. Moreover, our method can be used to protect the HR privacy of users directly and outperform two SOTA Privacy-protection-oriented methods significantly.

Recently, ShiftViT and its variants have attracted much attention for their simple and efficient shift operation, showing excellent efficacy in several tasks on natural images, surpassing Swin Transformer. However, considering the complexity of 3D multimodal images, which have higher dimensions than natural images, and the relative stability of the human tissue structure in medical images, the applicability of shift operation on 3D multimodal medical data has yet to be determined. This paper demonstrates that ShiftViT has enormous potential in 3D multimodal medical image analysis. Using 3D medical image segmentation as a representative downstream task, we investigate how shift operation can improve model performance. First, applying ShiftViT to 3D multimodal medical images not only effectively extracts global information but also significantly enhances the model's performance. Second, as a plug-and-play strategy, the shift operation can be integrated with other modules without adding additional computational burden, proving its flexibility in the overall system. Finally, we further investigate the generalizability of the shift operation by introducing a cascaded attention module, which provides useful insights to improve the generalizability of 3D medical image segmentation models. Through this study, we extend the application scope of ShiftViT and bring new exploration directions to the field of 3D multimodal medical image analysis. Our research results prove the feasibility of applying ShiftViT in 3D multimodal medical images and provide an effective and scalable model, which is expected to further promote the development of medical image processing technology. The code will be published in https://github.com/ydlam/3D-ShiftBTS.