Yuchen Pan

Recognizing depression from speech is a promising approach, yet it poses a significant privacy risk by exposing sensitive information like speaker identity, gender, and conversation content. While encryption protects data in transit and storage, it fails to secure the raw audio during capture. Conversely, simple noise jamming renders the audio clinically useless. This paper introduces a solution that achieves both state-of-the-art recognition performance and robust privacy preservation. In this study, we propose PROMIND (Privacy-Protected Mental Health Intelligence with Noise Defense), a novel framework featuring a lightweight noise injector that is jointly optimized end-to-end with a depression recognition model. The noise injector, designed for local offline deployment on devices like a Raspberry Pi, generates a content-adaptive acoustic perturbation that mixes with a user's speech. This joint adversarial training forces the system to discover a new signal representation that preserves clinical biomarkers while protecting private information. Our comprehensive experiments on multiple depression datasets and privacy-related datasets demonstrate that PROMIND achieves a great balance between depression recognition performance and privacy protection, successfully defending attacks aimed at recovering privacy information. Furthermore, we validate PROMIND's practical viability with a successful real-world hardware deployment. Code is available at https://github.com/divertingPan/PROMIND.

Depression recognition (DR) using facial images, audio signals, or language text recordings has achieved remarkable performance. Recently, multimodal DR has shown improved performance over single-modal methods by leveraging information from a combination of these modalities. However, collecting high-quality data containing all modalities poses a challenge. In particular, these methods often encounter performance degradation when certain modalities are either missing or degraded. To tackle this issue, we present a generalizable multimodal framework for DR by aggregating feature disentanglement and privileged knowledge distillation. In detail, our approach aims to disentangle homogeneous and heterogeneous features within multimodal signals while suppressing noise, thereby adaptively aggregating the most informative components for high-quality DR. Subsequently, we leverage knowledge distillation to transfer privileged knowledge from complete modalities to the observed input with limited information, thereby significantly improving the tolerance and compatibility. These strategies form our novel Feature Disentanglement and Privileged knowledge Distillation Network for DR, dubbed Dis2DR. Experimental evaluations on AVEC 2013, AVEC 2014, AVEC 2017, and AVEC 2019 datasets demonstrate the effectiveness of our Dis2DR method. Remarkably, Dis2DR achieves superior performance even when only a single modality is available, surpassing existing state-of-the-art multimodal DR approaches AVA-DepressNet by up to 9.8% on the AVEC 2013 dataset.

Automatic depression diagnosis is a challenging problem, that requires integrating spatial-temporal information and extracting features from audio-visual signals. In terms of privacy protection, the development trend of recognition algorithms based on facial landmarks has created additional challenges and difficulties. In this paper, we propose an audio-visual attention network (AVA-DepressNet) for depression recognition. It is a novel multimodal framework with facial privacy protection, and uses attention-based modules to enhance audio-visual spatial and temporal features. In addition, an adversarial multistage (AMS) training strategy is developed to optimize the encoder-decoder structure. Additionally, facial structure prior knowledge is creatively used in AMS training. Our AVA-DepressNet is evaluated on popular audio-visual depression datasets: AVEC 2013, AVEC 2014, and AVEC 2017. The results show that our approach reaches the state-of-the-art performance or competitive results for depression recognition.

Recent visual-based depression recognition methods mostly use hand-crafted features with information lost in color channels, or deep network features with a limited performance from the finite data. In this paper, we propose a method called Local Quaternion and Global Deep Network (LQGDNet) which can combine advantages from hand-crafted and deep features. Specifically, the Quaternion XOR Asymmetrical Regional Local Gradient Coding (XOR-AR-LGC) is firstly designed, which encodes the facial images with local textures in the quaternion domain to keep the dependence of color channels, and integrated into the Quaternion Feature Extractor (QFE). To the best of our knowledge, it is the first attempt to use a quaternion-based method for facial depression recognition. Second, we design the Local Quaternion Representation Module (LQRM) composed of Local Deep Feature Extractor (LDFE) and QFE to output local quaternion facial features. Third, global deep facial features are encoded from the Global Deep Representation Module (GDRM) with the deep convolutional neural network. Finally, the LQGDNet integrates LQRM and GDRM with the local quaternion and global deep features and predicts the depression score. The experimental results on AVEC 2013 and AVEC 2014 show the superiority of our method compared to the state-of-the-art approaches.