Dual Attention and Element Recalibration Networks for Automatic Depression Level Prediction
- 26 May 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 14 (3) , 1954-1965
Abstract
Physiological studies have identified that facial dynamics can be considered as biomarkers to analyze depression severity. This paper accordingly develops a Dual Attention and Element Recalibration (DAER) network to extract facial changes to predict the depression level. In this model, we propose two blocks: a Dual Attention (DA) block and Element Recalibration (ER) block. The DA block uses the self-attention to investigate the dynamic changes in the representation sequence of a facial video segment. It further examines the influence of feature components of the representation sequence on depression level prediction through bilinear-attention. Moreover, to improve the representation ability of network, the ER block is used to obtain the global information to recalibrate each element of the tensor. Adopting this approach, for the depression level prediction task, we first divide the long-term video into fixed-length segments and use the trained ResNet50 to encode each frame to generate the representation sequences of video segments. Second, the representation sequences are input into DAER network to obtain the depression level scores. Finally, the average of these scores yields the prediction result corresponding to the long-term video. Experiments on publicly available AVEC 2013 and AVEC 2014 depression databases illustrate the effectiveness of our method.Keywords
Funding Information
- National Laboratory of Pattern Recognition (202200012)
- National Natural Science Foundation of China (62071330)
- New Talent Project of Beijing University of Posts and Telecommunications (2021RC37)
This publication has 41 references indexed in Scilit:
- A temporally piece-wise fisher vector approach for depression analysis2015
- Beyond short snippets: Deep networks for video classification2015
- Automated Depression Diagnosis Based on Facial Dynamic Analysis and Sparse CodingIEEE Transactions on Information Forensics and Security, 2015
- Ensemble CCA for Continuous Emotion Prediction2014
- AVEC 20142014
- Fusing Affective Dimensions and Audio-Visual Features from Segmented Video for Depression Recognition2014
- Depression recognition based on dynamic facial and vocal expression features using partial least square regression2013
- Diagnosis of depression by behavioural signals2013
- A narrative review of the Beck Depression Inventory (BDI) and implications for its use in an alcohol‐dependent populationJournal of Psychiatric and Mental Health Nursing, 2010
- Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial ExpressionsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007