MTDAN: A Lightweight Multi-Scale Temporal Difference Attention Networks for Automated Video Depression Detection
- 25 September 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 15 (3) , 1078-1089
Abstract
Deep learning based video depression analysis has been recently an interesting and challenging topic. Most of existing works focus on learning single-scale facial dynamics of participants for depression detection. Besides, they usually adopt expensive deep learning models with high computational complexity, resulting in difficulty in real-time clinical applications. To address these two issues, this work proposes a lightweight Multi-scale Temporal Difference Attention Networks (MTDAN) integrating the temporal difference and attention mechanism to model both short-term and long-term temporal facial behaviors for automated video depression detection. Initially, two simple yet effective sub-branches, i.e., a Short-term Temporal Difference Attention Network (ST-TDAN), and a Long-term Temporal Difference Attention Network (LT-TDAN), are designed to perform individually short-term and long-term depressive behavior modeling. Then, a simple Interactive Multi-head Attention Fusion (IMHAF) strategy is employed for integrating short-term and long-term spatiotemporal features, followed by a linear fully-collected layer for depression score prediction. Experiments on two public AVEC2013 and AVEC2014 datasets show that our proposed method not only achieves highly competitive performance to state-of-the-art methods, but also has much smaller computational complexity than them on video depression detection tasks.Keywords
Funding Information
- National Natural Science Foundation of China (62276180, 61976149, 62125201, 61836002)
- Zhejiang Provincial National Science Foundation of China (LZ20F020002)
- Alibaba Group through Alibaba Research Fellow (ARF) Program
This publication has 53 references indexed in Scilit:
- Multimodal emotion recognition based on audio and text by using hybrid attention networksBiomedical Signal Processing and Control, 2023
- The role of alexithymia as a risk factor for self-harm among adolescents in depression – A systematic reviewJournal of Affective Disorders, 2022
- Machine learning-based diagnosis support system for differentiating between clinical anxiety and depression disordersJournal of Psychiatric Research, 2021
- Automatic depression recognition using CNN with attention mechanism from videosNeurocomputing, 2021
- Recognize Actions by Disentangling Components of Dynamics2018
- Deep Transformation Learning for Depression Diagnosis from Facial Images2017
- Temporal Segment Networks: Towards Good Practices for Deep Action Recognition2016
- A temporally piece-wise fisher vector approach for depression analysis2015
- Severity classification on the Hamilton depression rating scaleJournal of Affective Disorders, 2013
- The PHQ-8 as a measure of current depression in the general populationJournal of Affective Disorders, 2009