Trial Selection Tensor Canonical Correlation Analysis (TSTCCA) for Depression Recognition With Facial Expression and Pupil Diameter
- 5 October 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (6) , 3989-4000
Abstract
Facial expressions have been widely used for depression recognition because it is intuitive and convenient to access. Pupil diameter contains rich emotional information that is already reflected in facial video streams. However, the spatiotemporal correlation between pupillary changes and facial behavior changes induced by emotional stimuli has not been explored in existing studies. This paper presents a novel multimodal fusion algorithm - Trial Selection Tensor Canonical Correlation Analysis (TSTCCA) to optimize the feature space and build a more robust depression recognition model, which innovatively combines the spatiotemporal relevance and complementarity between facial expression and pupil diameter features. TSTCCA explores the interaction between trials and obtains an effective fusion representation of two modalities from a trial subset related to depression. The experimental results show that TSTCCA achieves the highest accuracy of 78.81% with the subset of 25 trials.Keywords
Funding Information
- National Key Research and Development Program of China (2019YFA0706200)
- National Natural Science Foundation of China (62227807)
- Natural Science Foundation of Gansu Province (22JR5RA488)
- Fundamental Research Funds for the Central Universities (lzujbky-2023-16)
- Supercomputing Center of Lanzhou University
This publication has 68 references indexed in Scilit:
- Multimodal fusion diagnosis of depression and anxiety based on CNN-LSTM modelComputerized Medical Imaging and Graphics, 2022
- Content-based multiple evidence fusion on EEG and eye movements for mild depression recognitionComputer Methods and Programs in Biomedicine, 2022
- Intelligent system for depression scale estimation with facial expressions and case study in industrial intelligenceInternational Journal of Intelligent Systems, 2021
- An experimental comparison of feature selection methods on two-class biomedical datasetsComputers in Biology and Medicine, 2015
- A systematic meta-analysis of the Stroop task in depressionClinical Psychology Review, 2012
- Remission Prognosis for Cognitive Therapy for Recurrent Depression Using the Pupil: Utility and Neural CorrelatesBiological Psychiatry, 2011
- Two-dimensional canonical correlation analysis and its application in small sample size face recognitionNeural Computing and Applications, 2009
- The Epidemiology of Major Depressive DisorderJAMA, 2003
- Relations Between Two Sets of Variates1992
- Introduction to Statistical Pattern Recognition1990