A Multimodal Deep Learning Framework for Depression Detection Using Vision Transformers and Large Language Models
Authors:
- 13 June 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 417-420
Abstract
This study proposes a novel multimodal deep learning framework for depression detection, integrating visual, audio, and textual data. Using OpenFace and Librosa for feature extraction, the system employs Vision Transformers (ViTs) to model cross-modal dependencies, while a large language model (Qwen3-32B) analyzes transcribed speech for interpretable symptom reasoning. Outputs from both modalities are fused via a confidence-based mechanism. Experimental validation on the LMVD dataset demonstrates superior performance, overcoming the limitations of unimodal approaches.Keywords
This publication has 4 references indexed in Scilit:
- DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptation2025
- A Survey on Evaluation of Large Language ModelsACM Transactions on Intelligent Systems and Technology, 2024
- A Comparative Analysis of Early and Late Fusion for the Multimodal Two-Class ProblemIEEE Access, 2023
- Machine Learning-Based Behavioral Diagnostic Tools for Depression: Advances, Challenges, and Future DirectionsJournal of Personalized Medicine, 2021