ContextVecNet: A Context-Driven Multimodal Learning Framework for Depression Detection
- 27 August 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. PP (21682194) , 1-12
Abstract
Depression is a major global health issue, and social media offers a rich source of data for early detection. However, existing multimodal approaches often fail to effectively capture the contextual relationships between text and images and tend to ignore temporal information, thereby limiting their predictive accuracy and reliability. To address these limitations, we propose ContextVecNet, a multimodal deep learning framework. It is built on a CLIP-based architecture enhanced by learnable context vectors, which are integrated into the text and image encoding branches. This design allows the model to dynamically learn and adapt to the specific visual and linguistic markers of depression present in social media data. The architecture further incorporates a cross-modal transformer with time-aware embeddings, which jointly models temporal dynamics and cross-modal interactions across user posts. Evaluated on a publicly available multimodal Twitter dataset, ContextVecNet achieves state-of-the-art performance, with an AUC of 0.9922 and an F1-score of 0.9619, significantly outperforming existing methods. An ablation study confirms that freezing the context vectors results in notable performance degradation, emphasizing their importance in depression detection.Keywords
This publication has 20 references indexed in Scilit:
- Depression Detection Using Digital Traces on Social Media: A Knowledge-aware Deep Learning ApproachJournal of Management Information Systems, 2024
- An attention-based CNN-BiLSTM model for depression detection on social media textExpert Systems with Applications, 2024
- XAI Transformer based Approach for Interpreting Depressed and Suicidal User Behavior on Online Social NetworksCognitive Systems Research, 2023
- Multitask learning for recognizing stress and depression in social mediaOnline Social Networks and Media, 2023
- MaPLe: Multi-modal Prompt Learning2023
- It’s Just a Matter of Time: Detecting Depression with Time-Enriched Multimodal Transformers2023
- MHA: a multimodal hierarchical attention model for depression detection in social mediaHealth Information Science and Systems, 2023
- Online social network individual depression detection using a multitask heterogenous modality fusion approachInformation Sciences, 2022
- Multimodal time-aware attention networks for depression detectionJournal of Intelligent Information Systems, 2022
- Depression detection from social network data using machine learning techniquesHealth Information Science and Systems, 2018