Revolutionizing Mental Health Sentiment Analysis With BERT-Fuse: A Hybrid Deep Learning Model
Open Access
- 9 May 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 85428-85446
Abstract
Mental health disorders profoundly affect intellectual ability, emotional resilience, and social stability, leading to lasting effects on both individuals and society. Increasing global concern over mental health calls for innovative solutions, with sentiment analysis emerging as a promising approach for early diagnosis and intervention. Traditional methods are limited by the complexity of mental health-related texts, which contain specialized terminology and domain-specific nuances. To address these challenges, we propose BERT-Fuse, a hybrid deep learning model combining traditional feature extraction techniques (TF-IDF and Bag-of-Words) with advanced neural architectures, including Bidirectional Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Transformer blocks, and Convolutional Neural Networks (CNN). We evaluated three word embedding techniques-BERT, GloVe, and FastText within BERT-Fuse and found that BERT embeddings achieve the highest performance in terms of accuracy, precision, recall, and F1-score. The model was trained on a balanced dataset of text samples representing seven mental health conditions. An ablation study highlights the contributions of key model components to its performance. BERT-Fuse shows promise as a scalable, high-precision system for mental health detection with an impressive average test accuracy of 97.05%. Our model bridges the gap between deep learning and psychological text analysis, paving the way for future mental health assessment tools that are more intelligent and easily accessible.Keywords
Funding Information
- Bangladesh University of Business and Technology (BUBT), Dhaka 1216, Bangladesh
This publication has 31 references indexed in Scilit:
- Multi task opinion enhanced hybrid BERT model for mental health analysisScientific Reports, 2025
- Sustainable Promotion of Mental Health and Prevention of Mental Health Disorders Across the World2024
- DepressionEmo: A novel dataset for multilabel classification of depression emotionsJournal of Affective Disorders, 2024
- An attention-based hybrid architecture with explainability for depressive social media text detection in BanglaExpert Systems with Applications, 2022
- An hybrid deep learning approach for depression prediction from user tweets using feature-rich CNN and bi-directional LSTMMultimedia Tools and Applications, 2022
- RoBERTa-LSTM: A Hybrid Model for Sentiment Analysis With Transformer and Recurrent Neural NetworkIEEE Access, 2022
- A review on sentiment analysis and emotion detection from textSocial Network Analysis and Mining, 2021
- Deep learning for prediction of depressive symptoms in a large textual datasetNeural Computing and Applications, 2021
- Identifying Emotion Labels From Psychiatric Social Texts Using a Bi-Directional LSTM-CNN ModelIEEE Access, 2020
- Global patterns of workplace productivity for people with depression: absenteeism and presenteeism costs across eight diverse countriesSocial Psychiatry and Psychiatric Epidemiology, 2016