Applications of Transformer-Based Language Models for Depression Detection: A Scoping Review
Authors:
Open Access
- 20 June 2025
- journal article
- Published by Global Science Publishing Pte. Ltd. in Journal of Integrated Social Sciences and Humanities
- p. 1-8
Abstract
Depression is a substantial public health issue, with global ramifications. Transformer-Based Language Models (TBLM), with their ability to capture nuanced emotional and semantic information, are particularly well-suited for identifying depressive states from such unstructured textual data. The objective of this scoping review is to examine the usefulness of TBLM in detecting depression on text-based data. This study is based on a comprehensive search of Web of Science Core Collection, which includes studies that focused on the application of TBLM in diagnosing and classifying depression through social media texts. Our findings indicate that models such as BERT and its variants predominate in the existing literature, due to BERT’s demonstrated generalizability in natural language understanding and its capacity to capture semantic and affective features in social media texts. However, generative models like GPT are rarely exploring. Twitter was the most frequently used data source, attributed to its public accessibility, real-time content generation, and large user base. Domain-adapted models, including MentalBERT and RedditBERT, demonstrate promising capabilities through fine-tuning on mental health-related corpora, potentially enhancing the detection of context-specific linguistic cues. This study further suggests that TBLM are increasingly being incorporated into psychiatric practice, with evidence of rapid advancements and encouraging outcomes.Keywords
This publication has 25 references indexed in Scilit:
- A hybrid BERT-CPSO model for multi-class depression detection using pure hindi and hinglish multimodal data on social mediaComputers and Electrical Engineering, 2024
- Novel Transformer Based Contextualized Embedding and Probabilistic Features for Depression Detection From Social MediaIEEE Access, 2024
- Harnessing the Power of Hugging Face Transformers for Predicting Mental Health Disorders in Social NetworksIEEE Access, 2024
- An ensemble approach to detect depression from social media platform: E-CLSMultimedia Tools and Applications, 2024
- Hybrid machine learning models to detect signs of depressionMultimedia Tools and Applications, 2023
- COVID-19 pandemic and mental health consequences: Systematic review of the current evidenceBrain, Behavior, and Immunity, 2020
- Depression, negative emotionality, and self-referential language: A multi-lab, multi-measure, and multi-language-task research synthesis.Journal of Personality and Social Psychology, 2019
- Depression and cardiovascular disease: Epidemiological evidence on their linking mechanismsNeuroscience & Biobehavioral Reviews, 2017
- Natural language processing in mental health applications using non-clinical textsNatural Language Engineering, 2017
- Classification for clinical practice: How to make ICD and DSM better able to serve cliniciansInternational Review of Psychiatry, 2007