INESC-ID @ eRisk 2025: Exploring Fine-Tuned, Similarity-Based, and Prompt-Based Approaches to Depression Symptom Identification
Preprint
Abstract
In this work, we describe our team's approach to eRisk's 2025 Task 1: Search for Symptoms of Depression. Given a set of sentences and the Beck's Depression Inventory - II (BDI) questionnaire, participants were tasked with submitting up to 1,000 sentences per depression symptom in the BDI, sorted by relevance. Participant submissions were evaluated according to standard Information Retrieval (IR) metrics, including Average Precision (AP) and R-Precision (R-PREC). The provided training data, however, consisted of sentences labeled as to whether a given sentence was relevant or not w.r.t. one of BDI's symptoms. Due to this labeling limitation, we framed our development as a binary classification task for each BDI symptom, and evaluated accordingly. To that end, we split the available labeled data into training and validation sets, and explored foundation model fine-tuning, sentence similarity, Large Language Model (LLM) prompting, and ensemble techniques. The validation results revealed that fine-tuning foundation models yielded the best performance, particularly when enhanced with synthetic data to mitigate class imbalance. We also observed that the optimal approach varied by symptom. Based on these insights, we devised five independent test runs, two of which used ensemble methods. These runs achieved the highest scores in the official IR evaluation, outperforming submissions from 16 other teams.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Overview of eRisk 2024: Early Risk Prediction on the Internet2024
- Linguistic markers of depression: Insights from english-language tweets before and during the COVID-19 pandemicLanguage and Health, 2023
- Overview of eRisk 2023: Early Risk Prediction on the Internet2023
- The relationship between linguistic expression in blog content and symptoms of depression, anxiety, and suicidal thoughts: A longitudinal studyPLOS ONE, 2021
- Comorbid depression in medical diseasesNature Reviews Disease Primers, 2020
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks2019
- eRISK 2017: CLEF Lab on Early Risk Prediction on the Internet: Experimental Foundations2017
- Depression and Self-Harm Risk Assessment in Online Forums2017
- The Psychological Meaning of Words: LIWC and Computerized Text Analysis MethodsJournal of Language and Social Psychology, 2009
- No health without mental healthThe Lancet, 2007