Arturo Montejo-Ráez

This paper describes the participation of the SINAI-UJA team in the eRisk@CLEF 2025 lab. Specifically, we addressed two of the proposed tasks: (i) Task 2: Contextualized Early Detection of Depression, and (ii) Pilot Task: Conversational Depression Detection via LLMs. Our approach for Task 2 combines an extensive preprocessing pipeline with the use of several transformer-based models, such as RoBERTa Base or MentalRoBERTA Large, to capture the contextual and sequential nature of multi-user conversations. For the Pilot Task, we designed a set of conversational strategies to interact with LLM-powered personas, focusing on maximizing information gain within a limited number of dialogue turns. In Task 2, our system ranked 8th out of 12 participating teams based on F1 score. However, a deeper analysis revealed that our models were among the fastest in issuing early predictions, which is a critical factor in real-world deployment scenarios. This highlights the trade-off between early detection and classification accuracy, suggesting potential avenues for optimizing both jointly in future work. In the Pilot Task, we achieved 1st place out of 5 teams, obtaining the best overall performance across all evaluation metrics: DCHR, ADODL and ASHR. Our success in this task demonstrates the effectiveness of structured conversational design when combined with powerful language models, reinforcing the feasibility of deploying LLMs in sensitive mental health assessment contexts.

This paper describes the participation of the SINAI team in the eRisk@CLEF lab. Specifically, two of the proposed tasks have been addressed: i) Task 1 on the early detection of signs of pathological gambling, and ii) Task 3 on measuring the severity of the signs of eating disorders. The approach presented in Task 1 is based on the use of sentence embeddings from Transformers with features related to volumetry, lexical diversity, complexity metrics, and emotion-related scores, while the approach for Task 3 is based on text similarity estimation using contextualized word embeddings from Transformers. In Task 1, our team has been ranked in second position, with an F1 score of 0.808, out of 41 participant submissions. In Task 3, our team also placed second out of a total of 3 participating teams.

This paper describes the participation of the SINAI team in the eRisk@CLEF lab. Specifically, one of the proposed tasks has been addressed: Task 2 on the early detection of signs of pathological gambling. The approach presented in Task 2 is based on pre-trained models from Transformers architecture with comprehensive preprocessing data and data balancing techniques. Moreover, we integrate Long-short Term Memory (LSTM) architecture with automodels from Transformers. In this Task, our team has been ranked in seventh position, with an F1 score of 0.126, out of 49 participant submissions and achieves the highest values in recall metrics and metrics related to early detection.

Evaluar la complejidad léxica en documentos utilizando los modelos Generative Pre-trained Transformer (GPT-4, GPT-4o y GPT-4 Turbo) permitió analizar su impacto en la comprensión del lenguaje específicamente en textos estatales ecuatorianos en español. Se aplicó la técnica de few-shot learning en todos los modelos, tomando el conjunto de datos GovAIEc. El enfoque aplicado de la investigación es cuantitativo y descriptivo, con un diseño cuasi experimental. Los modelos se evalúan mediante métricas del error común: MAE, MSE, RMSE. El objetivo de esta investigación es evaluar el rendimiento del modelo GPT-4 y sus modelos sucesores en la predicción de la complejidad léxica en textos públicos ecuatorianos mediante Few-Shot Learning. Los resultados indican que el modelo GPT-4 obtiene un mayor rendimiento frente a sus sucesores con un MAE = 0.2464, mientras que GPT-4o presenta un MAE = 0.3889, y finalmente los resultados aplicando el modelo GPT-4 Turbo dieron un MAE = 0.2540.

En el ámbito de las instituciones públicas, la accesibilidad y comprensión de los documentos gubernamentales suelen verse afectadas por la complejidad léxica y el uso de lenguaje técnico especializado. Este problema impacta tanto a ciudadanos como a funcionarios, generando malentendidos que pueden reducir la transparencia y dificultar la participación ciudadana. En este estudio, se analiza el desempeño del modelo RoBERTa-large-bne, basado en la arquitectura Transformer, en la predicción de la complejidad léxica en textos estatales ecuatorianos. Para ello, se implementó un ajuste fino del modelo con el fin de optimizar su rendimiento en esta tarea específica. Se llevó a cabo una evaluación comparativa con otros modelos de lenguaje pre-entrenados aplicados en el corpus GovAIEc, cuyos textos corresponden a instituciones estatales del Ecuador. Los resultados obtenidos buscan sentar las bases para el desarrollo de herramientas que faciliten la simplificación de documentos públicos, mejorando su accesibilidad y promoviendo una interacción más eficiente entre la ciudadanía y las instituciones gubernamentales.

In this work, we present a novel approach to lexical complexity prediction (LCP) that combines diverse linguistic features with encodings from deep neural networks. We explore the integration of 23 handcrafted linguistic features with embeddings from two well-known language models: BERT and XLM-RoBERTa. Our method concatenates these features before inputting them into various machine learning algorithms, including SVM, Random Forest, and fine-tuned transformer models. We evaluate our approach using two datasets: CompLex for English (a high-resource language) and CLexIS2 for Spanish (a relatively low-resource language in ), allowing us to study performance issues from a cross-lingual perspective. Our experiments involve different combinations of linguistic features with encodings from pretrained deep learning models, testing both token-based and sequence-related encodings. The results demonstrate the effectiveness of our hybrid approach. For the English CompLex corpus, our best model achieved a mean absolute error (MAE) of 0.0683, representing a 29.2% improvement over using linguistic features alone (MAE 0.0965). On the Spanish CLexIS2 corpus, we achieved an MAE of 0.1323, a 19.4. These findings show that handcrafted linguistic features play a fundamental role in achieving higher performance, particularly when combined with deep learning approaches. Our work suggests that hybrid approaches should be considered over full end-to-end solutions for LCP tasks, especially in multilingual contexts.