MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models

Authors:
Top Cited Papers
Open Access
Publisher website
Google Scholar
Abstract
As an integral part of people's daily lives, social media is becoming a rich source for automatic mental health analysis. As traditional discriminative methods bear poor generalization ability and low interpretability, the recent large language models (LLMs) have been explored for interpretable mental health analysis on social media, which aims to provide detailed explanations along with predictions in zero-shot or few-shot settings. The results show that LLMs still achieve unsatisfactory classification performance in a zero-shot/few-shot manner, which further significantly affects the quality of the generated explanations. Domain-specific finetuning is an effective solution, but faces two critical challenges: 1) lack of high-quality training data. 2) no open-source foundation LLMs. To alleviate these problems, we formally model interpretable mental health analysis as a text generation task, and build the first multi-task and multi-source interpretable mental health instruction (IMHI) dataset with 105K data samples to support LLM instruction tuning and evaluation. The raw social media data are collected from 10 existing sources covering 8 mental health analysis tasks. We prompt ChatGPT with expert-designed few-shot prompts to obtain explanations. To ensure the reliability of the explanations, we perform strict automatic and human evaluations on the correctness, consistency, and quality of generated data. Based on the IMHI dataset and LLaMA2 foundation models, we train MentaLLaMA, the first open-source instruction-following LLM series for interpretable mental health analysis on social media. We evaluate MentaLLaMA and other advanced methods on the IMHI benchmark, the first holistic evaluation benchmark for interpretable mental health analysis. The results show that MentaLLaMA approaches state-of-the-art discriminative methods in correctness and generates human-level explanations. MentaLLaMA models also show strong generalizability to unseen tasks. The project is available at https://github.com/SteveKGYang/MentaLLaMA.
Funding Information
  • The University of Manchester
  • New Energy and Industrial Technology Development Organization (JPNP20006)
  • Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology

This publication has 25 references indexed in Scilit: