Oscar Nils Erik Kjell

Humans use language to communicate psychological experiences. Advancements in Natural LanguageProcessing have enabled researchers to quantitatively study and assess psychological states andbehaviors through language (e.g., well-being and mental health). Insights from language analyses areoften best conveyed through visualizations. Data-driven visualizations of how language statisticallyrelates to psychological dimensions can help interpret assessment scores, uncover meaningful patterns,and understand the intricate relationships between language use and mental states. Visualizations canbe used to differentiate between related constructs (e.g., How does language related to depression versusanxiety differ?), understand the validity of assessment tools (e.g., Does the language related to highdepression severity match with depression theory?), and explain language-based assessments (e.g.,Which language is indicative of higher depression severity in a language-based assessment?).This tutorial demonstrates how to transform language data into visual insights using the text and topicspackages in R. We provide practical guidance on creating four types of visualizations based on differentlanguage analysis techniques, examining how language use patterns are statistically associated with apsychological construct. We visualize 1) individual words and phrases (n-grams), 2) topics (wordclusters), 3) words in the word embedding space, and 4) relevant language examples. These differentvisualizations offer complementary perspectives, visualizing linguistic elements from individual wordsto overarching topics and text examples. The underlying methods range from correlations betweenindividual words to more advanced techniques, including topic modeling and Large Language Models.By making these methods accessible, this tutorial aims to empower researchers to use languagevisualizations for meaningful, data-driven insights.

Cognitive theories are among the leading conceptualizations of posttraumatic stress disorder (PTSD). The core perspective shared by these theories is that distorted thinking (e.g., “no one can be trusted”) is involved in maintaining symptoms. However, the proximal mechanisms linking cognition and PTSD symptoms remains unclear due to two major methodological limitations: 1) lack of research on cognition-symptom dynamics at the daily level, and 2) reliance on self-reported thought distortions, which is a method ill-suited for capturing biases that people often do not have insight into or are outside of their awareness. The present study helps address these limitations in a sample of trauma-exposed patients (N=510) who completed daily video diaries and symptom ratings for 90 days (n=30,958 days). We used two natural language processing techniques to assess cognitive distortions from the diary transcripts: a lexica-based and large language model (LLM) approach. We then established the day-to-day sequencing of cognitive distortion and symptom fluctuations using dynamic structural equation models. Across the lexical and LLM models, distorted thinking did not predict worsening symptoms (bs=-.02–.01). Instead, worsening symptoms predicted more distorted thinking (bs=.03–.07). Also, symptoms and cognitive distortions tended to fluctuate together (bs=.08–.41). The current findings suggest that cognitive distortions do not precede PTSD symptoms as many theories have assumed, but may instead be a consequence or correlate of them. If replicated, our results could help refine cognitive and related models of PTSD.

While language is central to human communication, psychological science has relied on self-reported rating scales for decades because it was difficult to quantify language effectively. Large language models (LLMs) have changed that in recent years. Here, we used LLMs with machine learning to compare traditional ratings against language-based responses (short essays and descriptive words) of experienced well-being over 12 weeks from 764 individuals from an overburdened population highly exposed to emotional experience, U.S. hospitality workers. While accuracies were comparable by internal criterion (across 9 affective traits; words r = .53 vs. .51 for ratings), both language formats doubled the ratings’ accuracy (up to r = .18 and .24 for words and essays, respectively) across 24 external criteria: traits, demographics, and behavior. Beyond improved accuracy, language responses offered psychological insights: words provided concise emotional nuances, whereas essays encompassed broad life contexts. Though each language-based format differed in strengths and weaknesses, both demonstrated greater information-richness than traditional rating scales.

Humans use language to communicate psychological experiences. Advancements in natural language processing have enabled researchers to quantitatively study and assess psychological states and behaviors through language. Insights from these language analyses are often best conveyed through visualizations. Data-driven visualizations of how language statistically relates to psychological dimensions can help interpret assessment scores, uncover meaningful patterns, and understand the intricate relationships between language use and mental states. Visualizations can be used to differentiate between related constructs (e.g., How does language related to depression versus anxiety differ?), understand the validity of assessment tools (e.g., Does the language related to high depression severity match with depression theory?), and explain language-based assessments (e.g., Which language is indicative of higher depression severity in a language-based assessment?). In this tutorial, we demonstrate how to transform language data into visual insights using the text and topics packages in R. We provide practical guidance on creating four types of visualizations based on different language-analysis techniques, examining how language-use patterns are statistically associated with psychological constructs. We visualize (a) individual words and phrases (n-grams), (b) topics (word clusters), (c) words in the word-embedding space, and (d) relevant language examples. The different visualizations offer complementary perspectives, visualizing linguistic elements from individual words to overarching topics and text examples. The underlying methods range from correlations between individual words to more advanced techniques, including topic modeling and large language models. By making these methods accessible, we aim to empower researchers to use language visualizations for meaningful, data-driven insights.

Self-reported rating scales have been central to social science for decades, even though language is our primary form of communication. We used language analysis with machine learning to compare self-reported ratings with language-based responses of experienced well-being (i.e., daily emotions) linked with human traits, states, and behaviors. In a sample of 764 U.S. service workers who reported their feelings daily for up to 12 weeks, responses were provided in 1) numerical ratings, 2) short essays, and 3) descriptive words. When predicting nine self-rated affective traits, essays were slightly less accurate than the ratings (r = .49 v .55, p < .001), while the word-based responses slightly outperformed the ratings (r = .53 vs .51, p < .001). For external predictive validity across 24 traits, demographics, and behaviors, both language formats doubled the ratings’ accuracy, with words reaching an average of r = .18 and essays r = .24 (p < .001). Beyond their greater accuracy, both language formats offered rich psychological insights, with words focusing on emotions and essays encompassing broader life domains. While both language-based formats showed advantages and disadvantages, each demonstrated greater information-richness than traditional rating scales.

Background: Machine learning and natural language processing have demonstrated significant potential for mental health assessment: describing your mental health in your own words can offer a more ecologically valid approach than traditional rating scales. However, most models focus on specific diagnoses, conditions, or symptoms, which may prematurely assign labels and potentially reinforce stigma in the context of early-stage mental health screening. Objective: This study develops a language-based assessment model that assesses the need for mental health support based on probed natural language and validates it against best-estimate assessments from multiple experienced psychotherapists. Methods: We analyzed an enriched online sample (n=600 for development and n=212 for validation), in which about half reported experiencing internalizing symptoms (depression or anxiety). Participants described their mental health using open-ended responses regarding (1) mental health, (2) suicidal thoughts, (3) medical history, and (4) depression. The responses were converted into contextual word embeddings using a large language model and entered as predictors in a ridge regression using nested cross-validation. Two to three experienced psychotherapists assessed each participant’s need for mental health support on a scale from 1 (no support needed) to 5 (potential crisis). Their assessments were based on longitudinal clinical data (natural language, validated scales, clinical interview, sociodemographics, and clinical history) and were averaged into a best-estimate assessment for model validation. We used the Sequential Evaluation With Model Preregistration framework, which separates model development from validation in a held-out set to support robust estimations and generalizability. Results: The language-based assessments closely aligned with the best-estimate assessments ( r =0.82) and showed strong correlations with established clinical rating scales for depression (Patient Health Questionnaire-9), anxiety (Generalized Anxiety Disorder 7-Item Scale), stress (Perceived Stress Scale 10), and suicidality (Inventory of Depression and Anxiety Symptoms; r =0.62-0.77). Language-based visualizations of topics and word embeddings showed that low need for support assessments was associated with mentioning well-being and good health, while high assessments were related to depression, anxiety, and suicidality. Conclusions: This study demonstrates that natural language responses analyzed through large language models and machine learning can be used to assess individuals’ need for mental health support in close alignment with best-estimate assessments from experienced psychotherapists. Using less than 5 minutes of respondent time, this approach offers a practical tool for early-stage mental health screening in both clinical and self-guided screening contexts.

Large language models (LLMs) enable increasingly accurate language-based assessments (LBAs) of psychopathology. As these tools are more widely deployed in real-world settings, a critical question emerges: Do these tools work equally well for everyone? We examined whether probed LBAs, in which individuals respond to open-ended mental health questions in their own words, exhibit systematic biases that could undermine their clinical utility. We collected open-ended mental health descriptions from 1,260 participants alongside validated depression (PHQ-9) and anxiety (GAD-7) scores. We converted these descriptions to numeric representations using LLM-generated embeddings and applied ridge regression to predict symptom severity, achieving strong predictive accuracy (r > .60). We then examined whether assessment differences were systematically associated with sociodemographic factors (gender, age, education, employment, income) or linguistic characteristics. Our analysis revealed limited evidence of demographic or linguistic bias: most variables showed minimal association with assessment errors (i.e., the absolute differences between the LBAs and self reports). However, higher depression and anxiety severity were associated with larger absolute assessment differences (standardized β = .193, p = .038; β = .252, p = .003). We explored mitigation strategies, finding that downsampling training data eliminated the bias for depression and anxiety. Our findings indicate that probed language-based assessments exhibit limited bias overall, particularly regarding sociodemographic equity. However, the observed severity-related pattern highlights a fundamental challenge.

Language-based assessments (LBAs) have shown strong convergent and criterion validity cross-sectionally and longitudinally. However, no study has tested the ability of LBAs of affect to capture experiment-induced changes (i.e., causal validity). In this randomised mixed-design experiment, participants (N=892) completed a Mood Induction Procedure comprising different conditions (church, shopping mall, and park) using physical locations (N=153) and online video clips (N=739). Participants reported their pre- and post-induction affect through open-ended responses and ratings (i.e., the Positive and Negative Affect Schedule; PANAS). Using large language models and machine learning, we compared how accurately PANAS and language responses classified the experimental conditions. The LBA derived from the language responses exhibited significantly higher accuracy compared to the PANAS scores in classifying the conditions in both the training set (cross-validated AUC=.74 vs. .63), the online holdout set (AUC = .76 vs. .70), and the offline holdout set (AUC = .67 vs. .53). Apart from showing higher predictive accuracy, we assessed LBAs of specific affective dimensions (valence, arousal, and basic emotions). These analyses revealed that positive valence was most strongly expressed in the park condition, while anger was particularly elevated in the mall condition. Further, we visualized the language differences across conditions, showing more calm and peaceful expressions in the park condition and more tired and bored expressions in the mall condition. These findings support the causal validity of language-based affect assessments and its ability to provide insights into affective states. We discuss the potential of using LBA in experimental psychological research.

Accurate psychological evaluation is crucial for early detection, intervention and evaluation of mental health disorders. The validity and reliability of psychological evaluation methods have been systematically researched for many decades. However, assessment methods used by mental health professionals in clinical settings have not consistently been empirically documented. This study examines the extent to which standardized assessment methods are used in clinical settings, and clinicians’ attitudes toward these practices. Clinicians ( N = 585) from the U.S. (25%), the U.K. (28%), the Netherlands (18%), Sweden (22%), and other countries (7%) completed an online survey reporting their depression assessment methods. Clinicians reported using many different data collection methods (e.g., unstructured clinical interviews [83%], rating scales [83%]) in various combinations. A majority (80%) primarily employed clinical data combination methods, as opposed to statistical data combination methods (11%). Although they were confident in their assessments, the time to carry out assessments varied widely ( M = 87 [ SD = 108] minutes). 77% agreed on the importance of standardized, valid, and reliable assessments, and 43% reported interest in receiving AI decision-support. We did not find many significant cross-cultural differences. A majority of clinicians have not fully adopted standardized assessment methods. Predominant reliance on clinical judgment suggests a disconnect between best practices supported by research and everyday clinical practice.