Natural language analyzed with AI-based transformers predict traditional subjective well-being measures approaching the theoretical upper limits in accuracy
Open Access
- 10 March 2022
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 12 (1) , 1-9
Abstract
We show that using a recent break-through in artificial intelligence –transformers–, psychological assessments from text-responses can approach theoretical upper limits in accuracy, converging with standard psychological rating scales. Text-responses use people's primary form of communication –natural language– and have been suggested as a more ecologically-valid response format than closed-ended rating scales that dominate social science. However, previous language analysis techniques left a gap between how accurately they converged with standard rating scales and how well ratings scales converge with themselves – a theoretical upper-limit in accuracy. Most recently, AI-based language analysis has gone through a transformation as nearly all of its applications, from Web search to personalized assistants (e.g., Alexa and Siri), have shown unprecedented improvement by using transformers. We evaluate transformers for estimating psychological well-being from questionnaire text- and descriptive word-responses, and find accuracies converging with rating scales that approach the theoretical upper limits (Pearson r = 0.85, p < 0.001, N = 608; in line with most metrics of rating scale reliability). These findings suggest an avenue for modernizing the ubiquitous questionnaire and ultimately opening doors to a greater understanding of the human condition.Keywords
Funding Information
- Vetenskapsrådet (2019-06305)
- VINNOVA (2018-02007, 2018-02007)
- Familjen Kamprads Stiftelse (20180281, 20180281)
- National Institute on Alcohol Abuse and Alcoholism (R01 AA028032)
- Lund University
This publication has 35 references indexed in Scilit:
- Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary ApproachPLOS ONE, 2013
- Theory and Validity of Life Satisfaction ScalesSocial Indicators Research, 2012
- The Psychological Meaning of Words: LIWC and Computerized Text Analysis MethodsJournal of Language and Social Psychology, 2009
- Review of the Satisfaction With Life Scale2009
- The Philosophy of Harmony in Classical ConfucianismPhilosophy Compass, 2008
- A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge.Psychological Review, 1997
- The Correction for AttenuationEducational and Psychological Measurement, 1996
- Bootstrap Prediction Intervals for RegressionJournal of the American Statistical Association, 1985
- Subjective well-being.Psychological Bulletin, 1984
- Ridge Regression: Biased Estimation for Nonorthogonal ProblemsTechnometrics, 1970