Digital Commons for Public Health: Social Media Language Models Outperform Surveys in Estimating Population Well-Being

Authors:
Preprint
Publisher website
Google Scholar
Abstract
Measuring the subjective well-being of societies is important in its own right and as a determinant of health outcomes. Traditionally, self-report surveys such as Gallup’s daily poll have tracked well-being, but they are expensive and provide limited coverage of communities. Since 2010, research has shown that social media (e.g., Twitter/X) offers a cost-effective alternative. Yet, through ownership and policy changes, access to these “digital commons” is increasingly restricted. What are we losing without access for researchers? We document that well-being estimates derived from archival geolocated Twitter language likely yielded the most valid indicators of US population well-being available. We compare demographically post-stratified estimates for 1,208 US counties (~89% of the population) derived from 1.53 billion posts by 5.25 million users from 2009-2015 with Gallup estimates based on 1.9 million survey responses (2009-2016). Twitter-based estimates were more predictive of external health and economic indicators and met a wide variety of validity criteria, including a convergent correlation of r = .70 with Gallup, high test-retest stability, linguistic face validity, and generalizability across US cities, states, and to the UK. We further show that, even without Gallup data, Twitter/X data sufficed to measure well-being using an independent language model trained on N = 9,419 Twitter users, which again outperformed Gallup in predicting external variables. These findings establish that social media can capture population well-being more robustly and with better coverage than even the largest survey efforts, and that, like other public health datasets, researcher access to these data should be protected.