David Rozado

I report here a comprehensive analysis about the political preferences embedded in Large Language Models (LLMs). Namely, I administer 11 political orientation tests, designed to identify the political preferences of the test taker, to 24 state-of-the-art conversational LLMs, both closed and open source. When probed with questions/statements with political connotations, most conversational LLMs tend to generate responses that are diagnosed by most political test instruments as manifesting preferences for left-of-center viewpoints. This does not appear to be the case for five additional base (i.e. foundation) models upon which LLMs optimized for conversation with humans are built. However, the weak performance of the base models at coherently answering the tests’ questions makes this subset of results inconclusive. Finally, I demonstrate that LLMs can be steered towards specific locations in the political spectrum through Supervised Fine-Tuning (SFT) with only modest amounts of politically aligned data, suggesting SFT’s potential to embed political orientation in LLMs. With LLMs beginning to partially displace traditional information sources like search engines and Wikipedia, the societal implications of political biases embedded in LLMs are substantial.

Recent advancements in Large Language Models (LLMs) suggest imminent commercial applications of such AI systems where they will serve as gateways to interact with technology and the accumulated body of human knowledge. The possibility of political biases embedded in these models raises concerns about their potential misusage. In this work, we report the results of administering 15 different political orientation tests (14 in English, 1 in Spanish) to a state-of-the-art Large Language Model, the popular ChatGPT from OpenAI. The results are consistent across tests; 14 of the 15 instruments diagnose ChatGPT answers to their questions as manifesting a preference for left-leaning viewpoints. When asked explicitly about its political preferences, ChatGPT often claims to hold no political opinions and to just strive to provide factual and neutral information. It is desirable that public facing artificial intelligence systems provide accurate and factual information about empirically verifiable issues, but such systems should strive for political neutrality on largely normative questions for which there is no straightforward way to empirically validate a viewpoint. Thus, ethical AI systems should present users with balanced arguments on the issue at hand and avoid claiming neutrality while displaying clear signs of political bias in their content.

Few topics are currently as polarizing as the appropriate limits, and perceived dangers, of free speech on university campuses. A side effect of this polarized environment is that students themselves may be reluctant to speak publicly on politically sensitive topics. Indeed, recent surveys by the Heterodox Academy (HxA) revealed that a majority of American university students thought their campus was not conducive to the free expression of ideas, and a substantial minority were personally reluctant to discuss “hot topics” like politics or sexual orientation in class. To see whether these results are uniquely American phenomena, we reran the HxA’s survey on 791 students, recruited via advertisements, enrolled in New Zealand universities. As in the original survey, participants answered questions, administered online, about their comfort sharing their opinions on issues related to gender, politics, religion, and sexual orientation, as well as their estimates of other groups’ discomfort. Despite significant sociopolitical differences between the two countries, our results, generally speaking, bear out those in the United States. In both countries, politics elicited the most reluctance to speak, followed by religion, and then gender and sexual orientation (which were equivalent), and New Zealanders were more reluctant than Americans to speak on the latter two topics. Other similarities and differences between the two data sets are discussed, but it is clear that chilled campus speech is not confined to the United States.

This work describes a chronological (2000–2019) analysis of sentiment and emotion in 23 million headlines from 47 news media outlets popular in the United States. We use Transformer language models fine-tuned for detection of sentiment (positive, negative) and Ekman’s six basic emotions (anger, disgust, fear, joy, sadness, surprise) plus neutral to automatically label the headlines. Results show an increase of sentiment negativity in headlines across written news media since the year 2000. Headlines from right-leaning news media have been, on average, consistently more negative than headlines from left-leaning outlets over the entire studied time period. The chronological analysis of headlines emotionality shows a growing proportion of headlines denoting anger, fear, disgust and sadness and a decrease in the prevalence of emotionally neutral headlines across the studied outlets over the 2000–2019 interval. The prevalence of headlines denoting anger appears to be higher, on average, in right-leaning news outlets than in left-leaning news media.

The term political extremism is commonly used to refer to political attitudes considered to be outside the ideological mainstream. This study leverages computational content analysis of big data to longitudinally examine (1970–2019) the prevalence of terms denoting far-right and far-left political extremism in more than 30 million written news and opinion articles from 54 news media outlets popular in the United States and the United Kingdom. We find that the usage of terms denoting right and left political extremism has been increasing across news media outlets in both countries. This trend is particularly stark for far-right-denoting terms, which have been growing in prevalence since at least 2008. Most U.S. and U.K. news media outlets tend to use far-right-denoting terms substantially more often than they use far-left-denoting terms. The rising prevalence in news media of terms denoting political extremism is strongly correlated with similar growing usage of terms denoting prejudice and social justice discourse.

Previous scholarly literature has documented a pronounced increase in the prevalence of prejudice-denoting terms in American news media content. Some have referred to this shift in journalistic discourse and related public opinion trends signaling increasing perceptions of prejudice severity in U.S. society as The Great Awokening. This work analyzes whether the increasing prevalence of prejudice themes in American news media outlets has been replicated in the news media ecosystem of a Spanish-speaking country. Thus, we computationally analyzed the prevalence of words denoting prejudice in five million news and opinion articles written between 1976 and 2019 and published in three of the most widely read newspapers in Spain: El País, El Mundo and ABC. We report that within the studied time period, the frequency of terms that denote specific prejudice types related to gender, ethnicity, sexuality and religious orientation has also substantially increased across the analyzed Spanish news media outlets. There are, however, some notable distinctions in the long-term usage dynamics of prejudice-denoting terms between the leading Spanish newspaper of record, El País, and its U.S. counterpart, The New York Times.

The COVID-19 pandemic has been one of the most disruptive and painful phenomena of the last few decades. As of July 2021, the origins of the SARS-CoV-2 virus that caused the outbreak remain a mystery. This work analyzes the prevalence in news media articles of two popular hypotheses about SARS-CoV-2 virus origins: the natural emergence and the lab-leak hypotheses. Our results show that for most of 2020, the natural emergence hypothesis was favored in news media content while the lab-leak hypothesis was largely absent. However, something changed around May 2021 that caused the prevalence of the lab-leak hypothesis to substantially increase in news media discourse. This shift has not been uniformed across media organizations but instead has manifested itself more acutely in some outlets than others. Our structural break analysis of daily news media usage of terms related to the laboratory escape hypothesis provides hints about potential sources for this sudden shift in the prevalence of the lab-leak hypothesis in prestigious news media.