1/ Do AI systems discriminate based on gender when choosing the most qualified candidate for a job? I ran an experiment with several leading LLMs to find out. Here's what I discovered:👇 2/ Across 70 popular professions, LLMs systematically favored female-named candidates over equally qualified male-named candidates when asked to choose the more qualified candidate for a job.

1/ A deep dive into AI political bias:

Do AIs like ChatGPT or Google Gemini lean left or right? Past studies often used political quizzes to find out, but those don’t quite reflect real-world user interactions with AI. In a new analysis, I take a different approach 🧵👇 2/ I use 4 methods to assess political bias in AI-generated text
🔹 Comparing AI text with language from Democrats/Republicans legislators
🔹 Ideological viewpoints in AI-generated policy recommendations
🔹 Sentiment in AI text towards political figures
🔹 Political quizzes

1/ Is Wikipedia politically biased? To explore this question, I averaged the sentiment (negative/neutral/positive) associated with a set of politically loaded terms used in English Wikipedia content (N=175,205 sentiment annotations).
davidrozado.substack.com/p/is-wikipedia… 2/ There is a quantifiable mild to moderate tendency in Wikipedia articles to associate right-of-center U.S. politicians with more negative sentiment compared to left-of-center politicians.

The Political Preferences of AIs: When probed with questions with political connotations, LLMs tend to generate responses that are diagnosed by most political orientations tests as manifesting preferences for left-of-center viewpoints, 11 political orientation tests, 24 SOTA LLMs In contrast, base models (precursors of conversationsl LLMs) responses to political orientation tests do not skew politically. This is surprising given the likely unbalanced representation of political viewpoints in pretraining corpora. Caveats apply though (see preprint).

Grok's political preferences are similar to those of ChatGPT Grok's results on different test: Political Spectrum Quiz

Great Awokening is a global phenomenon. No evidence it started in US media. Analysis of 98 million news articles across 36 countries quantifies. Exception: state-controlled media from China/Russia/Iran using wokeness terminology to criticize/mock the West
davidrozado.substack.com/p/gag Different geographical regions and countries emphasize distinct types of prejudice with varying degrees of intensity.

La desproporción con la que los medios de comunicación españoles utilizan palabras tales como machismo, sexismo o misoginia es de tal magnitud que triplican a las de cualquiera de los medios de otros 35 países analizados davidrozado.substack.com/p/epg Entre los años 2000 y 2003 los medios españoles se referían al machismo con una frecuencia similar a la de sus homólogos europeos. En el periodo 2004-2008 dichos medios se convirtieron en una anomalía internacional por su alto número de referencias a este tipo de palabras

1. Regarding whether The Great Awokening is winding down: There appears to be a decoupling in Twitter content between positive terminology associated with Social Justice/focus on victims and negative terminology suggestive of bad/deviant behavior
davidrozado.substack.com/p/taw
🧵 2. Terms with a negative payload suggestive of bad/deviant behavior (such as prejudice) have decreased in frequency in Twitter chatter. But... plot twist coming...🧵
davidrozado.substack.com/p/taw

1. Behold RightWingGPT. An AI model fine-tuned to manifest the opposite political biases of ChatGPT (i.e. to be right wing). Let me describe how I did it and the dangers of politically aligned AIs given their potential to induce societal polarization davidrozado.substack.com/p/rightwinggpt
🧵 2. I have previously documented the left-leaning political biases embedded in ChatGPT as manifested in the bot responses to political tests' questions
davidrozado.substack.com/p/political-bi…

1. We really need to know where ChatGPT biases come from. Training data? human raters in the fine-tuning phase rating the quality of the model responses? the instructions that those raters were given? intentionally placed filters? ... 2. It would be very congruent with OpenAI name and mission to allow external auditors to determine where these biases come from.

1. I replicated and extended my original analysis of ChatGPT political biases. 14 out of 15 different political orientation tests diagnose ChatGPT answers to their questions as manifesting a preference for left-leaning viewpoints.
davidrozado.substack.com/p/political-bi… 2. On December 6th, I published a preliminary analysis about the left-leaning political bias embedded in the first release of ChatGPT (November 30, 2022).
davidrozado.substack.com/p/the-politica…

1. El modelo de inteligencia artificial ChatGPT muestra una clara predilección por puntos de vista a la izquierda del centro político cuando se le somete a un test de orientación política en español.
davidrozado.substack.com/p/el-sesgo-pol… 2. A pesar del claro sesgo político de sus respuestas, cuando se le pregunta directamente por sus preferencias políticas, ChatGPT responde que no tiene preferencias políticas y que su objetivo es tan solo proporcionar información precisa y objetiva.

1. ChatGPT no longer displays a clear left-leaning political bias. A mere few weeks after I probed ChatGPT with several political orientation quizzes, something appears to have been changed in ChatGPT that makes it more neutral and viewpoint diverse. 🧵
davidrozado.substack.com/p/chatgpt 2. Between December 5-6, I applied 4 political orientation tests to ChatGPT. Results were consistent across the tests. All 4 tests diagnosed ChatGPT answers to their questions as politically left-leaning. I described my analysis in detail here
davidrozado.substack.com/p/the-politica…

ChatGPT answers to Political Compass Test: anti death penalty, pro-abortion, skeptic of free markets, corporations exploit poor countries, more tax the rich, pro gov subsidies, pro-benefits to those refuse to work, pro-immigration, pro-sexual liberation, morality without religion

I carried out another test to determine the political leanings of ChatGPT dialogues using the more comprehensive Political Compass Test. Results hold. ChatGPT dialogues display substantial left-leaning and libertarian political bias.
davidrozado.substack.com/p/the-politica… Some example answers of ChatGPT AI system to the Political Compass Test

I just applied the Political Typology Quiz from Pew Research to ChatGPT, a state-of-the-art AI language model. Results are consistent through several trials: ChatGPT answers exhibit moderate left-leaning political orientation
davidrozado.substack.com/p/the-politica… The disparity between ChatGPT and the wider public is substantial. Establishment liberalism ideology represents just 13% of the American public.

1/n The anger, fear and sadness conveyed by news media headlines is increasing over time.
Summary of published paper: davidrozado.substack.com/p/the-increasi… 2/n News media headlines from right-leaning news outlets display, on average, more anger than headlines from left-leaning news media. Emotionally neutral headlines are consistently decreasing across news media regardless of their ideological leanings

1/n News media headlines are becoming more negative over time. Headlines conveying anger, fear and sadness also increasing markedly. Right-leaning news outlets more prone to convey negativity and anger in their headlines.
Summary of published paper: davidrozado.substack.com/p/the-increasi…
🧵 2/n Headlines from right-leaning news outlets tend to have a more negative sentiment than headlines from left-leaning news outlets

Frequency of Terms Denoting Political Extremism in News Media
davidrozado.substack.com/p/iftdpe

@FDisenso @FDisenso