Imagine if the whole field of primatology were focused on figuring out which primates were politically progressive.
E.g. whenever chimpanzees fought, researchers would try to map their conflict onto human political divides.
This is, alas, roughly analogous to current AI safety.
To be more direct, the analogy is that in both cases researchers are too focused on proving a point to develop a proper scientific understanding.
Tweet loosely inspired by the QT, but this is endemic throughout the field. The Alignment Faking paper is another prominent case.
By contrast, the US and China are *not* in Molochian competition. They could burn their values in many ways to be more competitive, and they mostly don't.
Indeed, the US in particular could become much more competitive without sacrificing *anything*. That's not very Molochian.
Mar 16 • 4 tweets • 2 min read
My sense is that trying to “automate alignment research” means doing roughly the same thing as automating capabilities research.
Except that you’re taking superintelligence more seriously, and therefore you’ll be much more effective.
A better strategy is to figure out the most promising path towards you personally making a breakthrough in our understanding of neural nets/learning/intelligence.
Then use AI whenever it seems like it’d help you towards this specific goal.
Designing policies for existing governments is too shallow an intervention to be robust to AGI.
Long-term AGI governance requires rethinking what nation-states even are.
The only serious attempt I’ve seen to intervene at that level was @Dominic2306’s Brexit campaign.
Brexit was a big win for national sovereignty—and it was driven by principled, courageous political philosophy.
Idk how explicitly Dom planned for Brexit to improve AI governance. But he was early to AI safety, so I expect it factored into his worldview.
Be more like Dom.
Feb 24 • 6 tweets • 2 min read
Challenge: “AI safety” people stop homing in with laser precision on the best ways to speed up AI.
This is primarily a problem with the EA-affiliated side of AI safety. Unfortunately, that’s most of the field by now.
EAs don’t have memetic defenses against conflating “do the most good” with “gain the most power” (or sometimes just “be in the room where it happens”).
Sep 15, 2025 • 5 tweets • 1 min read
Once you notice how often “beautiful” is used as a default compliment for women, it’s hard to unsee.
Instead of “my gorgeous wife”, may I suggest describing her as courageous, sincere, virtuous, earnest, patient, determined, or (as a generic fallback) loving.
These don’t roll off the tongue as easily, but partly that’s because we’re not used to them.
The other problem is that these words are too informative: saying them actually conveys an opinion, which is scary—unlike “beautiful” (or that most insipid of compliments, “lovely”).
Aug 31, 2025 • 4 tweets • 2 min read
Great thread. I’d even go further—I think we’re still recovering from Von Neumann’s biggest mistakes:
1. Implicitly basing game theory on causal decision theory 2. Founding utility theory on the independence axiom 3. Advocating for nuking the USSR as soon as possible
I’m uncertain about how to trace the effects of 3 but I’d be surprised if it hadn’t contributed to the US’s aggressive posture towards the USSR, and thereby some of the nuclear near misses.
Jul 9, 2025 • 5 tweets • 2 min read
In my head I’ve started referring to political quadrants in terms of properties of their preferred coordination networks.
Top two are centralized. Bottom two are distributed.
Left two are symmetric (aka egalitarian). Right two are asymmetric.
I like these because the standard names have so much baggage. “Authoritarian” has negative connotations, “libertarian” suggests the bottom right, and ofc left and right are overloaded.
Focusing on network structures is a step towards thinking of politics as an actual science.
Jul 8, 2025 • 4 tweets • 4 min read
Three political positions that I think are severely underrated given the development of AGI:
1. @nathancofnas’ “hereditarian revolution” - the idea that the intellectual dominance of left-wing egalitarianism relies on group cognitive differences being taboo - is already very important.
But existing group cognitive differences pale in comparison to the ones that will emerge between baseline humans and:
- humans who leverage AI most effectively
- humans with brain-computer interfaces
- genetically engineered humans
- AIs themselves
Current cognitive differences already break politics; these will break it far more. So we need to be preparing for a future in which egalitarianism as an empirical thesis is (even more) obviously false.
I don’t yet have a concise summary of the implications of this position. But at the very least I want a name for it. Awkwardly, we don’t actually have a good word for “anti-egalitarian”. Hereditarian is too narrow (as is hierarchist) and elitist has bad connotations.
My candidate is “asymmetrist”. Egalitarianism tries to enforce a type of symmetry across the entirety of society. But our job will increasingly be to design societies where the absence of such symmetries is a feature not a bug.
2. Protectionism. Protectionism gets a bad rap, because global markets are very efficient. But they are very much not adversarially robust. If you are a small country and you open your borders to the currency, products and companies of a much larger country, then you will get short-term wealthier but also have an extremely hard time preventing that other country from gaining a lot of power over you in the long term. (As a historical example, trade was often an important precursor to colonial expansion. See also Amy Chua’s excellent book World on Fire, about how free markets enable some minorities to gain disproportionate power.)
When you’re poor enough, or the larger power is benevolent enough, this may well be a good deal! But we’re heading towards a future in which a) most people become far wealthier in absolute terms due to AI-driven innovation, and b) AIs will end up wielding a lot of power in not-very-benevolent ways (e.g. automated companies that have been given the goal of profit-maximization).
Given this, protectionism starts to look like a much better idea. The fact that it slows growth is not a problem, because society will already be reeling from the pace of change. And it lets you have much more control over the entities that are operating within your borders - e.g. you can monitor the use of AI decision-making within companies much more closely.
To put it another way, in the future the entire human economy will be the “smaller country” that faces incursions by currency, products and companies under the control of AIs (or humans who have delegated power to AIs). Insofar as we want to retain control, we shouldn’t let people base those AIs in regulatory havens while still being able to gain significant influence over western countries.
Okay, but won’t protectionist countries just get outcompeted? Not if they start off with enough power to deter other countries from deploying power-seeking AIs. And right now, the world’s greatest manufacturing power is already fairly protectionist. So if the US moves in that direction too, it seems likely that the combined influence of the US and China will be sufficient to prevent anyone else from “defecting”. The bottleneck is going to be trust between the two superpowers.
(Continued in tweet below.)
3. National conservatism
All of the above is premised on the goal of preserving human interests in a world of much more powerful agents. This is inherently a kind of conservatism, and one which we shouldn’t take for granted. The tech right often uses the language of “winning”, but as I’ve observed before there will increasingly be a large difference between a *country* winning and its *citizens* winning. In the limit, a fully-automated country could flourish economically and politically without actually benefiting any of the humans within it.
National conservatism draws a boundary around a group of people and says “here are the people whose interests we’re primarily looking out for”. As Vance put it, America is a group of people with a shared history and a common future. Lose sight of that, and arguments about efficiency and productivity will end up turning it instead into a staging-ground for the singularity. (Nor can you run a country for the benefit of “all humans”, because then you’re in an adversarial relationship with your own citizens, who rightly want their leaders to prioritize them.)
China’s government has many flaws, but it does get this part right. They are a nation-state run by their own people for their own people. As part of that, they’re not just economically protectionist but also culturally protectionist - blocking foreign ideas from gaining traction on their internet. I don’t think this is a good approach for the West, but I think we should try to develop a non-coercive equivalent: mechanisms by which a nation can have a conversation with itself about what it should value and where it should go, with ideas upweighted when their proponents have “skin in the game”. Otherwise the most eloquent and persuasive influencers will end up just being AIs.
All of these ideas are very high-level, but they give an outline of why I think right-wing politics is best-equipped to deal with the rise of AI. There’s a lot more work to do to flesh them out, though.
Jun 21, 2025 • 4 tweets • 1 min read
Anthropic selection is usually invoked to explain one-off coincidences (e.g. fine-tuning).
But evolution shows that many rounds of selection can design complex traits.
So it’s plausible that complex high-level features of our species or civilization were “designed” by anthropics.
The most obvious mechanism by which anthropic selection could iterate many times is via quantum effects - there are a huge number of branching points throughout the history of the universe, only some of which lead to sophisticated civilizations.
Jun 18, 2025 • 7 tweets • 2 min read
Hypothesis: we’ll look back on mass migration as being worse for Europe than WW2 was.
Europe recovered quickly from WW2, because each country remained high-trust and homogeneous.
But you can’t just rebuild your way out of internal ethno-religious fractures.
Why compare mass migration with WW2 specifically? Because the “never again, at any cost” attitude towards WW2 from European elites has been a major cultural force pushing against national identity and for suicidal immigration policies.
“Costly signaling” is one of the most important concepts but has one of the worst names.
The best signals are expensive for others - but conditional on that, the cheaper they are for you the better!
We should rename them “costly-to-fake signals”.
Consider an antelope stotting while being chased by lions. This is extremely costly for unhealthy antelopes, because it makes them much more likely to be eaten. But the fastest antelopes might be so confident the lion will never catch them that it’s approximately free for them.
May 5, 2025 • 4 tweets • 1 min read
I became a virtue ethicist after observing the failures of consequentialism and deontology in the real world.
But I’ve seldom read academic philosophers analyzing such examples when arguing about which ethical theory to endorse.
What are the best examples of that?
Philosophers like Singer have made arguments that, *given* a certain ethical view, real-world evidence should motivate certain actions.
But that’s different from saying that the real-world evidence should motivate the ethical view in the first place.
Apr 20, 2025 • 4 tweets • 1 min read
Modernity is a war of high and low against middle not just for classes, but also for levels of societal structure.
The power of middle-sized groups (like families, communities and states) is flowing both down to individuals, and up to international organizations and ideologies.
Power flowing out from the middle is generally negative-sum though, because high and low are too different to collaborate productively.
So you get big governments and global ideologies ruling over increasingly dysfunctional and atomized societies.
Apr 17, 2025 • 4 tweets • 2 min read
The AI safety community is very good at identifying levers of power over AI - e.g. evals for the most concerning capabilities.
Unfortunately this consistently leads people to grab those levers “as soon as possible”.
Usually it’s not literally the same people, but here it is.
To be clear, I don’t think it’s a viable strategy to stay fully hands-off the coming AI revolution, any more than it would have been for the Industrial Revolution.
But it’s particularly jarring to see the *evals* people leverage their work on public goods to go accelerationist.
Mar 26, 2025 • 5 tweets • 2 min read
We're heading towards a world where, in terms of skills and power, AIs are as far above humans as humans are above animals.
Obviously this has gone very badly for animals. So in a recent talk I ask: what political philosophy could help such a future go well?
The history of politics is a tug-of-war between the rule of "innately superior" aristocrats and blank-slate egalitarianism.
But these are both essentialist philosophies which deny empirical truths.
Instead, the duty of skilled/powerful elites should be to empower everyone else.
Feb 1, 2025 • 7 tweets • 1 min read
My biggest concern about prediction markets is that they commoditize access to power.
In the same way that every prediction market is an assassination market, it’s also a “buying decisions” market.
But institutions work only insofar as their decisions *aren’t* for sale.
It’s well-known that you can incentivize assassination by subsidizing a market about which day someone dies.
The assassin’s profit is limited only by needing to avoid raising alarm.
But markets on which day X bans Y have even worse incentives: X can bet as much as they want!
Jan 25, 2025 • 4 tweets • 3 min read
This essay is much more misleading than insightful, for (at least) two reasons:
1. The concept of AGI fully substituting for human labor is an incoherent one because humans have inherent advantages at some jobs simply because they're human. This can arise via consumer preferences (e.g. therapists), political considerations (e.g. lobbyists) or regulations (e.g. judges). As AI automates everything else, Baumol's effect predicts that such jobs become a large proportion of the economy.
It's fine to set up a naive econ model which ignores these, but it's irresponsible to give many pages of arguments about the implications of that naive model for the economy while relegating these crucial factors I mentioned above to one sentence in the conclusion. The way the essay is framed makes it hard for people who don't already know why it's wrong to realize how fragile the arguments are.
2. The essay claims that "as we continue innovating, we will eventually enter [a] second regime... in which we approach the physical limits of technological progress". This is true. It is also *ridiculously* distant. Forget Dyson spheres, this regime is one where we've figured out how to move stars around and colonize whole new galaxies and break at least a few things we currently consider fundamental laws of science.
Trying to appeal to this regime to draw *any* conclusions about human wages is absurd. None of the main concepts in this essay are robust enough that we can meaningfully extrapolate them that far. The essay is talking about "human wages" in a setting so futuristic that what even counts as "human" will likely be unrecognizable (due to genetic engineering/uploading/merging with AIs/etc).
The overall lesson: when you're reasoning about world-historic changes, you can't just take standard concepts that we use today, do some basic modeling, and run with that. All the hard work is in figuring out how our current concepts break when extrapolated into this new regime, and what to replace them with.
I'm criticising this more directly than I usually would because I recently called out someone else's similarly-ungrounded forecasts about the economy, and as part of that thread made these very points to Matthew.
Men and women are so psychologically different it's kinda weird that they WEREN'T very ideologically polarized until recently.
What changed? One hypothesis: dating forces you to compromise. But maybe political identities are now so strong that people prioritize them over dating.
Another way of putting it: the stronger the forces behind memetic evolution become, the more political factions will end up polarizing on the most robust and deep-rooted axis of psychological variation. And that's gender.
Dec 26, 2024 • 5 tweets • 2 min read
We need to distinguish “America” as a set of people from “America” as an embodiment of technocapitalist meritocracy.
The former alone stagnates. The latter alone “wins” at the cost of becoming inhuman - literally, because AIs will soon outcompete humans at basically everything.
I am a tech bro and an immigrant so I’m naturally sympathetic to attracting the very best foreign talent. But I really dislike framing the future of America in terms of winning. Silicon Valley is meant to be the place where people think in positive-sum ways. This is the opposite.
