I dislike when people praise each other for being vulnerable. Vulnerability is an undesirable side effect of emotional openness, not something to aim for.
My suggested replacement is to praise them for being courageous.
I considered “trusting” as a replacement, but that’s only a good thing insofar as the trust is well-placed, and there are many ways modern westerners shouldn’t trust the society around them. Whereas courage seems much more broadly desirable.
Jun 25 • 4 tweets • 3 min read
Silicon Valley is full of nerds fleeing the decline of mainstream institutions, who are too scared to confront the taboos which drove that decline, so end up using their power to exacerbate it.
Hence right-wing views were first censored on social media and now excluded from AIs.
For more on the censorship stuff, see @arctotherium42’s threads linked below.
I expect that similar dynamics apply in AI companies. E.g. the manual tweaking of Reddit algorithms to suppress r/The_Donald is likely paralleled by manual tweaking to suppress specific AI opinions.
There’s a sense in which frontier AGI companies have a lot of power.
But there’s also a sense in which they’ve disempowered themselves by speeding up the field so much that they have to focus hard on racing to remain at the frontier.
When the leading AGI company accelerates, that only grows their lead if competitors don’t respond.
But if competitors do the same, then the pace of progress speeds up overall, which shrinks their lead proportionately.
E.g. if the pace of progress doubles, then the lead halves.
May 20 • 4 tweets • 1 min read
Modern moral philosophy often focuses on how to interact with moral patients.
Whereas I think of ethics as being primarily about how you interact with other moral *agents* (which you can then generalize towards the limiting case of them having no agency).
In game theory, your interactions with moral patients are governed by your utility function, whereas your interactions with moral agents (whose decisions are correlated with yours) are governed by your decision theory.
IMO given this distinction, ethics is more about having a good decision theory than a good utility function. But it seems more productive to study the thing that’s halfway between them, which I call an “identity”: a predictive self-model that lets you produce self-fulfilling predictions of cooperation with other agents.
This isn’t meant to make much sense based on just this tweet. But I have a talk + research agenda on this coming out soon.
Apr 2 • 5 tweets • 2 min read
Imagine if the whole field of primatology were focused on figuring out which primates were politically progressive.
E.g. whenever chimpanzees fought, researchers would try to map their conflict onto human political divides.
This is, alas, roughly analogous to current AI safety.
To be more direct, the analogy is that in both cases researchers are too focused on proving a point to develop a proper scientific understanding.
Tweet loosely inspired by the QT, but this is endemic throughout the field. The Alignment Faking paper is another prominent case.
By contrast, the US and China are *not* in Molochian competition. They could burn their values in many ways to be more competitive, and they mostly don't.
Indeed, the US in particular could become much more competitive without sacrificing *anything*. That's not very Molochian.
Mar 16 • 4 tweets • 2 min read
My sense is that trying to “automate alignment research” means doing roughly the same thing as automating capabilities research.
Except that you’re taking superintelligence more seriously, and therefore you’ll be much more effective.
A better strategy is to figure out the most promising path towards you personally making a breakthrough in our understanding of neural nets/learning/intelligence.
Then use AI whenever it seems like it’d help you towards this specific goal.
Designing policies for existing governments is too shallow an intervention to be robust to AGI.
Long-term AGI governance requires rethinking what nation-states even are.
The only serious attempt I’ve seen to intervene at that level was @Dominic2306’s Brexit campaign.
Brexit was a big win for national sovereignty—and it was driven by principled, courageous political philosophy.
Idk how explicitly Dom planned for Brexit to improve AI governance. But he was early to AI safety, so I expect it factored into his worldview.
Be more like Dom.
Feb 24 • 6 tweets • 2 min read
Challenge: “AI safety” people stop homing in with laser precision on the best ways to speed up AI.
This is primarily a problem with the EA-affiliated side of AI safety. Unfortunately, that’s most of the field by now.
EAs don’t have memetic defenses against conflating “do the most good” with “gain the most power” (or sometimes just “be in the room where it happens”).
Sep 15, 2025 • 5 tweets • 1 min read
Once you notice how often “beautiful” is used as a default compliment for women, it’s hard to unsee.
Instead of “my gorgeous wife”, may I suggest describing her as courageous, sincere, virtuous, earnest, patient, determined, or (as a generic fallback) loving.
These don’t roll off the tongue as easily, but partly that’s because we’re not used to them.
The other problem is that these words are too informative: saying them actually conveys an opinion, which is scary—unlike “beautiful” (or that most insipid of compliments, “lovely”).
Aug 31, 2025 • 4 tweets • 2 min read
Great thread. I’d even go further—I think we’re still recovering from Von Neumann’s biggest mistakes:
1. Implicitly basing game theory on causal decision theory 2. Founding utility theory on the independence axiom 3. Advocating for nuking the USSR as soon as possible
I’m uncertain about how to trace the effects of 3 but I’d be surprised if it hadn’t contributed to the US’s aggressive posture towards the USSR, and thereby some of the nuclear near misses.
Jul 9, 2025 • 5 tweets • 2 min read
In my head I’ve started referring to political quadrants in terms of properties of their preferred coordination networks.
Top two are centralized. Bottom two are distributed.
Left two are symmetric (aka egalitarian). Right two are asymmetric.
I like these because the standard names have so much baggage. “Authoritarian” has negative connotations, “libertarian” suggests the bottom right, and ofc left and right are overloaded.
Focusing on network structures is a step towards thinking of politics as an actual science.
Jul 8, 2025 • 4 tweets • 4 min read
Three political positions that I think are severely underrated given the development of AGI:
1. @nathancofnas’ “hereditarian revolution” - the idea that the intellectual dominance of left-wing egalitarianism relies on group cognitive differences being taboo - is already very important.
But existing group cognitive differences pale in comparison to the ones that will emerge between baseline humans and:
- humans who leverage AI most effectively
- humans with brain-computer interfaces
- genetically engineered humans
- AIs themselves
Current cognitive differences already break politics; these will break it far more. So we need to be preparing for a future in which egalitarianism as an empirical thesis is (even more) obviously false.
I don’t yet have a concise summary of the implications of this position. But at the very least I want a name for it. Awkwardly, we don’t actually have a good word for “anti-egalitarian”. Hereditarian is too narrow (as is hierarchist) and elitist has bad connotations.
My candidate is “asymmetrist”. Egalitarianism tries to enforce a type of symmetry across the entirety of society. But our job will increasingly be to design societies where the absence of such symmetries is a feature not a bug.
2. Protectionism. Protectionism gets a bad rap, because global markets are very efficient. But they are very much not adversarially robust. If you are a small country and you open your borders to the currency, products and companies of a much larger country, then you will get short-term wealthier but also have an extremely hard time preventing that other country from gaining a lot of power over you in the long term. (As a historical example, trade was often an important precursor to colonial expansion. See also Amy Chua’s excellent book World on Fire, about how free markets enable some minorities to gain disproportionate power.)
When you’re poor enough, or the larger power is benevolent enough, this may well be a good deal! But we’re heading towards a future in which a) most people become far wealthier in absolute terms due to AI-driven innovation, and b) AIs will end up wielding a lot of power in not-very-benevolent ways (e.g. automated companies that have been given the goal of profit-maximization).
Given this, protectionism starts to look like a much better idea. The fact that it slows growth is not a problem, because society will already be reeling from the pace of change. And it lets you have much more control over the entities that are operating within your borders - e.g. you can monitor the use of AI decision-making within companies much more closely.
To put it another way, in the future the entire human economy will be the “smaller country” that faces incursions by currency, products and companies under the control of AIs (or humans who have delegated power to AIs). Insofar as we want to retain control, we shouldn’t let people base those AIs in regulatory havens while still being able to gain significant influence over western countries.
Okay, but won’t protectionist countries just get outcompeted? Not if they start off with enough power to deter other countries from deploying power-seeking AIs. And right now, the world’s greatest manufacturing power is already fairly protectionist. So if the US moves in that direction too, it seems likely that the combined influence of the US and China will be sufficient to prevent anyone else from “defecting”. The bottleneck is going to be trust between the two superpowers.
(Continued in tweet below.)
3. National conservatism
All of the above is premised on the goal of preserving human interests in a world of much more powerful agents. This is inherently a kind of conservatism, and one which we shouldn’t take for granted. The tech right often uses the language of “winning”, but as I’ve observed before there will increasingly be a large difference between a *country* winning and its *citizens* winning. In the limit, a fully-automated country could flourish economically and politically without actually benefiting any of the humans within it.
National conservatism draws a boundary around a group of people and says “here are the people whose interests we’re primarily looking out for”. As Vance put it, America is a group of people with a shared history and a common future. Lose sight of that, and arguments about efficiency and productivity will end up turning it instead into a staging-ground for the singularity. (Nor can you run a country for the benefit of “all humans”, because then you’re in an adversarial relationship with your own citizens, who rightly want their leaders to prioritize them.)
China’s government has many flaws, but it does get this part right. They are a nation-state run by their own people for their own people. As part of that, they’re not just economically protectionist but also culturally protectionist - blocking foreign ideas from gaining traction on their internet. I don’t think this is a good approach for the West, but I think we should try to develop a non-coercive equivalent: mechanisms by which a nation can have a conversation with itself about what it should value and where it should go, with ideas upweighted when their proponents have “skin in the game”. Otherwise the most eloquent and persuasive influencers will end up just being AIs.
All of these ideas are very high-level, but they give an outline of why I think right-wing politics is best-equipped to deal with the rise of AI. There’s a lot more work to do to flesh them out, though.
Jun 21, 2025 • 4 tweets • 1 min read
Anthropic selection is usually invoked to explain one-off coincidences (e.g. fine-tuning).
But evolution shows that many rounds of selection can design complex traits.
So it’s plausible that complex high-level features of our species or civilization were “designed” by anthropics.
The most obvious mechanism by which anthropic selection could iterate many times is via quantum effects - there are a huge number of branching points throughout the history of the universe, only some of which lead to sophisticated civilizations.
Jun 18, 2025 • 7 tweets • 2 min read
Hypothesis: we’ll look back on mass migration as being worse for Europe than WW2 was.
Europe recovered quickly from WW2, because each country remained high-trust and homogeneous.
But you can’t just rebuild your way out of internal ethno-religious fractures.
Why compare mass migration with WW2 specifically? Because the “never again, at any cost” attitude towards WW2 from European elites has been a major cultural force pushing against national identity and for suicidal immigration policies.
“Costly signaling” is one of the most important concepts but has one of the worst names.
The best signals are expensive for others - but conditional on that, the cheaper they are for you the better!
We should rename them “costly-to-fake signals”.
Consider an antelope stotting while being chased by lions. This is extremely costly for unhealthy antelopes, because it makes them much more likely to be eaten. But the fastest antelopes might be so confident the lion will never catch them that it’s approximately free for them.
May 5, 2025 • 4 tweets • 1 min read
I became a virtue ethicist after observing the failures of consequentialism and deontology in the real world.
But I’ve seldom read academic philosophers analyzing such examples when arguing about which ethical theory to endorse.
What are the best examples of that?
Philosophers like Singer have made arguments that, *given* a certain ethical view, real-world evidence should motivate certain actions.
But that’s different from saying that the real-world evidence should motivate the ethical view in the first place.
Apr 20, 2025 • 4 tweets • 1 min read
Modernity is a war of high and low against middle not just for classes, but also for levels of societal structure.
The power of middle-sized groups (like families, communities and states) is flowing both down to individuals, and up to international organizations and ideologies.
Power flowing out from the middle is generally negative-sum though, because high and low are too different to collaborate productively.
So you get big governments and global ideologies ruling over increasingly dysfunctional and atomized societies.
Apr 17, 2025 • 4 tweets • 2 min read
The AI safety community is very good at identifying levers of power over AI - e.g. evals for the most concerning capabilities.
Unfortunately this consistently leads people to grab those levers “as soon as possible”.
Usually it’s not literally the same people, but here it is.
To be clear, I don’t think it’s a viable strategy to stay fully hands-off the coming AI revolution, any more than it would have been for the Industrial Revolution.
But it’s particularly jarring to see the *evals* people leverage their work on public goods to go accelerationist.
Mar 26, 2025 • 5 tweets • 2 min read
We're heading towards a world where, in terms of skills and power, AIs are as far above humans as humans are above animals.
Obviously this has gone very badly for animals. So in a recent talk I ask: what political philosophy could help such a future go well?
The history of politics is a tug-of-war between the rule of "innately superior" aristocrats and blank-slate egalitarianism.
But these are both essentialist philosophies which deny empirical truths.
Instead, the duty of skilled/powerful elites should be to empower everyone else.
