There is no silver bullet for risk. Safety is achieved through defense in depth, and through the orchestration of a wide variety of solutions.

Last month, in a private talk, I gave an example: fire safety. It resonated strongly, so I wrote it up.

(Thread here, blog link at end) City fires were once a great hazard:

Neighborhoods were full of wooden buildings without fire alarms or sprinkler systems

Open flames were used everywhere for lighting, heating, and cooking

Fire departments worked with inadequate equipment and insufficient water supply

One model for Elon is that he is perpetually operating in wartime mode

Move as fast as possible, bring overwhelming force, extract superhuman effort, accept collateral damage

Fits a range of things from SpaceX to DOGE. Anyway, I find it easier to understand him using this lens I first thought of this when I read this story in a review of a book about SpaceX

What explains this? In what model of the world could it be rational? The only answer is wartime. These are the actions of a soldier on the front lines

thepsmiths.com/p/review-reent…

A new article in Science voices concern about a line of biological research which, if successful long-term, could create a grave threat to humanity and to most life on Earth.

Fortunately, the threat is distant, and avoidable—but only if we have common knowledge of it. The risk is from “mirror bacteria.”

Mirror bacteria are bacteria whose biomolecules all have the opposite chirality to that of normal organisms: all left-handed molecules replaced by right-handed ones, and vice versa.

(Below: normal glucose, and its mirror twin.)

The steam engine was invented in 1712. An observer at the time might have said: “The engine will power everything: factories, ships, carriages. Horses will become obsolete!”

And they would have been right—but two hundred years later, we were still using horses to plow fields. In fact, it took about a hundred years for engines to be used for transportation, in steamboats and locomotives, both invented in the early 1800s. It took more than fifty years just for engines to be widely used in factories.

What happened? Many factors, including:

This is a prescription for re-enslaving women to domestic service, and ensuring that only the wealthy can live with the basic dignity of cleanliness.

What is described here is exactly how we used to do laundry, and it was terrible. Laundry was difficult manual labor that took up an entire day of the week, and was part of why being a housewife was a full-time job.

To quote a scholar who actually knows this topic (Ruth Schwartz Cowan, More Work for Mother):

“For most women, for most of the year, the labor of doing laundry meant carrying heavy buckets of water from tap to stove and from stove to tub, repeatedly overturning the tubs and refilling them, as well as carrying, scrubbing, wringing, and hanging the heavy fabrics that were the only ones cheap enough for poor people to buy. The labor of getting the family bathed was similar, lacking only the carrying, scrubbing, wringing, and hanging of the wash.…

“Even if a dwelling contained a sink, it was usually not deep enough for doing laundry and may not have had a drain. Public laundries were few and far between; and so, for that matter, were public bath houses…

“The net result of the profound difficulty that washing and bathing presented was that precious little of it got done; underclothing might be changed only once a week, or even once a season; sheets likewise (if they were used at all, since featherbeds did not require them); outerclothes might do with just a brushing; shirts or shirtwaists might go for weeks without benefit of soap; faces and hands might get splashed with water once a day; full body bathing might occur only on Saturday nights (and then with a sponge and a wooden tub and water that was used and reused) or only when underwear was changed—or never at all.

“‘Some women have a feeling that cleanliness is a condition only for the rich,’ one home economist remarked of the immigrant women with whom she worked…”

Cowan also quotes an early 20th-century writer as saying:

“Many people do not sufficiently realize the extent to which the increase in cleanliness of home and person contributes toward the growth of democracy. So long as the upper classes felt the necessity of using smelling salts whenever approached by one of the common people, just so long would they despise the vile-smelling yokels. Cleanliness is not only next to Godliness, but it is essential to the establishment of the Brotherhood of Man.” If you don't believe history, maybe you'll believe someone who's lived this.

“Routine tasks, like scrubbing clothes, are such a waste of humans' capacity for creativity and innovation”

Listen to @_alice_evans:

https://x.com/_alice_evans/status/1836797985658957859

Academia cares whether an idea is new. It doesn't really have to work

Industry only cares if an idea works. Doesn't matter if it's new

This creates a gap. Actually a few gaps: 1. It creates a culture gap

Academics look at industry people trying to get an idea to work, and complain, “they aren't doing anything new!”

https://twitter.com/jasoncrawford/status/1752891641566781942

If “low-hanging fruit” or “ideas getting harder to find” was the main factor in the rate of technological progress, then the fastest progress would have been in the Stone Age.

Ideas were *very easy to find* in the Stone Age! There was *so much* low-hanging fruit! Instead, the pattern we see is the opposite: progress accelerates over time. (Note that the chart below is *already on a log scale*)

Clearly, there is some positive factor that more than makes up for ideas getting harder to find / low-hanging fruit getting picked.

Suppose you give an AI an innocuous-seeming goal, like playing chess, fetching coffee, or calculating digits of π. What could go wrong?

Well, there is an argument that even “safe” goals for AI could be very dangerous.

I'm going to give the argument—and then push back on it. This thread is adapted from an essay here, in case you prefer that format: rootsofprogress.org/power-seeking-…

There is an AI doom argument that goes, in essence:

1. Sufficiently advanced AI will be smarter than us
2. Anything smarter than us, we cannot control
3. Having something in the world that we cannot control would be bad

∴ Sufficiently advanced AI would be bad. QED One counter is to deny (1), eg: AI will never be that smart; intelligence is multi-dimensional and it doesn't make sense to compare them; super-human intelligence is so far in the future that we shouldn't worry about it; etc

This is becoming less popular recently as AI advances.

Levels of safety for a technology

1. So dangerous that no one can use it safely
2. Safe if used carefully, dangerous otherwise
3. Safe if used normally, dangerous in malicious hands
4. So safe that even bad actors cannot cause harm

Important to know which you are talking about. Arguably:

Level 1 should be banned
Level 2 requires licensing/insurance schemes
Level 3 requires security against bad actors
Level 4 is ideal!

“Optimal Policies Tend to Seek Power” supposedly gives a theoretical basis for power-seeking behavior from AI

But it seems to just analyze a toy model and show that if you head towards a larger part of the state space, you are more likely to optimize a random reward function? The intro claims that “power-seeking tendencies arise not from anthropomorphism, but from certain graphical symmetries present in many MDPs [markov decision processes]”

But what is actually demonstrated seems much more trivial than that. What am I missing?

“I wonder that the Lord God has kept such things hidden from so many learned, pious and noble people since the beginning of the world, preserving them for you and only revealing them in this last age.”

King James I, when presented with an invention by Cornelis Drebbel, in 1607 Mokyr was right: “The idea of progress is logically equivalent to an implied disrespect of previous generations”

If a technology may introduce catastrophic risks, how do you develop it?

The Wright Brothers' approach to inventing the airplane is one case study: The catastrophic risk, of course, was dying in a crash.

This is exactly what happened to one of the Wrights' predecessors, Otto Lilienthal, who attempted to fly using a kind of glider. He had many successful experiments, but one day he lost control, fell, and broke his neck.

Imagine you could go back in time to the ancient world to jump-start the Industrial Revolution.

You carry with you plans for a steam engine, and you present them to the emperor, explaining how the machine could be used at mines, mills, blast furnaces, etc.

But to your dismay… The emperor responds:

“Your mechanism is no gift to us. It is tremendously complicated; it would take my best master craftsmen years to assemble. It is made of iron, which could be better used for weapons and armor.…

The most important issue in the AI safety debate right now is not *what* the risks of AI are (object level), but how we should even approach the problem (meta level).

And the biggest question is rationalism vs. empiricism. Empiricism is the sober, historical approach. It says that we create AI safety the same way we have always created safety from technological risks: through trial and error.

“Instead of a desk, I would like to have a very large lazy susan in my office.… My whole life would be spread out on this lazy susan.… With my projects laid out on my lazy susan, they would each have a claim on my attention that they could never have if they were filed away.” h/t @TrevMcKendrick

source: cs.bilkent.edu.tr/~david/plea.ht…

Are we going through a crisis of meaning in our jobs? Are workers increasingly detached, disengaged, and alienated from their roles?

Recently I have argued that the overall historical trend in jobs has been toward *more* meaning. This seems to be an unpopular opinion.

Thread: .@drorpoleg and @collinconnors argue against me. (Connor replying to me replying to Dror below.)

Dror argues that more professionals today work for large bureaucracies and lack autonomy. Connor blames “virtualization”, “atomization”, and people not working in the physical world.

A review of Seeing Like a State in 6 tweets.

James Scott says that “tragic episodes” of social engineering have four elements: the administrative ordering of society (“legibility”), “high-modernist” ideology, an authoritarian state, & a society that lacks the capacity to resist. This is a bit like saying that the worst wildfires have four elements: an overgrowth of brush and trees, a prolonged dry season, a committed arsonist, and strong prevailing winds.

One of these things is not like the others!

“Traditional foods” are not very old.

The French baguette: adopted nationwide only after WW2

Greek moussaka: created early 20th c. to Frenchify Greek food

Tequila? The Mexican film industry made it the national drink in the 1930s

(All from an excellent @rachellaudan article) And “ethnic” dishes were invented for aristocrats:

“This is as true of the lasagna of northern Italy as it is of the chicken korma of Mughal Delhi, the moo shu pork of imperial China, and the pilafs, stuffed vegetables & baklava of the great Ottoman palace in Istanbul”

What is this thing where people who come to your house text you “I'm here” but *don't* also knock or ring the bell? (Is this a millennial/zoomer thing?) Note, I'm talking about a situation where the visitor is expected.

Most common answer is dogs/babies. That makes sense. (We don't have a dog, and our baby doesn't wake up from the doorbell, so I guess I haven't felt this need personally)

Amazing progress on tap water hookups in rural India: h/t @salonium. Source: theprint.in/india/governan…