Academia cares whether an idea is new. It doesn't really have to work

Industry only cares if an idea works. Doesn't matter if it's new

This creates a gap. Actually a few gaps: 1. It creates a culture gap

Academics look at industry people trying to get an idea to work, and complain, “they aren't doing anything new!”

https://twitter.com/jasoncrawford/status/1752891641566781942

If “low-hanging fruit” or “ideas getting harder to find” was the main factor in the rate of technological progress, then the fastest progress would have been in the Stone Age.

Ideas were *very easy to find* in the Stone Age! There was *so much* low-hanging fruit! Instead, the pattern we see is the opposite: progress accelerates over time. (Note that the chart below is *already on a log scale*)

Clearly, there is some positive factor that more than makes up for ideas getting harder to find / low-hanging fruit getting picked.

Suppose you give an AI an innocuous-seeming goal, like playing chess, fetching coffee, or calculating digits of π. What could go wrong?

Well, there is an argument that even “safe” goals for AI could be very dangerous.

I'm going to give the argument—and then push back on it. This thread is adapted from an essay here, in case you prefer that format: rootsofprogress.org/power-seeking-…

There is an AI doom argument that goes, in essence:

1. Sufficiently advanced AI will be smarter than us
2. Anything smarter than us, we cannot control
3. Having something in the world that we cannot control would be bad

∴ Sufficiently advanced AI would be bad. QED One counter is to deny (1), eg: AI will never be that smart; intelligence is multi-dimensional and it doesn't make sense to compare them; super-human intelligence is so far in the future that we shouldn't worry about it; etc

This is becoming less popular recently as AI advances.

Levels of safety for a technology

1. So dangerous that no one can use it safely
2. Safe if used carefully, dangerous otherwise
3. Safe if used normally, dangerous in malicious hands
4. So safe that even bad actors cannot cause harm

Important to know which you are talking about. Arguably:

Level 1 should be banned
Level 2 requires licensing/insurance schemes
Level 3 requires security against bad actors
Level 4 is ideal!

“Optimal Policies Tend to Seek Power” supposedly gives a theoretical basis for power-seeking behavior from AI

But it seems to just analyze a toy model and show that if you head towards a larger part of the state space, you are more likely to optimize a random reward function? The intro claims that “power-seeking tendencies arise not from anthropomorphism, but from certain graphical symmetries present in many MDPs [markov decision processes]”

But what is actually demonstrated seems much more trivial than that. What am I missing?

“I wonder that the Lord God has kept such things hidden from so many learned, pious and noble people since the beginning of the world, preserving them for you and only revealing them in this last age.”

King James I, when presented with an invention by Cornelis Drebbel, in 1607 Mokyr was right: “The idea of progress is logically equivalent to an implied disrespect of previous generations”

If a technology may introduce catastrophic risks, how do you develop it?

The Wright Brothers' approach to inventing the airplane is one case study: The catastrophic risk, of course, was dying in a crash.

This is exactly what happened to one of the Wrights' predecessors, Otto Lilienthal, who attempted to fly using a kind of glider. He had many successful experiments, but one day he lost control, fell, and broke his neck.

Imagine you could go back in time to the ancient world to jump-start the Industrial Revolution.

You carry with you plans for a steam engine, and you present them to the emperor, explaining how the machine could be used at mines, mills, blast furnaces, etc.

But to your dismay… The emperor responds:

“Your mechanism is no gift to us. It is tremendously complicated; it would take my best master craftsmen years to assemble. It is made of iron, which could be better used for weapons and armor.…

The most important issue in the AI safety debate right now is not *what* the risks of AI are (object level), but how we should even approach the problem (meta level).

And the biggest question is rationalism vs. empiricism. Empiricism is the sober, historical approach. It says that we create AI safety the same way we have always created safety from technological risks: through trial and error.

“Instead of a desk, I would like to have a very large lazy susan in my office.… My whole life would be spread out on this lazy susan.… With my projects laid out on my lazy susan, they would each have a claim on my attention that they could never have if they were filed away.” h/t @TrevMcKendrick

source: cs.bilkent.edu.tr/~david/plea.ht…

Are we going through a crisis of meaning in our jobs? Are workers increasingly detached, disengaged, and alienated from their roles?

Recently I have argued that the overall historical trend in jobs has been toward *more* meaning. This seems to be an unpopular opinion.

Thread: .@drorpoleg and @collinconnors argue against me. (Connor replying to me replying to Dror below.)

Dror argues that more professionals today work for large bureaucracies and lack autonomy. Connor blames “virtualization”, “atomization”, and people not working in the physical world.

A review of Seeing Like a State in 6 tweets.

James Scott says that “tragic episodes” of social engineering have four elements: the administrative ordering of society (“legibility”), “high-modernist” ideology, an authoritarian state, & a society that lacks the capacity to resist. This is a bit like saying that the worst wildfires have four elements: an overgrowth of brush and trees, a prolonged dry season, a committed arsonist, and strong prevailing winds.

One of these things is not like the others!

“Traditional foods” are not very old.

The French baguette: adopted nationwide only after WW2

Greek moussaka: created early 20th c. to Frenchify Greek food

Tequila? The Mexican film industry made it the national drink in the 1930s

(All from an excellent @rachellaudan article) And “ethnic” dishes were invented for aristocrats:

“This is as true of the lasagna of northern Italy as it is of the chicken korma of Mughal Delhi, the moo shu pork of imperial China, and the pilafs, stuffed vegetables & baklava of the great Ottoman palace in Istanbul”

What is this thing where people who come to your house text you “I'm here” but *don't* also knock or ring the bell? (Is this a millennial/zoomer thing?) Note, I'm talking about a situation where the visitor is expected.

Most common answer is dogs/babies. That makes sense. (We don't have a dog, and our baby doesn't wake up from the doorbell, so I guess I haven't felt this need personally)

Amazing progress on tap water hookups in rural India: h/t @salonium. Source: theprint.in/india/governan…

Famine has been eliminated in much of the world not only because of high agricultural productivity, but also because communication and transportation networks allow aid to come quickly to stricken areas: No famine in the Americas since 1900, in the Middle East since 1920, in Europe since 1950, or in Asia since 2000

Mark Twain wrote to Walt Whitman for his 70th birthday, and his letter was a celebration of progress:

“What great births you have witnessed!” “The steam press, the steamship, the steel ship, the railroad, the perfected cotton-gin, the telegraph, the phonograph, the photograph, photo-gravure…

Please welcome @HeikeLarson as the incoming CEO for @rootsofprogress!

https://twitter.com/rootsofprogress/status/1598090595678814210

A few months ago @rootsofprogress announced a major expansion of our activities, from supporting just my work to supporting a broader network of progress writers. Along with that, we launched a search for a CEO to lead the new organization:

rootsofprogress.org/seeking-a-ceo

A working horse generates 30–50 pounds of manure and a gallon of urine per *day*. In 1900, when Manhattan had only 1.8M people, it had 130,000 horses (do the math). Horses also die on the job, leaving carcasses in the street until they can be hauled away. You will not find those facts mentioned in this @guardian story. (h/t @tylercowen in @MargRev)

theguardian.com/world/2022/nov…

Should we “go against nature”? Or live in “harmony” with it?

There are two senses of “nature.” Teasing them apart clarifies this issue. “Nature” can mean immutable natural law. We defy this at our peril.

If we dump sewage in our drinking water, we will suffer epidemics. If we expose ourselves to radiation, we will die from cancer. If we fail to irrigate our fields, we will go hungry at the first drought.