I genuinely think "consciousness" is simply the modern, secular term for "soul". Both refer to unfalsifiable concepts used to determine who is in or out of our moral ingroup. Neither are empirical designations discovered through experiment, but socially constructed categories. People often argue that future AIs won't be conscious, saying that AIs will only simulate consciousness. However, if there's no experimental method to distinguish genuine consciousness from a simulation, the simplest explanation is that "genuine consciousness" itself isn't real.

This is a good time to reflect on the "AI effect". Before a benchmark is solved, people often think we'll need "real AGI" to solve it. Then, afterwards, we realize the benchmark can be solved using mere tricks.

Will this benchmark fall in the same way? Honestly, I'm not sure.🧵

https://twitter.com/EpochAIResearch/status/1854993676524831046

The first thing to understand about FrontierMath is that it's genuinely extremely hard. Almost everyone on Earth would score approximately 0%, even if they're given a full day to solve *each* problem. For fun, here's what a few people on Reddit said after looking at the problems.

On a first approximation, I think the right way of approaching p(doom | AI) is to ask: will we be able to implement a legal system in which people can peacefully coexist and prosper alongside each other despite the fact that some legal persons will be way smarter than others? In the foom scenario, a single unified agent takes control over the entire world, and largely abolishes law itself. With no inter-agent competition, the AI's values are fulfilled ~exactly, without need for compromise with other agents. In this scenario, I agree p(doom) is high.

Joseph Carlsmith estimated that the human brain uses approximately 10^15 FLOP/s. Over 30 years, that's about 10^24 FLOP.

Language models exploded in popularity in the last year, timed almost exactly with the release of ML models trained using over 10^24 FLOP. What's remarkable about this milestone is that it could have been forecasted many decades ago. One paper reviewed estimates of the size of the human brain over 150 years and found that, "Overall... most of the [estimates] are relatively consistent."
ncbi.nlm.nih.gov/pmc/articles/P…

I recently criticized the calls to pause model scaling. However, my arguments were brief. Therefore, I thought it might be valuable to elaborate on my view that we should be cautious about slowing down AI progress. 🧵 It appears likely to me that we cannot delay the arrival of advanced AI for a more than about a decade without extreme, draconian regulations. The reason is because AI is very useful, and research can be done using relatively few resources.

I currently think this open letter is quite bad, and possibly net harmful. The proposed policy appears vague and misguided. I want to explain some of my thoughts. 🧵
futureoflife.org/open-letter/pa… Frustratingly, the open letter lumps concerns of unemployment and misinformation in with existential risks. By contrast, I am more far more optimistic about AI in the short term since I don't think AI systems are yet close to posing any existential risk.

I performed a basic analysis to see if I could retrodict GPT-4's score on the MMLU benchmark (a set of 57 high school and college exams) using only publicly available data.

My very simple model predicted a score of 85.4%. GPT-4's actual score was 86.4%.🧵 I used data from Papers with Code to create a linear model of MMLU benchmark performance. The independent variable was -log(estimated reducible loss) according to the Chinchilla scaling law for each model.
paperswithcode.com/sota/multi-tas…

I think the idea that automation always increases employment is really wrong. Here's a somewhat substantive thread that critiques this idea. 🧵

https://twitter.com/JosephPolitano/status/1632029149828202496

It's true, for example, that the prime age labor force participation rate in the United States went up in the last 70 years, peaking in the 90s. However, this mostly reflects a shift from women doing non-market labor to holding formal jobs.

Something that surprised me last year regarding LLMs was their ability to do mathematics well. I now suspect that mathematics is not much harder for computers to understand than ordinary natural language documents. This has pretty interesting implications. 🧵 I was previously too anchored to statements that researchers made about how we weren't making progress.

For example: "While scaling Transformers is automatically solving most other text-based tasks, scaling is not currently solving MATH."
arxiv.org/abs/2103.03874

New estimate of the consumer surplus of the internet just dropped. If it's accurate, then there was no great stagnation.

Let me explain.🧵

https://twitter.com/JgaltTweets/status/1618783360608006145

The great stagnation theory is the idea that economic progress has been slowing down in recent decades. It has often been assumed to be true on the basis of GDP statistics, which show declining rates of per-capita economic growth in recent times.
en.wikipedia.org/wiki/The_Great…

One of the most common arguments against AGI being near is the following take: AI has gone through many boom and bust cycles before in which people thought we were close, but we ended up being far. This boom will also bust.

Ultimately, I find this argument quite weak. 🧵 Perhaps the clearest thinker who uses this argument is @robinhanson. For example, in 2019 he outlined his perspective that in the past we've seen "repeated booms of AI concern and interest" that eventually went bust, going back to (at least) the 1930s.
aiimpacts.org/conversation-w…

Some improvements we might start to see more in large language models within 2 years:

- Explicit memory that will allow it to retrieve documents and read them before answering questions arxiv.org/abs/2112.04426 - A context window of hundreds of thousands of tokens, allowing the model to read and write entire books arxiv.org/abs/2202.07765

There's been a lot of low quality GPT-4 speculation recently. So, here's a relatively informed GPT-4 speculation thread from an outsider who still doesn't know that much. 🧵 In a blog post from 2020, Microsoft announced a new supercomputer for the exclusive purpose of training large ML models for OpenAI. They stated that "Compared with other machines listed on the TOP500 supercomputers in the world, it ranks in the top five".

blogs.microsoft.com/ai/openai-azur…

Disagreement about AI timelines is often framed as a disagreement about the anticipated rate of future AI progress. However, I believe the real disagreement is often not about the rate of progress, but about the threshold required for AI to be transformative. This disagreement manifests, for example, in Eliezer Yudkowsky's statement that it would "not surprise [him] in the least" if AGI is created and destroys the world before consumers are able to purchase self-driving cars.
lesswrong.com/posts/7im8at9P…

As Putin appears ready to invade Ukraine, it's worth looking at some trends and historical facts, to get a better sense as to what's happening and why. 🧵

For the last 30 years, public opinion in Russia has largely regretted the collapse of the Soviet Union. The data is likely driven by nostalgia. Old people are more likely to regret the fall of communism.

Suppose you are initially 99% confident in a claim that you spent 2 hours researching. You then learn that Carl Sagan spent 1000 hours researching the claim and came to the opposite conclusion. Where does your credence land after learning this? Suppose you are initially 99% confident in a claim that you spent 2 hours researching. You then learn that Bill Nye spent 1000 hours researching the claim and came to the opposite conclusion. Where does your credence land after learning this?

A common belief about the Soviet Union is that famines were rampant. In fact, the truth is largely the opposite: widespread food waste and overproduction plagued their system. nintil.com/the-soviet-uni…

But the common belief is based on real history. Let me elaborate... 1/ It is true that the Soviet Union experienced extreme famine in the 1930s. The most famous incident was the Holodomor, a famine in Ukraine that most scholars believe was partially intentional.

en.wikipedia.org/wiki/Holodomor

2/