The dirty secret of machine learning is benchmarks are all we have.

https://twitter.com/RogerGrosse/status/1838153056665448923

Since the very beginning of whatever the hell we call AI, we've had benchmarks. Checkers and chess in the 1950s, OCR in the 1960s, speech in the 1980s... 2/x

This is a great thread. But there is an alternative conclusion: Reproducibility crises themselves are overblown and missing the point.

https://twitter.com/dandekadt/status/1667831071541526528

There has never been a time in science when most published findings were true. And yet, many disciplines managed to have a profound impact nonetheless, why?

There was a thread on here last week where two Turing award winners are jovially promoting the idea that mathematical statistics is better at determining causation than common sense. 1/10 We have known for at least 50 years that this is wrong, and yet academics continue to push this illusion.

Here are some of my favorite critiques… I’ll give 6 because I don’t believe in 93 part threads. 2/10

I have a recommended reading list for Artificial Intelligence, and it hasn't changed since 2019. I give this list to my grad students, but all of the articles are broadly accessible if you're interested. Very short 🧵. 1) Ted Chiang's critique of the threat of superintelligence.

buzzfeednews.com/article/tedchi…

The IBM 704 was the most amazing general purpose AI computer ever made. Released in 1954, the 704 could compute 12 thousand floating point operations a second. And it ran on vacuum tubes.

By now on this fine Friday you have all seen this terrible MMWR study. It has many flaws, but I particularly am incensed that it insults my favorite observational design, the test-negative control design.

cdc.gov/mmwr/volumes/7… Test-negative control is used to evaluate vaccine effectiveness. It attempts to avoid confounding by restricting its subjects to those seeking medical attention.

This short twitter thread provides a brief tutorial on how to read econ papers without knowing any math.

https://twitter.com/dylanmatt/status/1489289414509568004

The intro will be in plain English and will usually claim some "counterintuitive surprise." But all of the main math results will be summarized in table 1 or table 2. 2/9

What are examples of observational data analysis leading to widespread, faulty scientific consensus? Context: I ask because I just re-read a lovely essay "On Types of Scientific Inquiry: The Role of Qualitative Reasoning" by David Freedman. 2/10

A few follow up thoughts on our protagonist Bill Highleyman. (1/x)

https://twitter.com/beenwrekt/status/1450824589907353605

It blows my mind that Bill invented so many powerful machine learning primitives---finding linear functions that minimize empirical risk, gradient descent to minimize the risk, train-test splits, convolutional neural networks---all as part of his PhD dissertation project. (2/x)

I’m excited to share a new textbook @mrtz and I wrote: "Patterns, Predictions, and Actions: A Story about Machine Learning." 1/5

mlstory.org We cover the foundations of prediction and pattern recognition, moving from decision theory to supervised learning to causality and reinforcement learning. 2/5

I want to take a moment to discuss the terminology "random search." (1/x) In my blogs and our recent paper, we used "random search” to refer to the algorithm that treats a random finite difference as a gradient. (2/x)