, 12 tweets, 4 min read Read on Twitter
As part of my resolutions for 2019, I want to think more critically about the papers I'm reading. I'll start with "When Will AI Exceed Human Performance? Evidence from AI Experts" (arxiv.org/abs/1705.08807).

#AI #predictions #paperreview
The paper "When Will AI Exceed Human Performance? Evidence from AI Experts" analyses predictions about progress in AI from researchers who published at the 2015 NIPS and ICML conferences. Only a quick survey was used to gather the data.

arxiv.org/abs/1705.08807
A quick survey does not evoke thoughtful forecasts. It provokes fast thinking and sampling from intuition. Daniel Kahneman's "Thinking fast and slow" provides ample examples of how fast thinking is skewed by bias.

amazon.co.uk/Thinking-Fast-…
The paper even admits that different framing of the same question elicited vastly different predictions: the predicted time to full automation doubled depending on whether it was framed to include consequences to the labour market or not.
AI researchers were more reluctant to predict full automation in the near future when the potential negative effects were on their minds.
The prediction of when we will automate the work of a surgeon also varied significantly between Europa and the US. This might tell us more about how much value people in different regions assign to surgeons than about the difficulty of automating the tasks they are performing.
To get meaningful estimates, we need to engage participants in slow thinking.

Participants could enumerate the biggest unsolved challenges related to each task before making their prediction, or they could answer what the biggest risks to their predictions are.
This could provide more interesting results.

For example, one challenge is that Moore's law seems to be petering out. Roughly, it predicts that computer performance doubles every 18 months. However, it has been become difficult to keep it up.
Without it, most predictions will be off by a wide margin.

But, in general, why should we trust predictions of people involved in the field now---especially, when AI has become such a hyped field again?
In the 50s and 60s, during the first boom cycle, AI researchers boldly predicted that artificial general intelligence was around the corner. They predicted that computer vision could be solved over summer by an intern.
It took another 50 years to solve the initial questions in computer visions reliably.

This paper provides interesting predictions. Caution is advised, though, on how seriously to take them.
Other reviews: technologyreview.com/s/607970/exper…

Please let me know what you think.
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Andreas Kirsch
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member and get exclusive features!

Premium member ($3.00/month or $30.00/year)

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!