,
12 tweets,
4 min read
Read on Twitter
As part of my resolutions for 2019, I want to think more critically about the papers I'm reading. I'll start with "When Will AI Exceed Human Performance? Evidence from AI Experts" (arxiv.org/abs/1705.08807).
#AI #predictions #paperreview
#AI #predictions #paperreview
The paper "When Will AI Exceed Human Performance? Evidence from AI Experts" analyses predictions about progress in AI from researchers who published at the 2015 NIPS and ICML conferences. Only a quick survey was used to gather the data.
arxiv.org/abs/1705.08807
arxiv.org/abs/1705.08807
A quick survey does not evoke thoughtful forecasts. It provokes fast thinking and sampling from intuition. Daniel Kahneman's "Thinking fast and slow" provides ample examples of how fast thinking is skewed by bias.
amazon.co.uk/Thinking-Fast-…
amazon.co.uk/Thinking-Fast-…
The paper even admits that different framing of the same question elicited vastly different predictions: the predicted time to full automation doubled depending on whether it was framed to include consequences to the labour market or not.
AI researchers were more reluctant to predict full automation in the near future when the potential negative effects were on their minds.
The prediction of when we will automate the work of a surgeon also varied significantly between Europa and the US. This might tell us more about how much value people in different regions assign to surgeons than about the difficulty of automating the tasks they are performing.
To get meaningful estimates, we need to engage participants in slow thinking.
Participants could enumerate the biggest unsolved challenges related to each task before making their prediction, or they could answer what the biggest risks to their predictions are.
Participants could enumerate the biggest unsolved challenges related to each task before making their prediction, or they could answer what the biggest risks to their predictions are.
This could provide more interesting results.
For example, one challenge is that Moore's law seems to be petering out. Roughly, it predicts that computer performance doubles every 18 months. However, it has been become difficult to keep it up.
For example, one challenge is that Moore's law seems to be petering out. Roughly, it predicts that computer performance doubles every 18 months. However, it has been become difficult to keep it up.
Without it, most predictions will be off by a wide margin.
But, in general, why should we trust predictions of people involved in the field now---especially, when AI has become such a hyped field again?
But, in general, why should we trust predictions of people involved in the field now---especially, when AI has become such a hyped field again?
In the 50s and 60s, during the first boom cycle, AI researchers boldly predicted that artificial general intelligence was around the corner. They predicted that computer vision could be solved over summer by an intern.
It took another 50 years to solve the initial questions in computer visions reliably.
This paper provides interesting predictions. Caution is advised, though, on how seriously to take them.
This paper provides interesting predictions. Caution is advised, though, on how seriously to take them.
