Thread by @nicklockwood

The problem with the Turing test is that it hinges on the ability to fool a human, but humans have an innate desire to be fooled.

That’s why true scientific experiments rely on “double blind” conditions, so the experimenter’s own credulity is eliminated as a factor in the result

If we look at that Google AI demo, there were several remarkable technical advances on display, but what was the thing that impressed the audience most?

It was the fact that the AI said “um” between sentences. A cheap trick that required no advances in AI whatsoever.

I thought that demo was remarkable, but not for the reasons many people seem to.

It’s not clear to me that the tech on display has anything to do with the Turing test at all.

It has never been a condition of the Turing test that the machine can understand or synthesise speech.

The Turing test, as originally envisaged, required participants to communicate via a text terminal.

That was probably because the speech tech we have now seemed impossibly far off for Turing, but the terminal interface actually serves a valuable purpose - it removes distraction.

Speech is distracting. It’s distracting if it’s not done perfectly, but it’s even more distracting if it *is*.

We know that speech recognition and synthesis doesn’t require true AI, but it’s so damn impressive that it makes us forget what we’re supposed to be measuring.

If you remove speech from that Google demo, what’s left?

A chatbot that can ask a very narrow question and understand a slightly garbled response.

That is impressive, but we’ve had chatbots for a while, and it’s not clear from that brief demo that this is a big leap forward.

If we want to talk about cool (possibly rather creepy) advances in machines emulating human speech, the Google chatbot saying “um” is clearly a game changer.

If we want to talk about meaningful advances in artificial general intelligence, that demo was mostly smoke and mirrors.