1) I just had a conversation with Mitsuku, "a record breaking five-time winner of the Loebner Prize Turing Test, [...] the world's best conversational chatbot." Five times? How could this fool anyone ONCE? This is what happens when you don't try to *test* something.
2) By contrast, THIS is what happens when you actually try to *test* it. 
3) Is AI a threat to testers? It's could eventually be a threat to testers who see their jobs as demonstrating that something works. It could be a threat to testers who are not prepared to look for problems, or to testers who don't consider what statistical analysis misses.
4) Against machine learning, test cases and formally scripted testing (do this, do this, observe that) is NOT going to cut the mustard. ML *by design* changes its outputs over time, especially when given new inputs. We'll need experiment and critical thinking about lots of stuff.
5) We'll need to evaluate training and validation data sets critically, looking for biases. We'll need to look at the outputs, too, and monitor carefully to see how they affect things in the world. We'll need to look at behaviour that changes in varying circumstances.
6) We'll need to design experiments that represent the real world in challenging ways; that present obstacles that the technology will have to deal with. We'll need to do that cleverly, instead of inflicting negligently tested and highly inadequate stuff on people.
7) Testing specialists will need to develop rapid learning approaches to acquire interactional expertise in new domains. We'll need to develop skill in anthropology, sociology, technology, and all the domains in which dazzled, googly-eyed, credulous people will try to apply AI.
8) We'll need to develop deep skepticism; strong knowledge, understanding, and application of ethics; bravery to blow whistles from time to time. We'll need to examine and articulate risk, starting with fundamental questions: "Is there a problem here? Could there be? For whom?"
9) As both @atscmc and @FionaCCharles emphasized in their #EuroSTARConf keynotes, we'll need to peer past the hype and the snake oil salespeople, and ask "What could possibly go wrong?" Because money times hype times uncritical buy-in ALWAYS goes wrong in the end.
@atscmc @FionaCCharles 10) Test cases are going to be utterly helpless and worthless in the face of AI. Getting machines to push buttons (GeMPuB) will pass to the machines. But that's basically okay, because skilled testing was never about that stuff anyway. It's about the things I've mentioned above.
