Thread by @rao2z

There seems to be an almost willful confusion about the need and role for explainability of #AI systems on #AI twitter.

Contrary to the often polarizing positions, it is neither the case that we always need explanations nor is it the case that we never need explanations. 🧵1/

We look for explanations of high level decisions of (what for us are) explicit knowledge tasks; and where contestability and collaboration are important.

We rarely look for explanations of tacit knowledge/low level control decisions. 2/

I don't need explanation on why you see a dog in a picture; why you put your left foot 3 mm ahead of your left, or why facebook recommends me yet another page.

I do want one if am denied a loan, or I need a better model of you so I can coordinate with you. 3/

Trust can reduce the need for explanations, but trust has to be earned and can't always be legislated. I ask explanations from my doctor despite the many impressive degrees on his wall. 4/

Explainability doesn't necessarily require full understanding on the part of the receiver. I ask my doctor for explanations of his diagnosis, despite the fact that I don't quite understand all the details. 5/

Explanations are always about the receiver's mind (..and vocabulary and reasoning..). After all, a doctor explains her decision in different ways to the patient and her colleague..

(This also explains the popularity of those viral @wired 5 levels of explanations videos!) 6/

It also doesn't matter whether your #AI system uses symbolic or sub-symbolic reasoning--it's explanation to the humans in the loop should rightfully be on their terms. This is the compelling reason why #AI systems need to afford a symbolic interface.. 7/

https://twitter.com/rao2z/status/1357785690394615812

We should stop conflating Interpretability and Explainability. Interpretability is w.r.t. a large population (e.g. human race) and explainability is w.r.t. individuals/specific groups.

The interpreter does the heavy lifting in the former; the explainer does it in the latter. 8/

The Rosetta Stone is interpretable to the human race (thanks to the heavy lifting by Young and Champollion).

I want my loan approval decision be explainable to me without me having to break my back.. 9/

An autonomous car's control decisions need to be interpretable to the (determined) investigators after a crash, its decision to take a particular route should be easily explainable to the rider in the car. 10/

Explanations are sought after the fact--and thus do allow post hoc rationalizations.

Fear of this possibility shouldn't make them irrelevant!

We should design #AI systems to provide "truthful" explanations, but the comprehensibility may well necessitate approximation. 11/

Explanations are most definitely not a soliloquy. A direct trace of your or the #AI system's internal reasoning rarely makes for a good explanation--given the differing mental models, vocabulary and inferential abilities of the humans receiving the explanation. 12/

[fwiw, a lot of the above is related to our ongoing research on explainable human-#AI interaction. If you are interested:

👉A monograph: bit.ly/3ysF22Z

👉A #AAAI2022 blue sky paper: bit.ly/3IGdsD6

👉A breezy talk: bit.ly/35uZt17 ] #XAI 13/

The popular "saliency region" painting explanations are a comforting soliloquy that is neither sensitive to the receiver nor scalable beyond shared images; sequential decisions would require annotating space-time signal tubes!

Humans evolved symbolic explanations instead.. 14/