Thread by @HIThomasFranke

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP Sometimes I have the feeling that we scientists find it difficult to detach ourselves from the concepts of logic when it comes to theorising about explanations. Already the label of 'explainable' AI is difficult here (I suppose we all see the weaknesses in this conceptualisation)

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP Then we start structuring the conceptual space around explanations in XAI with labels/concepts such as counterfactuals, diagnosticity, ...
Sometimes I think, we have to step back and think about what we actually want to achieve here:

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP Isn't XAI an act of communication - a system (typically an AI agent, maybe also indirectly the AI developer who wants to communicate/highlight/... a certain idea or feature of the AI sydtem) that wants to make itself understood to another system (typically a human agent).

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP And that means we have to root HCXAI even more deeply in the theorizing on human communication (that is: human-human). So in the end: aren't we parents to a new generation of technology that wants to make itself understood - same as in most human-human relationships?

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP And we are just now trying to invent a universal language for forming meaningful (and effective, efficient and satisfying) relationships between such 'intelligemt' agents and humans?
That said: in my view this task is long overdue because it does not just become relevant when ...

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP ... when we face AI systems but any automation of information processing and action-regulation (sure - to a different degree as interaction with many non-AI systems my work well based on the 'explanations' that users come up with based on system interaction (or reading) - ...

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP ...i.e. the mental models users construct. To cut a long story short my feeling is:
(1) the grammar of XAI has yet to be developed and
(2) a deep dive in human-human communication might help
Sorry for the long-winded argument and de-railing the discussion a bit!

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP Ah - what I missed to write in the first place: our cognitive bias as scientists may be "explanations = this cool end product that we are trying to achieve within the process of scientific discover to reduce the opacity of the world around us; so to be useful...

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP ...we need to find the most parsimonious explanation that explains as much as possible variance in the data & explicates the causal chains behind (or causal networks around) certain phenomena. And while this epistemological view on finding the optimal explanation is important ...

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP ...it is only half of the story, because we want to explain ourselves (or the machines we invented / or we want to teach our machines this ability) to humans - not scientists. So we have to integrate very well with human information processing - and even more so ...

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP ... with human action-regulation (so e.g. with bounded rationality in human perception of environmental dynamics & decision making). I think this is also the reason why I think that we have to be very careful in selecting (or developing) the metrics that we use ...

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP ... for evaluating the quality of explanations in XAI working towards #HCXAI. Firs of all there is no other approach then centering XAI benchmark metrics around actual subjective user experience indicators (maybe I am wrong here) ...

@mark_riedl @TimSchrills @UpolEhsan @MarisaTPP ... and second: it is all about variables that influence human action regulation (i.e. most closely align to how humans control their goal-directed behavior and what they experience in the process).
So, I really should stop writing for now - sorry for the many/long comments.