Looking forward to the keynote by @khornbaek “Implications for theory” at #NordiCHI2022
Reminds me of my PhD, which I had the pleasure to defend with Kasper in the jury in 2015.
Reminds me of my PhD, which I had the pleasure to defend with Kasper in the jury in 2015.
@khornbaek’s keynote argument:
1. HCI engages in too limited ways with theory
2. This has negative consequences for our field
3. We can do something about it
Let’s do more impactful research, by paying more attention to theory. #NordiCHI2022 @NordiCHI2022
1. HCI engages in too limited ways with theory
2. This has negative consequences for our field
3. We can do something about it
Let’s do more impactful research, by paying more attention to theory. #NordiCHI2022 @NordiCHI2022
What is theory anyway?
“Scientifically acceptable general principle or body of principles offered to explain phenomena” (merriam-Webster)
A tool that we construct to help make sense of the world.
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
“Scientifically acceptable general principle or body of principles offered to explain phenomena” (merriam-Webster)
A tool that we construct to help make sense of the world.
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
What is a theory in HCI?
At @acm_chi and @NordiCHI2022 theories typically include the following:
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
At @acm_chi and @NordiCHI2022 theories typically include the following:
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
What could we do with theory?
Theory is useful to:
- Predict: foresee outcomes of behaviors
- Understanding: explain the mechanisms or the processes involved in behaviors
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
Theory is useful to:
- Predict: foresee outcomes of behaviors
- Understanding: explain the mechanisms or the processes involved in behaviors
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
For instance, at @NordiCHI2022, there are 98 papers, and only 9 occurrences of “theor*” in 5 papers.
@khornbaek looked more systematically at @acm_chi 2017 best papers (N=25).
@khornbaek looked more systematically at @acm_chi 2017 best papers (N=25).
Data from @acm_chi CHI2021
743 papers published, only 243 used some form of theory
Among 100 of those, 19 had some indications of “implications for theory” (example of what that means / is formulated on the picture)
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
743 papers published, only 243 used some form of theory
Among 100 of those, 19 had some indications of “implications for theory” (example of what that means / is formulated on the picture)
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
According to @khornbaek (keynote #NordiCHI2022) a section “implications on theory” is one of the many ways theories can play a role in a paper.
From CHI 2021, we see the use of theory:
- as background
- for discussion
- as analysis tool
- as methodological tool
- for design
From CHI 2021, we see the use of theory:
- as background
- for discussion
- as analysis tool
- as methodological tool
- for design
Some authors used keywords analysis to argue that HCI lack “motor themes” (which should be at the core of a discipline). There are no themes that people return to over and over again and these authors argue this is a big gap in HCI
@khornbaek keynote #NordiCHI2022
@khornbaek keynote #NordiCHI2022
Lack of replications
The repliCHI initiative (glad I was part of, kudos to @drmaxlwilson !) has highlighted the need.
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
The repliCHI initiative (glad I was part of, kudos to @drmaxlwilson !) has highlighted the need.
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
But… we might think… we have theories on usability for instance?
Mostly taxonomies of dimensions actually, even after 40 years or work on the topic. That’s embarrassing to admit.
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
Mostly taxonomies of dimensions actually, even after 40 years or work on the topic. That’s embarrassing to admit.
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
What about interaction?
We do have a definition yes, but what else do we actually know?
“The term interaction is field defining, yet surprisingly confused”
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
We do have a definition yes, but what else do we actually know?
“The term interaction is field defining, yet surprisingly confused”
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
The lack of theories has negative consequences for our field. why don’t people do theory work? @khornbaek (keynote #NordiCHI2022)
There’s a belief that “all good things come to those who iterate” (“let’s get real” Landauer 1991). So why bother with theory when we can A/B test?
There’s a belief that “all good things come to those who iterate” (“let’s get real” Landauer 1991). So why bother with theory when we can A/B test?
The inductive hamster wheel
- No generalization or discovery that we already know appli to new situations.
Theoretical power posing
- we use theory to show position and intellectual heritage
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
- No generalization or discovery that we already know appli to new situations.
Theoretical power posing
- we use theory to show position and intellectual heritage
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
Putting a hat on a horse
- theory plays a minor role relative to data or is an afterthought in a discussion
The trouble with triangles
- people, tasks and tools are intertwined (@professorBodker 1987)
Thus we need to start from scratch if users, tasks or tech changes
- theory plays a minor role relative to data or is an afterthought in a discussion
The trouble with triangles
- people, tasks and tools are intertwined (@professorBodker 1987)
Thus we need to start from scratch if users, tasks or tech changes
Now that we saw some reasons why people do not use theory much, a few tips from @khornbaek #NordiCHI2022 about how to use theory more.
1️⃣ Reason with theory about new situations
2️⃣ Form expectations before analysis
3️⃣ focus on why’s and mechanisms
1️⃣ Reason with theory about new situations
2️⃣ Form expectations before analysis
3️⃣ focus on why’s and mechanisms
Test theory
Dubin (1978) argues that the job of researchers is to test theories.
> what is it that you test?
> casual or positioning use of theory is not the right focus
Look for forks in the tree or research.
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
Dubin (1978) argues that the job of researchers is to test theories.
> what is it that you test?
> casual or positioning use of theory is not the right focus
Look for forks in the tree or research.
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
BUILD THEORY
Date to make claims and be wrong
> most findings in HCI are not even wrong or capable of “being wrong”
Theory building with a lower case t
- use Whetten’s 1989 Ws
Let us tackle the key issue in Interactive behavior!
@khornbaek keynote #NordiCHI2022 @NordiCHI2022
Date to make claims and be wrong
> most findings in HCI are not even wrong or capable of “being wrong”
Theory building with a lower case t
- use Whetten’s 1989 Ws
Let us tackle the key issue in Interactive behavior!
@khornbaek keynote #NordiCHI2022 @NordiCHI2022
EMBRACE THEORY PLURALISM
Methodological pluralism is well understood.
Stop using theories for positioning or identity, have a scout mindset, compare theories
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
Methodological pluralism is well understood.
Stop using theories for positioning or identity, have a scout mindset, compare theories
@khornbaek keynote at #NordiCHI2022 @NordiCHI2022
CREATE IMPLICATIONS FOR THEORY
Let’s not be indifferent to implications for theory! Let’s do better 💡
That’s a wrap! Thanks @khornbaek keynote at #NordiCHI2022 @NordiCHI2022
Let’s not be indifferent to implications for theory! Let’s do better 💡
That’s a wrap! Thanks @khornbaek keynote at #NordiCHI2022 @NordiCHI2022
Q&A keynote @khornbaek #NordiCHI2022
Impression that @khornbaek did not tell us how theories can actually play a role
> would be nice if theories would play a larger generative role. There is a role for an understanding of people in thinking about design.
Impression that @khornbaek did not tell us how theories can actually play a role
> would be nice if theories would play a larger generative role. There is a role for an understanding of people in thinking about design.
Q&A keynote @khornbaek #NordiCHI2022
What you didn’t say is that HCI steal or borrow/adapt from other fields, we get a lot from other traditions and try to apply them. The deeply you dig, one realizes it’s very hard. We’re not even part in the discussions in the original field.
What you didn’t say is that HCI steal or borrow/adapt from other fields, we get a lot from other traditions and try to apply them. The deeply you dig, one realizes it’s very hard. We’re not even part in the discussions in the original field.
So we use theories the way they were used in these fields 20 years ago or more. It’s indeed a lot of work to find out which are very relevant to us or less. Examples of self determination theory or activity theories.
There is a lot out there we can use, if we do the translation.
There is a lot out there we can use, if we do the translation.
Another question to @khornbaek #NordiCHI2022
Disagrees with the arguments about the use of theories. Theory is very much saturated in HCI…
> @khornbaek replies: what’s the question? 😂
Disagrees with the arguments about the use of theories. Theory is very much saturated in HCI…
> @khornbaek replies: what’s the question? 😂
Q&A keynote @khornbaek #NordiCHI2022
Question: People might use a lot of theories that are not labeled as theories. It would be interesting to formulate our own HCI theories.
Perhaps wrong to look at @acm_chi papers, rather looking at theories in the PhD theses.
Question: People might use a lot of theories that are not labeled as theories. It would be interesting to formulate our own HCI theories.
Perhaps wrong to look at @acm_chi papers, rather looking at theories in the PhD theses.
Q&A keynote @khornbaek #NordiCHI2022
Answer: explains the reasons why looking at #CHI2021 using a keyword based approach, agrees that one can look at journal or theses but if we can’t talk about theories at our flagship conference there is something wrong.
Answer: explains the reasons why looking at #CHI2021 using a keyword based approach, agrees that one can look at journal or theses but if we can’t talk about theories at our flagship conference there is something wrong.
Q&A keynote @khornbaek #NordiCHI2022
Comment: don’t think we’re stealing theories. In psychology what they use the theories for in this domain is very different than what we aim at. Would be interesting to see how to better use theories in HCI.
Comment: don’t think we’re stealing theories. In psychology what they use the theories for in this domain is very different than what we aim at. Would be interesting to see how to better use theories in HCI.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
