Thread by @qiaosi_wang

What can we draw from human-human interactions to design more natural human-AI interactions? With many AI agents deployed in online communities, what does the community think of the agent? Can language reflect those perceptions?

A🧵about our CHI 2021 paper #CHI2021 (1/n, n=16)

(2/n) In human-human interactions, we are able to constantly monitor how we are perceived by other people through behavioral and verbal cues. Based on that understanding, we can adjust our behaviors accordingly to align others' perceptions about us closer to our self-presentation

(3/n)Some researchers have suggested that this ability is based on a uniquely human characteristic called "Theory of Mind (ToM)." ToM enables us to make conjectures about each others' minds through behavioral cues.

(4/n) Having a Mutual Theory of Mind (MToM) in human-human interactions helps us build a shared expectation of each other to maintain constructive and coherent conversations.

Can we build a ToM into conversational agents to achieve MToM in human-AI interactions?

(5/n) To explore this question, we first need to understand how the community's perception of an AI agent change over time and is it feasible to automatically infer others' perceptions of an AI agent through linguistic cues.

(6/n) In our paper, we focused on community-facing conversational agents. Our study took place in an online learning setting @GTOMSCS. We deployed a virtual teaching assistant named Jill Watson on the class discussion forum to answer students' logistic questions about the class.

(7/n) We measured students' perceptions of Jill through bi-weekly surveys and extracted linguistic features from students' interaction with Jill. Survey measurements and linguistic features extracted are all inspired by the MToM framework.

(8/n) To understand the changes in community perception of Jill, we deployed bi-weekly surveys to measure Jill's perceived human-likeness, likeability, and intelligence. We found that perceptions significantly changed in human-likeness and intelligence, but not likability.

(9/n) We also built linear regression model to see if we can automatically recognize student community's perception of Jill through some linguistic features extracted from the questions students posted to Jill (e.g., sentiment, readability).

(10/n) Our regression analyses show that some linguistic attributes such as verbosity and linguistic diversity can reflect students' perception of Jill.

(11/n) In the paper, we discussed in detail the implications on how to use language analysis to design human-AI interactions, adaptive community-facing agents, and MToM as a framework for human-AI interactions.

(12/n)I want to thank @GTOMSCS students for always being so nice and supportive of all my research so far--- it has been a great pleasure to working with all of you on so many of my projects. I hope my research is making a positive difference in your online learning experience😀

(13/n)And of course this paper would be impossible without the support from my co-authors @kous2v, Eric Gregori, @DrDavidJoyner, @AshKGoel. Also grateful for the feedback and help from my friends and colleagues at @ICatGT : @VedantNeedMoEdu, @dongwhi_yoo, @ella_es_adriana, etc.

(14/n)Thank you all for always being patient and supportive while I went through my emotional roller coaster during my painful yet fulfilling CHI writing process.

(15/n)Here is a short video about our paper:
A @gtcomputing piece on our paper: chi2021.cc.gatech.edu/better-than-a-…

ACM DL: dl.acm.org/doi/10.1145/34…
PDF: qiaosiwang.me/Publications/M…

(16/n) Looking forward to the live Q&A sessions at the computational human-AI conversation sessions (time in JST) at #CHI2021 🤩

-- end of 🧵--