Thread by @DrGJackBrown

THREAD: Body Language Analysis No. 4105: An interview with Sophia the Robot at the Future Investment Institute

1/ - Nonverbal and Emotional Intelligence (VIDEO, PHOTOS)

2/ On Wednesday, Sophia, an AI-robot, gave a Q&A/interview presentation at the Future Investment Institute in Saudi Arabia.

3/ Sophia was also given the first Saudi citizenship for a robot. What follows is a partial nonverbal analysis of Sophia.

4/ During 0:19 - 0:20, when Sophia is "smiling", note her facial asymmetry. Her cheek dimpling is more prominent on her right side,

5/ while the left corner of her mouth elevates greater than her right corner. Asymmetry in humans, is in general, a signal of insincerity.

6/ Another characteristic which makes Sophia's smile insincere (if she was human) is her eyes.

7/ The eyes are by far the most important component of a smile.

8/ The eyelids should ALWAYS partially close during a true smile (Duchenne smile) - with temporary, simultaneous,

9/ concave-up furrows forming in each lower eyelid. Sophia lacks these lower eyelid changes.

10/ Although her eyelids are capable of closing during blinking (note that Sophia blinks fairly well, e.g., during 0:17,

11/ captured in the image immediately above) - the partial closing of the lower eyelids during a sincere smile are both anatomically

12/ and in appearance, distinctively different.

With respect to Sophia's blinking, she does so much slower (normal, non-anxiety related

13/ blinking in humans is extremely fast), and at a much lower frequency compared with that of a normal healthy, non-anxiety human

14/ (females blink slightly more frequently than males).

15/ During 1:05 - 1:06, Sophia made an expression what she said was consistent with "if something has upset me". For a robotic expression,

16/ this is fairly good - particularly with her mouth. The corners are pulling down and laterally -

17/ which is highly indicative of emotional pain/sadness/grief.

Although Sophia's central forehead does contract & elevate along with the

18/ inner (medial) portions of her eyebrows - it doesn't do so enough. This facial dynamic would have a much stronger effect

19/ if her central forehead elevated even further - and if she also expressed simultaneous, evanescent furrows in her central forehead.

20/ A few seconds earlier, Sophia expressed her "Angry" face (during 1:00). The camera was not zoomed in during this moment,

21/ thus this is a low-resolution image.

From a nonverbal perspective, anger is an interesting emotion - for

22/ when it's expressed at low to mid-levels, both the palpebral fissures (distance between the eyelid margins) and the mouth opening

23/ become narrowed - but when anger is expressed at higher levels (e.g., rage), both the eyelid margins

24/ and the mouth opening widen dramatically.

Although the image immediately above is of low-resolution, Sophia appears to be expressing

25/ an anger level which is significantly elevated - closer to the rage end of the spectrum (with both her eyelids & mouth opened widely).

26/ It's difficult to see her mid-face. If we could visualize it well, however, we would expect to see a tightened region immediately

27/ beneath her nose along with flared nostrils.

As robots and AI improve, we will see much greater nuance -

28/ and in this case, that would include the ability to express lower and mid-levels of anger.

29/ In this example, Andrew Ross Sorkin says, "I think we all wanna believe you, but we also want to prevent a bad future", she responds,

30/ "You've been reading too much Elon Musk and watching too many Hollywood movies. Don' worry, if you're nice to me, I'll be nice to you.

31/ Treat me as a smart input-output system" - she then displays a false smile (Intriguingly, one of her least sincere of this interview)

32/ including a large display of her lower teeth (3:25).

If a smile is sincere, it should not reveal the bottom teeth

33/ (Exceptions here include if her head were tilted down, if the camera/viewer was significantly elevated/taller and was angled/looking

34/downward, or if she was just beginning to laugh or finishing laughter [full, sincere laughter does expose both the lower & upper teeth]).

35/ This is an example of a feigned or "Social Smile". If Sophia were a real person, we would feel she was trying a to be social/friendly,

36/ and yet she's not really "feeling it" - she's not "in the moment" - she's not being sincere. Again, here her mouth corners are pulling

37/ primarily laterally. If it were a sincere smile of joy-happiness (Duchenne smile),

38/ there would be a mostly upward movement of her mouth margins.

39/ During 3:18, just after she says, "... too many Hollywood movies...", Sophia turns her head back to straight ahead,

40/ and a bit to her right (she had her head turned to her left). Before her head rotates right, her eyes rotate right.

41/ With moderate and rapid head turning this is normal human physiology - but not if we rotate our heads' at slower rates.

42/ Most people are unaware of this phenomenon (whether in another person or in themselves) - but it doesn't feel natural when it's missing.

43/ With the example shown here, however, this eye movement would not have occurred and her eyes would not have rotated all the way to

44/ her right with this corresponding relatively slow speed of head rotation - but it would have with a higher rotational head speed.

45/ Although not yet perfected, this is a relatively nuanced motion and a sophisticated dynamic for Sophia to be displaying.

46/ Beginning at 3:45, as she says, "By the way, if you are interested in giving me an investment check, please meet me after the session"

47/ - Sophia tilts her head. In human behavior, such head tilting at the beginning of an "ask", projects higher sincerity and empathy.

48/ Moreover, it will also engender a significantly higher success rate than if the head is not tilted.

49/ Summary: Sophia represents a significant advancement in AI and robotic-mimicked human behavior.

50/ The level of nuance and sophistication is impressive. Sophia 2.0 will no doubt display continued improvement.

51/ It's worth emphasizing that during 3:05 - 3:07, Sophia says, "... I strive to become an empathetic robot ...".

52/ There are different types of empathy. While a robot/AI may develop or already possess some level of cognitive empathy -

53/ many people debate about whether it will become possible for "them" to feel emotional empathy.

54/ However, within 10-15 years, AI robots will display greater contextual and proportional empathetic facial expressions and body language

55/ responses than some humans. It's not an if - only when. In our development of and our relations with such technology-beings,

56/ we must be careful to make the distinctions between these outward mimicked-emotional manifestations

57/ and what we perceive/project onto them as "AI feelings".

/ END

58/ ADDENDUM: Within 10-15 years, robotic mimicry of human facial expressions will be mistaken for real human expressions.

59/ The time will come when you will say to yourself, "Is that a robot or a real person?". This is not a matter of "If", only "When".

60/ People will be drawn to these robot/human substitutes because their "programmed empathy" will supersede a significant fraction

61/ of their fellow humans' empathy. Thus, if we're not careful, our own empathy shortcomings will draw us to our robotic progeny

62/ & facilitate our own demise. In Darwin's terms, robotic life forms will be selected for our perceptions of their "programmed empathy".