Thread by @JL_Kroger

I'll summarize our study on the privacy implications of eye tracking, which was widely shared & discussed online over the last weeks 🥳 Time for a thread. #eyetracking #privacy #dataprotection #machinelearning #AI #inferences 1/19

Link to paper (open access): rd.springer.com/content/pdf/10…. In the paper, we present the results from sifting through a ton of literature and patents to answer one question: What categories of personal information can be inferred from eye-tracking (ET) data? 2/19

In short: recorded eye activity can reveal information about a user's sex, age, ethnicity, personality traits, drug-consumption habits, emotions, fears, skills, interests, sexual preferences, and physical and mental health. Details in the paper. Here are some examples: 3/19

Gaze abnormalities are a defining characteristic of various mental disorders, incl. depression & schizophrenia. Apart from detecting acute cases from ET data, researchers found biases in visual attention predictive of future depression scores at a delay of >2 years. 👀 4/19

Pupil dilation & spontaneous attention to specific stimuli can reveal a lot about a user’s phobias and aversions (e.g., fear of spiders/needles) but also about interests & preferences, incl. sexual preferences towards specific faces, age groups, body shapes & body parts. 5/19

ET can also allow inferences about a user's ethnic background, knowledge of certain cultural practices (e.g., eating with chopsticks) and native language (based on gaze patterns during reading). 6/19

As a basis for inferences, eye trackers capture not only the dispersion, duration, amplitude, acceleration, velocity and chronological sequence of eye movements … 7/19

… but often also the distance between eyelids, blink duration, blink frequency, ocular microtremors, pupil size, pupil reactivity, iris texture, facial expressions and facial attributes (e.g., skin color, eye shape, wrinkles). 8/19

Eye movements cannot only be tracked in head-mounted devices (e.g., VR headsets) but also through built-in front cameras in laptops, tablets and smartphones. The latter is less accurate, but these methods will quickly improve & make eye tracking ubiquitous in everyday life. 9/19

Many aspects of gaze behavior are not under volitional control (e.g., stimulus-driven glances, pupil dilation, ocular tremor, spontaneous blinks). Thus, it’s impossible for users to understand or control what information is revealed. 10/19

Drawing inferences from ET data is not trivial & the cited inference methods are not perfect. However, for many attacks and profiling purposes, 100% accuracy is not needed. Inaccurate methods will be used nonetheless, which can cause additional discriminatory side-effects. 11/19

Inference methods are mostly developed & deployed behind closed doors, subject to non-disclosure agreements. Based on R&D investments, some companies likely have far greater capabilities than what is known from published research. 12/19

ET has the potential to improve our lives in many ways. It is precisely the richness of gaze data that make the rising technology so valuable & useful. But to exploit this potential in a socially acceptable manner, adequate privacy protection is needed. 13/19

Existing technical & legal countermeasures are limited and don’t offer reliable protection against undesired inferences. Considering the rapid proliferation of eye tracking technology, more effective safeguards and means of enforcement are urgently needed. 14/19

Given the subject’s complexity, users cannot be expected to “defend themselves” or give truly “informed consent”, calling into question the prevalent legal paradigm of notice-and-consent (a.k.a. privacy self-management). 15/19

Since it is unlikely that companies will voluntarily refrain from using or selling personal information that can be extracted from already collected data, there should be strong regulatory incentives and controls. 16/19

Our paper received encouraging feedback, incl. 100s of tweets (e.g., @jordanbpeterson, @mserdark, @SteveStuWill), 100k downloads on Springer and numerous mentions across Facebook, Reddit, blogs, podcasts, policy reports and news outlets. altmetric.com/details/773833… 17/19

Of course, the threat of undesired inferences goes far beyond ET, encompassing countless other sensors and data sources. In other recent work, we have examined inferences from voice recordings (e.g., voice messages, voice memos, voice commands): rd.springer.com/content/pdf/10… ... 18/19

… and accelerometer data (dl.acm.org/doi/10.1145/33…). The accelerometer is the most widely used sensor in mobile devices and is commonly accessed by mobile apps without any request or notification to the user. 19/19