Thread by @oulasvirta

Rant: Nine reasons why I don't believe in current VR/AR technology.

HoloLens, Magic Leap, and Oculus: Mind-blowing videos, and the market is estimated to explode to $200 billion by 2025 (Statista). So what's wrong?

HCI research tells why we haven't seen a killer app yet: 1/22

First, the gorilla arm. The videos show smiling users holding their arms up for extended periods. But that will cause shoulder pain. research.aalto.fi/files/18129437… The Consumed Endurance model estimates that a user can hold arm up for just 90 seconds before starting to fatigue. 2/22

Anecdotally, when @drsrinathsridha was working on a CHI paper, he could not find a comfortable posture for doing freehand gestures and had days of sore neck and arm. Well, you could try "gun slinging, but then you lose hand-to-display coupling & fall back to mouse-like input 3/22

So, either we lose hand input or we're limited to applications that that don't need more than 90 seconds of "air time".

You could also lay on the floor like this user: 4/22

Second, our hands have evolved for manipulating objects, not for poking in the air.
amazon.com/gp/product/069… 5/22

While vision-based tracking of hand movement has taken leaps forward, tracking of hands WITH objects hasn't. Here's a screencap from our ECCV paper from a few years back. Tracking works if you're slow and occlusions are bearable
handtracker.mpi-inf.mpg.de/projects/Realt… 6/22

So, it seems you can forget interactions with (arbitrary) physical objects. We're going to be designing poking-in-the-air applications for years to come.
cnet.com/news/microsoft… 7/22

Third, poking-in-the-air is also inferior as a means of interaction. Mechanoreceptive feedback is specialized and important for input. We don't want to lose it. Even pressing a button or hitting a virtual ball becomes hard. See our ICMI'17 paper: research.aalto.fi/files/21407685… 8/22

Well, you could try 3 things: 1) forget applications where contact is needed (boo!), 2) wear gloves (clumsy, unhygienic), or 3) assume an instrumented environment. Ultrahaptics - as far as I have tried it - is too weak to replace real contact.

Users will miss real buttons. 9/22

Fourth, gesturing is not "natural". There are few if any instinctual gestures. HCI research has sought for natural gestures for a decade, but found out that the gesture elicitation method artificially inflates claimed consensus:
hal.archives-ouvertes.fr/hal-01788775/d… 10/22

So, either you forget "natural interaction", and assume that your users are willing to spend time learning new gestures, or you're stuck with simple pointing-and-pinching type gestures that users can transfer from the mobile device. 11/22

Fifth, body misalignment. This is my fav: The virtual and the physical body will never be perfectly aligned in time and space. The sensing+computation pipeline that mediates motion and display can only worsen alignment. Try petting a cat with Leap Motion. Ouch. 12/22

This is a serious issue for motor control, causing coordinate disturbance, temporal asynchrony, and poorer cue integration. Users cannot rely on their senses as they normally do when moving. Imagine tracking a bee in 3D space: Hard IRL, harder in VR. 13/22

Sixth, say goodbye to text entry. Unless you fall back to in-hand controllers, you can forget applications that require fast and precise motor control. Entering text is very prevalent in computer use but practically unusable with mid-air input. 14/22

Example: VULTURE is a gestural mid-air text entry technique. A CHI Best Paper. But how fast is it? Meagre 21 words per minute. kasperhornbaek.dk/papers/CHI2014… Adults type around 35-40wpm on smartphones. Can you imagine their reaction when they're dumbed down to half of that? 15/22

Well, you could design a new open-loop gesture set, like a sign language. We tried exactly that (CHI'15), predicting that you might get up to 50 WPM with simple gestures and LeapMotion-level sensing handtracker.mpi-inf.mpg.de/projects/HandD… Downside? You need to practice around 10-20 hours 16/22

Seventh, simulator sickness, a long-standing problem, still unsolved. This is what NASA wrote about its VR adventure in the 1990s: "The only thing consistently real about VR were headaches and motion sickness"
science.nasa.gov/science-news/s… 17/22

But surely we have solved motion sickness by now with better hardware? No. Depending on task, up to 56% of participants felt motion sickness with Oculus Rift in a recent study:
link.springer.com/article/10.100… 18/22

Eighth, locomotion: also still unsolved, at least for customer-grade devices. How to move around comfortably when you don't have a treadmill or full body tracking? By throwing yourself, pulling, teleporting or what? Poor replacements for just walking uploadvr.com/oculus-perfect… 19/22

Ninth, another persisting problem still with us: the narrow field of view. HoloLens had a notoriously narrow FOV "like a card deck".

When the HMD's FOV is narrower than your (real) peripheral vision, you lose visual context. Users need to turn and glance around to orient. 20/22

But peripheral vision is actually important in many visual tasks. NASA reported that pilots did better with traditional displays than with VR because of this. science.nasa.gov/science-news/s… 21/22

To sum up, definite progress has been made, but the scope of today's VR/AR technology is narrow. It is locked into applications where a few seconds of waving in the air is worth the hassle of putting on an HMD. 22/22

Thanks for reading and thanks to @drsrinathsridha for pointers and starting the discussion

Thank you for overwhelming feedback.🙏 100K views and 3K comments/likes. I am considering writing this up as a "proper" article