Thread by @guyhoffman

After spending the last three days as a judge for the @xprize #robotics AVATAR, here are some thoughts on the state of #robot telepresence. A 🤖 🧵

This XPRIZE challenge is for teams to build robot and operator systems that would enable a human operator to control a remote robotic avatar through a number of tasks that prove that it can see, hear, feel, and interact at a distance.

Teams had 1 hour to train an operator judge on their system and then had to sit back to watch their robot go through 10 challenges. For more details: avatar.xprize.org/prizes/avatar

Many of the contenders had good-enough robotics technology. In the end it all came down to the user experience and operator-interaction. All of the top-scoring teams focused on systems that had a very low-cognitive-effort and natural interface.

The top-scoring system (NimbRo from Bonn University in Germany nimbro.net/AVATAR/) had some serious technology, especially on the operator side. They basically built a full replica of the robotic arms for the operator to manipulate

Operators reported that there was no "mapping". Whatever they did on the control arm happened on the robot. They reported feeling that they felt they "held on" to the remote robot's arms.

They also had an amazing 6DoF arm holding the camera, enabling the operator to view the scene from many angles. Their UI was also well honed: when the robot backed up, a rear camera was displayed, and then hidden again when not in use.

But you didn't need two full robotic arms and a custom-built foot interface. Second place, @pollenrobotics sent a tiny team, and hardly used any custom hardware for the operator. It was all off-the-shelf VR hardware with a bit of hacking.

Their robot was also mostly 3D printed, low cost and elegant. But their interaction software was amazing, thoughtful, clean and simple, and adaptive to the context. When the robot was driving, the LIDAR map grew to be more visible and automatically shrunk in manipulation mode.

Operators said that the robot just "did what you wanted them to do", hiding the complexity of robotics behind smart software and a simple interface.

One thing that was suspiciously missing was any kind of shared autonomy. Operators did a lot of things "manually" that would have better been handed over to an autonomous subroutine. There was no waypoint navigation or grasping routines.

A team that would have used better hybrid control would have done much better IMHO. If this competition ever repeats, I would guess that there would be a lot of shared autonomy for better remote presence.

Finally, it was depressing to see the lack of diversity in the teams. From what I could see, out of the 17 finalists, a shocking 8 appeared to have all-male teams, with 5 more having a single woman on the team. Sadly, even in 2022, some parts of robotics hasn't changed much.

The future of robotic telepresence comes down to UX and for the most part the homogenous, engineering-heavy, teams either ignored or "trust-me-im-an-engineer'ed" the interaction. It didn't work for operators and thus for the task. UX design and HRI was the missing ingredient.