Thread by @atroyn

this idea comes up frequently and in practice is much more difficult than it seems. the reasons why are illustrative of the software/hardware disconnect in dealing with the real world

https://twitter.com/balajis/status/1153378100702371840

teleoperation seems like both a great and simple idea; if you can replace the “””AI””” part of robotics with a human you get a big gain for free right?

not simple in practice, and about as intractable as the robotics problem on its own in many cases. here are some reasons

the “””AI””” part of robotics is seldom the hard part in many applications. you don’t need much of a sophisticated on-line planning algo to drive a tractor around a field.

you hit the first problems in control and actuation. electromechanical actuators are not perfect. they do not respond linearly to control inputs. their sensors are noisy; the linkage is not necessarily in the position the sensor says it’s in

controllers are not perfect. the input you’re giving your actuator may not be the same one as the controller is supposed to give. the sensors that tell you this aren’t perfect.

so even assuming perfect perception and planning, in a zero latency environment, teleoperation has to contend with getting the machine to do what you want it to do

to some degree a human can compensate for these facts and indeed they do; when you’re in the tractor you’re dealing with these same problems to some extent

but when you’re in the tractor you have human level perception. you don’t get to have that over teleoperation. even before latency, you immediately lose proprioception and inertial sensing, as well as haptic feedback. you can build haptics in but we’re back where we started then

so let’s say eventually we rig something up that’s fine for humans to drive a tractor around a field remotely. fine, but you still have every other part of farm operations to automate, plus the teleoperation overhead you’ve sunk a lot of resources into

Moravec’s paradox cruelly strikes again; the stuff we can do teleoperation on we can just automate, the stuff we can’t automate we can’t do teleoperation on either

this is not universally true; there are useful operating points for teleoperation. Supervision of autonomous operation in environments where there are a lot of long tail cases is one

detecting anomalous environments and calling for help is a simpler problem than just solving autonomy

but for many of the “””obvious””” cases where we ‘just’ replace the “””AI””” with a human, it’s either not feasible or else not interesting

dealing with the physical world is much more difficult than dealing with any software environment regardless of the verisimilitude or your simulation, not because of something special about the physical world but because of the shortcomings of software

this is before we get into the economics of any of this