RL enables robots to navigate real-world environments, with diverse visually indicated goals: sites.google.com/view/ving-robo…
w/ @_prieuredesion, B. Eysenbach, G. Kahn, @nick_rhinehart
paper: arxiv.org/abs/2012.09812
video:
Thread below ->
w/ @_prieuredesion, B. Eysenbach, G. Kahn, @nick_rhinehart
paper: arxiv.org/abs/2012.09812
video:
Thread below ->
The idea: use RL + graph search to learn to reach visually indicated goals, using offline data. Starting with data in an environment (which in our case was previously collected for another project, BADGR), train a distance function and policy for visually indicated goals.
2/n
2/n
Once we have a distance function, policy, and graph, we search the graph to find a path for new visually indicated goals (images), and then execute the policy for the nearest node. A few careful design decisions (in the paper) make this work much better than prior work.
3/n
3/n
This system can then be used in a few interesting ways: we can define "contactless delivery" targets just by having someone take a photo of their front door, and the robot then navigates to their front door to deliver their package.
4/n
4/n
Or you could command the robot to patrol an area by giving it visually indicated waypoints, simply by snapping photographs of each waypoint.
5/n
5/n
I think this method is promising for a few reasons:
- offline training using large prior datasets
- fully autonomous
- flexible, new goals specified just using images
In experiments, data was collected ~6 mo prior (for another project), so it's even robust to changing seasons :)
- offline training using large prior datasets
- fully autonomous
- flexible, new goals specified just using images
In experiments, data was collected ~6 mo prior (for another project), so it's even robust to changing seasons :)
• • •
Missing some Tweet in this thread? You can try to
force a refresh
