Thread by @svlevine

Reusable datasets, such as ImageNet, are a driving force in ML. But how can we reuse data in robotics? In his new blog post, Frederik Ebert talks about "bridge data": multi-domain and multi-task datasets that boost generalization of new tasks: bair.berkeley.edu/blog/2021/11/1…

A thread:

The reason reusing data in robotics is hard is that everyone has a different lab environment, different tasks, etc. So to be reusable, the dataset needs to be both multi-domain *and* multi-task, so that it enables generalization across domains.

So if I collect a little bit of data for my task in my new domain, can I use a reusable dataset to boost generalization of this task? This is not a trivial question, since the "bridge data" does not contain either the new domain or the new task.

However, if it has enough *different* tasks and domains, this turns out to actually work! In fact, including bridge data when training new tasks in new domains (in this case with imitation learning) leads to about a 2x improvement in performance on average!

The full bridge dataset that we are releasing uses a cheap widowX arm to collect 7,200 demonstrations for 71 tasks across 10 different domains (themed around household tasks).

Find dataset, code, and paper link on the project website: sites.google.com/view/bridgedata