Lawrence Yunliang Chen Profile picture
PhD Student in Robotics @UCBerkeley @berkeley_ai Advised by Prof. @Ken_Goldberg

Mar 23, 2023, 11 tweets

Can a robot open a crumpled plastic bag and pack objects into it? Weโ€™re developing algorithms for autonomous bagging. @UCBerkeley BAIR @berkeley_ai, @CMU_Robotics, and @ToyotaResearch #ICRA2023 ๐Ÿงต๐Ÿ‘‡ (1/11)

๐Ÿ”Ž Thin plastic bags are ubiquitous. โ€œBaggingโ€ is useful for many tasks such as grocery shopping, cleaning, recycling, and packing. (2/11)

While easy for humans, bagging is very challenging for robots. Bags can deform in many ways, creating complex self-occlusions. Furthermore, many plastic bags are reflective, or translucent, making perception challenging. (3/11)

Bags are also difficult to manipulate because different parts of the bag move differently, making it difficult to orient. Moreover, a one-millimeter difference in the gripper height can also significantly affect grasping. (4/11)

๐Ÿ’ก To address these challenges, the robot learns a novel representation of plastic bags with segmentation of bag handles and rim. Inspired by All You Need is LUV (arxiv.org/abs/2203.04566), we use UV light to generate labeled training data under self-supervision: (5/11)

๐ŸŽจ We paint a set of training bags with UV-fluorescent paints. The bag looks normal under regular lighting. But under UV lights, the paints glow with unique colors, allowing the robot to collect high-accuracy ground-truth labels automatically ๐ŸŒŸ. (6/11)

๐Ÿค–๐Ÿฆพ The robot systematically explores the bag state space by sampling actions to manipulate the bag into diverse configurations. This makes data collection efficient and scalable. (7/11)

We use the collected data to train a segmentation model that recognizes the bag rim and handles at execution time without UV lights. We propose two metrics to quantify how large and how round the bag opening is. (8/11)

With the perception module, our algorithm divides the bagging task into 3 stages: (1) Orienting the bag upward, (2) enlarging the opening, and (3) inserting the objects and lifting the bag. (9/11)

Physical experiments are encouraging ๐Ÿ”ฅ (10/11)

๐Ÿ’ฅ This is a collaboration with @BaiyuShi147 @Ken_Goldberg @UCBerkeley @AUTOLab_Cal + @daniel_t_seita @davheld @CMU_Robotics + Richard Cheng @tkollar @ToyotaResearch.
๐Ÿ‘€ Paper: arxiv.org/abs/2210.17217
๐ŸŒ Project website: sites.google.com/view/autobag (11/11)

Share this Scrolly Tale with your friends.

A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.

Keep scrolling