Lawrence Yunliang Chen Profile picture
Mar 23, 2023 β€’ 11 tweets β€’ 7 min read β€’ Read on X
Can a robot open a crumpled plastic bag and pack objects into it? We’re developing algorithms for autonomous bagging. @UCBerkeley BAIR @berkeley_ai, @CMU_Robotics, and @ToyotaResearch #ICRA2023 πŸ§΅πŸ‘‡ (1/11)
πŸ”Ž Thin plastic bags are ubiquitous. β€œBagging” is useful for many tasks such as grocery shopping, cleaning, recycling, and packing. (2/11)
While easy for humans, bagging is very challenging for robots. Bags can deform in many ways, creating complex self-occlusions. Furthermore, many plastic bags are reflective, or translucent, making perception challenging. (3/11)
Bags are also difficult to manipulate because different parts of the bag move differently, making it difficult to orient. Moreover, a one-millimeter difference in the gripper height can also significantly affect grasping. (4/11)
πŸ’‘ To address these challenges, the robot learns a novel representation of plastic bags with segmentation of bag handles and rim. Inspired by All You Need is LUV (arxiv.org/abs/2203.04566), we use UV light to generate labeled training data under self-supervision: (5/11)
🎨 We paint a set of training bags with UV-fluorescent paints. The bag looks normal under regular lighting. But under UV lights, the paints glow with unique colors, allowing the robot to collect high-accuracy ground-truth labels automatically 🌟. (6/11)
πŸ€–πŸ¦Ύ The robot systematically explores the bag state space by sampling actions to manipulate the bag into diverse configurations. This makes data collection efficient and scalable. (7/11)
We use the collected data to train a segmentation model that recognizes the bag rim and handles at execution time without UV lights. We propose two metrics to quantify how large and how round the bag opening is. (8/11)
With the perception module, our algorithm divides the bagging task into 3 stages: (1) Orienting the bag upward, (2) enlarging the opening, and (3) inserting the objects and lifting the bag. (9/11)
Physical experiments are encouraging πŸ”₯ (10/11)
πŸ’₯ This is a collaboration with @BaiyuShi147 @Ken_Goldberg @UCBerkeley @AUTOLab_Cal + @daniel_t_seita @davheld @CMU_Robotics + Richard Cheng @tkollar @ToyotaResearch.
πŸ‘€ Paper: arxiv.org/abs/2210.17217
🌐 Project website: sites.google.com/view/autobag (11/11)

β€’ β€’ β€’

Missing some Tweet in this thread? You can try to force a refresh
γ€€

Keep Current with Lawrence Yunliang Chen

Lawrence Yunliang Chen Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(