With the start of 2022 🎉, it's a good time to share appreciation for everything we've learned and the people that I had the fortune to work with.
It was an extraordinary year for us in #robotics and #machinelearning. And thanks to it, 2023 has some incredible projects lined up!
It was an extraordinary year for us in #robotics and #machinelearning. And thanks to it, 2023 has some incredible projects lined up!
Hierarchical & non-hierarchical RL together!
We learned how a general dual system approach 🧠 to reinforcement learning can enable us to learn faster while retaining strong final performance. We also have some great results with robot arms and humanoids!
We learned how a general dual system approach 🧠 to reinforcement learning can enable us to learn faster while retaining strong final performance. We also have some great results with robot arms and humanoids!
https://twitter.com/markus_with_k/status/1600044517075337216?s=20&t=AU7Z2MjtxUwJeh83g0gzSA
Work led with Giulia Vezzani, Dhruva Tirumala and a great team across @DeepMind!
DQN is all you need?
We learned that simple solutions are often competitive. A minor variation of DQN is able to solve continuous control tasks on par with your favorite SOTA algorithm (SAC, MPO, D4PG, DrQv2, DreamerV2).
We learned that simple solutions are often competitive. A minor variation of DQN is able to solve continuous control tasks on par with your favorite SOTA algorithm (SAC, MPO, D4PG, DrQv2, DreamerV2).
https://twitter.com/markus_with_k/status/1585274003631087617?s=20&t=AU7Z2MjtxUwJeh83g0gzSA
Minor variation = bang bang control and decoupled optimisation per action dimension.
Led by @timseyde (PhD student at @MIT_CSAIL) and in collaboration with @igilitschenski (@UofTRobotics) and many others.
Led by @timseyde (PhD student at @MIT_CSAIL) and in collaboration with @igilitschenski (@UofTRobotics) and many others.
Humanoid Soccer!
We learned that RL combined with skill transfer, imitation learning and self-play can scale to competitive and collaborative behaviour in soccer ⚽️ with exceedingly high-dimensional embodiments.
We learned that RL combined with skill transfer, imitation learning and self-play can scale to competitive and collaborative behaviour in soccer ⚽️ with exceedingly high-dimensional embodiments.
https://twitter.com/hardmaru/status/1397909538863423490?s=20&t=AU7Z2MjtxUwJeh83g0gzSA
https://twitter.com/DeepMind/status/1565041456372273152?s=20&t=AU7Z2MjtxUwJeh83g0gzSA
Work with @liusiqi42, Guy Lever and many others @DeepMind!
Imitate and repurpose!
We learned that 2- and 4-legged robots can apply behaviour priors from MOCAP to enable safe, robust and efficient learning. Plus, our robot dog now plays soccer!
We learned that 2- and 4-legged robots can apply behaviour priors from MOCAP to enable safe, robust and efficient learning. Plus, our robot dog now plays soccer!
https://twitter.com/_akhaliq/status/1509829356021071876?s=20&t=AU7Z2MjtxUwJeh83g0gzSA
Work with Steven Bohez, Saran Tunyasuvunakool and many others @DeepMind!
Hybrid continuous-discrete systems!
We learned that we can solve some really complex robot manipulation tasks by combining the benefits of discrete and continuous behaviour spaces!
Work led by Dushyant Rao!
We learned that we can solve some really complex robot manipulation tasks by combining the benefits of discrete and continuous behaviour spaces!
https://twitter.com/_akhaliq/status/1469179400151261193?s=20&t=O5OGeiHbB3L2WInMVaWxOQ
Work led by Dushyant Rao!
Data generation for offline RL!
We learned about which properties are best for dataset generation for later offline RL. And how does artificial curiosity fit into the mix?
Work led by @natolambert (now research scientist at @huggingface)
We learned about which properties are best for dataset generation for later offline RL. And how does artificial curiosity fit into the mix?
https://twitter.com/natolambert/status/1488618625086877696?s=20&t=O5OGeiHbB3L2WInMVaWxOQ
Work led by @natolambert (now research scientist at @huggingface)
Lifelong robot learning!
We learned about how to address issues with distribution shifts in our data for long-term robot deployment.
Work led by @Wenxuan_Zhou (currently PhD student @CMU_Robotics )
We learned about how to address issues with distribution shifts in our data for long-term robot deployment.
https://twitter.com/Wenxuan_Zhou/status/1560163328634146816?s=20&t=O5OGeiHbB3L2WInMVaWxOQ
Work led by @Wenxuan_Zhou (currently PhD student @CMU_Robotics )
World models for hierarchical RL!
We learned what roles world models can play when learning skills from offline data.
Work led by @sasha_salter (now at @MetaAI )
We learned what roles world models can play when learning skills from offline data.
https://twitter.com/DeepMind/status/1570017522333663234?s=20&t=O5OGeiHbB3L2WInMVaWxOQ
Work led by @sasha_salter (now at @MetaAI )
Bimanual robot manipulation!
We learned that combining demonstrations and hindsight relabelling facilitates the solution of complex, bimanual cable insertion tasks.
Work led by Todor Davchev (now research scientist at @DeepMind)
We learned that combining demonstrations and hindsight relabelling facilitates the solution of complex, bimanual cable insertion tasks.
https://twitter.com/markus_with_k/status/1470447686692315138?s=20&t=JPc4UUyPnPMKWCUiZeT1Mg
Work led by Todor Davchev (now research scientist at @DeepMind)
Particularly impressive: the last four projects were led by interns within a very short time frame!
Every year, I'm humbled to see what impact our students can achieve while finding their way through infra, teams, research, and the best snacks in the building.
Every year, I'm humbled to see what impact our students can achieve while finding their way through infra, teams, research, and the best snacks in the building.
Key shared insights are coming during the next days. Now let's get back to spending time with our families! 😉
Start of 2023 😉
How do you edit posts? Was this not introduced as a new feature?
How do you edit posts? Was this not introduced as a new feature?
• • •
Missing some Tweet in this thread? You can try to
force a refresh
