Tesla is building the foundation models for autonomous robots twitter.com/i/web/status/1…
Our multi-modal neural networks are already in customer vehicles—these networks take in arbitrary modalities such as camera videos, maps, navigation, IMU (Inertial Measurement Unit), GPS etc.
Tasks such as Occupancy prediction are already quite general in what they represent—in some ways, they are ontology-free & simply predict the probability that some 3D position is occupied.
Such occupancy can be used for collision avoidance by any robot.
All of this is enabled by fleet scale auto-labelling. By using video data from multiple trips in the same location, we can reconstruct the entire scene
In addition, we’re building off state-of-the-art generative modeling techniques—enabling us to predict possible outcomes given past observations, in a jointly consistent manner across multiple camera views
These imagined futures can be action-conditioned to produce different outcomes.
For example, the videos below are generated entirely by the neural network by simply using different prompts
These models will learn from a huge set of extremely diverse data from the Tesla fleet
And will be trained on enormous amounts of compute
These video foundation models will serve as the brain of both the car & Optimus robot 🚘🧠🤖
Join the Tesla AI team to build the future of robotics!
→ tesla.com/AI twitter.com/i/web/status/1…
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.