Tesla is building the foundation models for autonomous robots
Our multi-modal neural networks are already in customer vehiclesâthese networks take in arbitrary modalities such as camera videos, maps, navigation, IMU (Inertial Measurement Unit), GPS etc.
Tasks such as Occupancy prediction are already quite general in what they representâin some ways, they are ontology-free & simply predict the probability that some 3D position is occupied.
Such occupancy can be used for collision avoidance by any robot.
Such occupancy can be used for collision avoidance by any robot.
All of this is enabled by fleet scale auto-labelling. By using video data from multiple trips in the same location, we can reconstruct the entire scene
In addition, weâre building off state-of-the-art generative modeling techniquesâenabling us to predict possible outcomes given past observations, in a jointly consistent manner across multiple camera views
These imagined futures can be action-conditioned to produce different outcomes.
For example, the videos below are generated entirely by the neural network by simply using different prompts
For example, the videos below are generated entirely by the neural network by simply using different prompts
These models will learn from a huge set of extremely diverse data from the Tesla fleet
And will be trained on enormous amounts of compute
These video foundation models will serve as the brain of both the car & Optimus robot đđ§ đ€
Join the Tesla AI team to build the future of robotics!
â tesla.com
Join the Tesla AI team to build the future of robotics!
â tesla.com
Loading suggestions...