Chris Profile picture
Aug 23, 2021 12 tweets 7 min read Read on X
1/ Setting the record straight: What you see here is Sim 1.0, while the industry is moving towards Sim 2.0. Example are @Waymo Simulation City and @RaquelUrtasun GeoSim. Ex: the below reconstruction was manually recreated in Unreal Engine 4 and would take *hours/days. #TeslaAIDay
2/ Even environments recreated with procedural tools (1.0 tech) are limited by hand modeled assets & textures by artists, introducing huge domain gaps. Its not scalable & is why Sim 1.0 is being depreciated (taking backseat) in some AV companies while Tesla is just introducing it
3/ .@theinformation said in Q4 2018 that Tesla's simulation were "in their infancy". Sim 1.0 is ~2015 era tech that gives perfect labeled ground truth, procedural scenario generation & reconstruction, etc. @aurora_inno goes into details of Sim 1.0 here.
aurora.tech/blog/scaling-s…
4/ Sim 2.0 intends to bridge the domain gap & generate realistic simulation of every part of the AV stack at scale. So instead of using a game engine to create an environment, its automatically generated from real world data with dynamic objects removed/added using NNs.
5/ That same environment is then populated by realistic sensor sim of objects/actors. Ex: GeoSim automatically generates synthetic and realistic geometry & traffic aware camera data for training & testing at scale. Unlike Tesla, manually designing 3d models in 3d max/maya/blender
6/ The 3d models are extracted from real camera data and then shaded, realistically lit and shadowed using neural rendering. Each picture below contains a GeoSim vehicle. Can you tell me what the simulated cars are?
7/ Instead of doing manual labor and hiring 3D environmental artists to model 3d objects and environments. Then spend compute resources to render them. You automatically generate it from video footage in an instant. Each example posted below again has sim cars, which ones is it?
8/ In Tesla's scenario. Geosim & other NN technique that @RaquelUrtasun team have developed would give you the ability to remove & add car/peds in the video. Have 10 real peds crossing the street instead of one. Generate thousands of new scenarios using the same cars in the video
8/ Change their speed, position, trajectory & behavior (aggressive, cautious, drunk). Sim 2.0 involves the realistic generation of sensor data. Camera/lidar/radar, not just from a game engine like Sim 1.0 but from a NN at SCALE while bridging the domain gap. @Waymo's sim lidar
9/ Raquel Urtasun goes in-depth on Sim 2.0 here. While @Tesla is just starting work on "neural rendering techniques" others are deploying GeoSim & Simulation City in their production pipeline that renders sim 1.0 obsolete.
10/ At least Tesla has now changed its mind on simulation from "its grading your own homework" to now "we are excited about what sim can achieve". Here you see SurfelGAN which Waymo's Simulation City is based on.
11/ One thing @Tesla lack when discussing their simulation is disclosing how they are working to incorporate complex smart agents that represent the entire distribution of human driving behavior. Drago from @Waymo goes in-depth on it here. 42 mins

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Chris

Chris Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @Christiano92

Aug 20, 2021
So instead of a working HPC, what we saw in #Dojo is a single node that’s is running a rudimentary NN on a single test bench. Designing a chip is easy. What’s hard is building the compiler, runtime scheduler in a HPC environment at scale. none of which @Tesla is anywhere close to Image
When asked, its brushed off as “No but we will”. This exapod doesn’t exist it’s a photoshopped image and won’t exist for many years. That’s why there’s no MLPerf benchmarks. There is only one exapod in existence & that’s @Google TPU version 4. In use today by @Waymo #Dojo Image
By the time this is working and ready in the presented form and specs there will already be a TPU V5. @Tesla will always be 3-4 years behind on this front. But this won’t stop the fanboys because ignorance is bliss.

Read 4 tweets
Aug 19, 2021
I will be live tweeting @Tesla AI Day Here. Personally looking at the data flow, latency, low power and performance of #Dojo and how it compares to @Google #TPU version 4 and other HPC systems and how it directly relate to #AutonomousVehicles #FSDBeta
Well as usual, Tesla is fashionably late and we are almost 18 mins past the start time. And the beat goes on...
#AIDay starts with a demo of FSD with the driver griping the steering wheel with his left hand in what looks to be the streets of California. #DojovsTPU #Dojo @Tesla Image
Read 22 tweets
May 7, 2021
With @Tesla CJ Moore, Director of Autopilot saying "Elon’s tweet does not match engineering reality". Here's Elon's tweets/statement about #FSD $TSLA

Dec 2015: "We're going to end up with complete autonomy, and I think we will have complete autonomy in approximately two years."
January 2016: "In ~2 years, summon should work anywhere connected by land & not blocked by borders, eg you're in LA and the car is in NY"

June 2016: "I really consider autonomous driving a solved problem, I think we are less than two years away from complete autonomy, safer than humans, but regulations should take at least another year"

theguardian.com/technology/201…
Read 24 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(