Thread by @TaylorOgan

Far too many people think Tesla is much closer to being able to take the human out of the driver's seat than it really is. Here's why it is nowhere close.

2/ Let's go way back to Waymo's 2016 Autonomous Vehicle Disengagement Report. By the end of the 2016 reporting period, Google/Waymo had operated in autonomous mode for 2.3 million miles, 636k of which occurred on public roads. (Mostly in Mountain View & neighboring communities)

3/ In 2016, 60 Waymo vehicles drove 636k miles autonomously on public roads in California. During those, there were 124 disengagements reported while the vehicles were in autonomous mode. In other words, 5,128 miles driven autonomously between reported disengagements.

4/ It is important to note that not every time the safety driver turns the car off of autonomous mode is considered a "reportable disengagement" by the CA DMV. Most (unreported) disengagements are for planned takeovers. Only unplanned takeovers/disengagements count.

5/ Among the causes for disengagements, software discrepancy and unwanted maneuver of the vehicle were the leading reasons the safety drivers felt the need to take over control.

6/ Among the least common causes for disengagements were for: emergency vehicles, road debris, construction zones, & once for weather. This was over the course of a year.

7/ While it is unclear how much of the autonomous driving was done on highways vs. urban streets, most of the disengagements were on urban streets, as you would expect.

8/ @CA_DMV requires AV testing permit holders to categorize and describe EVERY unexpected disengagement. Tesla doesn't report its data because it is still a Level 2 system. Even its FSD City Streets Beta is not subject to autonomous mileage disclosure.

9/ So let's compare Waymo in 2016 to Tesla FSD City Streets Beta today.

Miles between disengagements
Waymo (2016): 5,128mi
Tesla FSD Beta (today, observed): ~2mi

10/ "But FSD is still in Beta." Okay, let's compare something Tesla thinks is "feature-complete" enough to recognize deferred revenue: Navigate on Autopilot - Highway.

11/ Let's use the best available dataset on Tesla Autopilot disengagements: the controversial (for being too pro-Tesla) MIT AgeLab paper on Tesla AP.

Of 112,427 miles driven on Autopilot, there were 18,928 disengagement epochs.

5.94mi per disengagement

12/ But 54% of those would not be considered "reportable disengagements" by the CA DMV. So let's use the ones that would be counted: "Tricky Situations". MIT normalized the # and only counted daytime driving, which resulted in an Autopilot disengagement every 9.2 miles.

13/ Now that's highway driving, which Tesla claims its system is very good at navigating. I use it a lot, and it is quite good. 9.2 miles sounds accurate on highway driving. That's Tesla's BEST statistic.

Miles between disengagements
Waymo (2016): 5,128mi
Tesla Autopilot: 9.2mi

14/ And even that comparison isn't fair. Waymo's 2016 disengagement rate is mostly in urban/suburban driving with turns, traffic lights/signs, etc., whereas Tesla's disengagement rate is on highways and well-lined lanes with NO turns, traffic lights/signs, etc.

15/ At this point, some may be thinking that Waymo's 2016 CA AV driving data was conducted in a small HD-mapped area, and that Tesla uses a more generalized approach. Well, Tesla's latest FSD Beta cannot drive more than a few miles before driver take over ANYWHERE...

16/ ...meanwhile, @Waymo has robotaxi fleets in operation with no human driver behind the wheel. Sure, Waymo's robotaxis may look ugly & cost as much as a Tesla Model S with all the sensors & computers they use, but Waymo has successfully taken the human out of the driver's seat.

17/ Tesla FSD fans often tout how much Tesla's FSD Beta is improving with each iteration. Waymo has also improved a lot since 2016: +484%.

Last year, Waymo logged 29,945 autonomous miles between reported disengagements.

18/ Waymo isn't the only company with impressive disengagement stats. China's @autoxtech and @PonyAI_tech logged 20k and 10.7k miles between disengagements, respectively.

19/ In California, the @CA_DMV and @californiapuc award self-driving permits to companies with years of autonomous testing data, with tens of thousands of autonomous miles between disengagements.

20/ And these actual robotaxis are not just limited to California. AV companies are deploying robotaxis all over the world.

https://twitter.com/bridgemccarthy_/status/1458158346813517833?s=20

21/ Now think back to @Waymo's 2016 disengagements by cause, & compare those situations to Tesla FSD Beta videos you have seen. Note how many times in just the videos you have seen the driver has to disengage for debris, construction zones, emergency vehicles, etc. vs. Waymo's 7.

22/ Tesla's FSD Beta software improvements do make for good YouTube content, but the rate of improvement and actual capability pales in comparison to other AV companies. This isn't about geofencing, or the cost of the components, or HD mapping, etc.

https://twitter.com/TaylorOgan/status/1450547719836209154?s=20

23/ This is about actual, functioning robotaxis vs. two PowerPoints (2019 & 2021) and misleading label for a (mostly) useful ADAS.

There's a reason you'll get arrested for doing this in a Tesla, but have to pay to do it in an actual robotaxi.