,
10 tweets,
3 min read
Read on Twitter
NHTSA's 2017 finding that Tesla Autopilot reduced crashes by 40 percent was based on embarrassingly basic math errors. arstechnica.com/cars/2019/02/i… 
Some Tesla fans are questioning QCS/Randy Whitfield's credentials or claiming his critique is hard to follow, but you don't need to take Whitfield's word for it. The raw NHTSA spreadsheet is available here and not hard to understand. safetyresearch.net/Library/2018-1…
Here is an excerpt from NHTSA's raw data. Column B shows the mileage at which Tesla says Autosteer was activated for this car. Since the odometer starts at 0, NHTSA treated this as the number of miles the car drove before Autosteer activation—Column F.
Now look at row 43780, which I've highlighted. Tesla didn't provide odometer readings for Autosteer installation (columns B and C). So NHTSA counted all miles as post-Autosteer miles (Column G) and assumed there were 0 pre-Autosteer miles.
But column D shows that there was an "airbag event" (a crash) prior to the activation of Autosteer. But the car drove 0 miles pre-Autosteer. This doesn't make sense!
Obviously the "-" in columns B and C mean that there's data missing, not that the car had Autosteer active from the time it left the factory. A competent analyst would have thrown out this row and all others like it.
(NB for those checking this: I deleted some empty columns and re-organized the rows, so the column letters and row numbers won't match what you find in the original document. The data is otherwise unaltered.)
There are 18 vehicles in the data set with "-" in the "Previous Mileage before Autosteer Install" column and "TRUE" in the "Occurrence of Airbag events before Autosteer" column—exactly the number QCS says in its report.
Whitfield says that 29,051 out of 43,781 rows have this kind of problem—about half of those 29k are missing "Previous Mileage before Autosteer Install" while the other half are missing both "Previous Mileage before Autosteer Install" and "Next Mileage after Autosteer Install."
I haven't tried to confirm all of Whitfield's numbers but I have no doubt that his basic critique of NHTSA's methodology is correct. This was an embarrassingly basic math error and they should have owned up to it a long time ago.
