Thread by @Tweetermeyer

Imagine this: in the midst of the space race, with the US and USSR pumping billions into moon landing missions, Boeing suddenly claims they could land people on the moon in a modified 707.

That's roughly how Tesla's Full Self-Driving claims come across to AV developers.

AVs are taking longer to reach maturity than a lot of people expected, and that's using a ton of super high-end cameras (>20 in the new Waymo stack) plus 360 degree coverage of short- and long-range lidar, and high-performance radar. Typical sensor suites costs 6 figures per car!

To see Tesla's claims as credible, we're forced to believe that the companies who have been leading in this technology could be using hardware that costs at least an order of magnitude less... but either don't realize it, or aren't smart enough to make it work.

The leaders of these other companies were almost all pioneers of modern AV tech who broke new ground during the DARPA challenges. Some of them have already shipped autonomous products, like the Caterpillar mining trucks Argo's Bryan Salesky developed post-gazette.com/business/tech-…

The question I can't get a straight answer to is: why would these people be developing a (metaphorical) spaceship if an airplane could get them to the moon? Why do all of the pioneers in AV technology agree that it requires sensors and features that add huge expense and dev time?

Equally puzzling: why aren't there more companies pursuing L5 autonomous systems that use such low-cost hardware? They exist, Wayve being one example... but they aren't getting the big investments, or making groundbreaking demonstrations. Why not? venturebeat.com/2019/11/17/way…

Mobileye, the leader in computer vision for automated driving, has been working this problem for over 20 years, collects 3.7m miles of sensor data every day, and can demonstrate impressive camera-only AV performance... but says safety demands radar/lidar. arstechnica.com/cars/2020/01/i…

Why would the leader in camera-only driving automation say it won't put a camera-only AV into production? The answer can be found in Intel/Mobileye's work to create a safety case for their vehicles, which you can find (in white paper form) here [PDF]: arxiv.org/pdf/1708.06374…

Here is a selection from that white paper that give you a sense of what Intel/Mobileye is trying to accomplish with their AV development: they are balancing safety and scalability (i.e. cost) but they need to validate to 3 orders of magnitude better safety than human drivers.

Note: this isn't some regulatory standard, as no regulator has promulgated such an "incident per miles driven" standard. This is the level of safety they think they need to achieve in order to avoid risking public acceptance of autonomous vehicles. This is crucial!

To prove that your system actually meets a given safety threshold, the miles over which you validate have to be representative of the operational domain. That's possible with L3/4, but how do you ensure that a L5 system has been validated in any/all conditions it might see?

This is a critical question, because an AV system can't actually be used properly (without an occupant, for example) until you can validate its safety in a domain-representative way. You can't just drive around the block a billion times! Level 5 makes that part muuuch harder.

Until you validate safety across the domain, "safety drivers" will be the only thing ensuring safety. As the thread linked here shows, AV developers use highly-trained teams of at least two with rotating responsibilities to prevent inevitable complacency.

https://twitter.com/Tweetermeyer/status/1306002308950495232

Using far less-capable hardware, less (or in some cases, no) sensor redundancy, no HD maps and unvalidated software with only untrained consumers as a safety net suggests that the key advantage lies not in Tesla's technology but in its risk tolerance. See: finance.yahoo.com/news/happened-…

The critical point: if this wild gamble doesn't pay off, it could profoundly affect public perceptions of autonomous drive technology. This isn't just a bet on a theory or a stock, it's gambling with human lives and the future of the most important technology of this century.

That's why I want to make sure that we're going into this with our eyes open: it's super important that public understand that Tesla's approach is wildly divergent from the industry consensus, on multiple levels. Otherwise, the entire sector will pay for risks they refuse to take

We've already seen this happen: a @faizsays story about the "techlash" and AVs showed that even though public concerns about tech culture apply more to Tesla's approach, they are identified with "real AVs" made by firms with strong safety cultures. thedrive.com/tech/30151/sil…

So forgive me in advance for tweeting a lot about this issue over the coming weeks and months. I genuinely hope the rollout of "Full Self-Driving" is safe, but if it's not we're going to have to work hard to make sure that the takeaway isn't "all AVs are unsafe/doomed" #NotAllAVs