Thread by @Tweetermeyer

Just wanted to add a little bit of context to this, to help make sense of what Shashua was saying. I'm hoping to go into a little more depth with him on this in a future episode of @TheAutonocast, but for now you can watch the whole thing here:

https://twitter.com/Keubiko/status/1351884457507254276

The question here is: what is the right way to approach a massively complex machine learning problem with an unforgiving standard for reliability/accuracy? If you have access to fleet data, do you use it to find and train for edge cases (Tesla) or build maps (Mobileye)?

The core of Shashua's argument, as I understand it, is that using fleet data to identify and train for edge cases is of limited utility because you ultimately run into overfitting. Training for specific cases makes your solution less generalizable. en.wikipedia.org/wiki/Overfitti…

The alternative, using the same fleet data to create "maps," approaches the same problem from a different direction: instead of tweaking the algos in a million different directions, you reduce the problem space by creating a semantically segmented "map."

So, instead of having to run inference on EVERYTHING the vehicle sees you have a fully automated, crowdsourced map that creates validated certainty about fixed features which reduces the number of things you have to identify/classify/predict, and thus reducing failure rates.

I certainly can't tell you that one of these approaches is definitely better than the other, but on a philosophical level the Mobileye approach seems built on an acceptance of the fundamental limitations/challenges of probabilistic systems in safety-critical applications.

ML tends to be a "90-10" (or even "99-1") solution: it's relatively easy to solve 90% of a problem with ML, but it gets dramatically harder as you approach 100%. This fundamentally suggests that reducing the problem area is likely to work better than endlessly tweaking algos.

This is a super high-level explanation, and it's just my understanding right now. As I mentioned, I look forward to focusing on this issue more in a future conversation with Shashua because I think it gets to the heart of some thorny but important issues in AV development.

Final note, for now: the comments on that InsideEVs story dramatically illustrate the challenges we face in public education around AV technology, and why I really need to get off Twitter and do my damn job already!