Thread by @corry_wang

1/ I don’t think people have realized how much this new Waymo scaling laws paper is basically an admission that “Waymo was wrong, Tesla was right”

Hopefully this becomes a call to action internally within Waymo

https://twitter.com/waymo/status/1933543634583818390

2/ This paper shows autonomous driving follows the same scaling laws as the rest of ML - performance improves predictably on a log linear basis with data and compute

This is no surprise to anybody working on LLMs, but it’s VERY different from consensus at Waymo a few years ago

3/ Waymo built its tech stack during the pre-scaling paradigm. They train a tiny model on a tiny amount of simulated and real world driving data and then finetune it to handle as many bespoke edge cases as possible

This is basically where LLMs back in 2019

4/ The bitter lesson in LLMs post 2019 was that finetuning tiny models on bespoke edge cases was a waste of time. GPT-3 proved if you just to train a 100x bigger model on 100x more data with 10,000x more compute, all the problems would more or less solve themselves!

5/ If the same thing is true in AV, this basically obviates the lead that Waymo has been building in the industry since the 2010s. All a competitor needs to do is buy 10x more GPUs and collect 10x more data, and you can leapfrog a decade of accumulated manual engineering effort

6/ In contrast to Waymo, it’s clear Tesla has now internalized the bitter lesson

They threw out their legacy AV software stack a few years ago, built a 10x larger training GPU cluster than Waymo, and have 1000x more cars on the road collecting training data today

7/ I’ve never been that impressed by Tesla FSD compared to Waymo. But if Waymo’s own paper is right, then we could be on the cusp of a “GPT-3 moment” in AV where the tables suddenly turn overnight

The best time for Waymo to act was 5 years ago. The next best time is today!