Thread by @olivercameron

I recently left @Cruise after 2 amazing years.

In that time, we launched & scaled cars with no driver to tens of thousands of happy customers. I'm super proud to have worked at such a special place.

What's next after 7 years in self-driving cars & AI, and what did I learn? 👇

7 years ago, self-driving cars were heavily rules-based, with limited use of ML (mostly for object detection).

On a good day, with perfect conditions, this produced impressive demos. But, place the car in a dynamic, complex scenario, and it fell apart.

Rules were insufficient!

So, in 2016, a team of us (including @SebastianThrun) began building an open-source self-driving car.

Thousands of engineers from around the world joined in.

Our goal was to accelerate self-driving car and ML dev, so we open-sourced data and set challenges for the community.

Inspired by work from @nvidia, one challenge we set was to train ML models that could predict the correct steering angle of a car from solely a camera image.

No lidar or radar allowed. Just end-to-end ML.

51 teams contributed, with dozens of encouraging ML models generated.

A few months later, we set another challenge.

We offered a $100K prize for the best-performing object tracker & detector, this time allowing lidar and radar inputs.

Basically, use ML to significantly improve the vision of a self-driving car.

Over 2,000(!) teams participated.

After 5 challenges, what was clear to me:

1. ML would dramatically improve the performance of self-driving cars.

2. New, novel NN architectures were now needed.

3. The entire stack—from perception to prediction to planning to controls—should transition from rules to ML models.

In early 2017—impatient with the speed of self-driving car deployment—we started our own startup, @voyage.

We wanted to deliver self-driving cars to those who needed them most, starting with senior citizens.

We raised $5M from @khoslaventures and others to get started.

To deliver a self-driving car fast, we started by developing & deploying our fleet in retirement communities.

There's so much demand from the 70+ year-old residents for safe, autonomous transportation.

Plus, retirement communities have quiet and slow (~25 MPH) roads.

Over 4 years, the @voyage team delivered milestone after milestone.

Each milestone increased the performance of our self-driving car, enabling us to move more customers to more places.

Unsurprisingly—you guessed it—our biggest leaps in performance came because of ML.

My favourite example was HQDM, or High-Quality Decision Making, where we replaced our rules-based Behavior Planner with ML.

Basically, we taught our self-driving car to make more human-like decisions in scenarios where rules simply aren't sufficient.

news.voyage.auto/teaching-a-sel…

In early 2021, @Cruise was on the march to commercialization, which was our speciality. To supercharge that, they acquired @voyage.

@Cruise had a mind-blowing stack powered almost entirely by—you guessed it again—ML.

We joined forces to rollout fully driverless cars faster.

Now inside @Cruise, I saw how broadly ML was applied. Dozens of ML models ran in real-time on each @Cruise car.

This explained why @Cruise's technology was so much better at handling crazy, dynamic traffic in SF.

Watch this for examples.

After 2 years at @Cruise, I've seen ML become even more ingrained in our stack. Think way beyond even perception, planning, prediction, and controls.

For customers, this has resulted in superhuman driving, in both comfort and—most importantly—safety.

https://twitter.com/kvogt/status/1651934421367545856

I cannot emphasize enough how incredible this tech is and the impact of ML.

7 years ago, rules-based tech struggled to drive a few blocks.

Now, ML has enabled cars to drive by themselves all day long in downtown SF, resulting in a remarkable product.

So, after 7 years in self-driving, what do I now know?

1. The need for humans to manually drive steel boxes is ending.

2. ML was the enabling technology necessary to replace the need for human drivers.

3. ML will eventually replace hand-written code.

https://twitter.com/olivercameron/status/1652007924867428352

The first transition of a complex human task to ML has now happened with self-driving cars.

I am confident we'll now see this same transition occur in other human tasks within all industries: entertainment, finance, defense, education, construction, energy, aerospace, and more.

So, what's next for me?

If ML can outperform humans at a crazy complex & dynamic task like driving, what else is it capable of?

That—as vague as it sounds today—is what I’m passionate about, and what I’m exploring now after leaving @Cruise.

I couldn't be more excited.

I'm also excited to continue to invest in AI startups.

With the recent explosion of LLMs, computer vision, and beyond, it's a special time to build. If you're developing game-changing AI products, please get in touch.

https://twitter.com/olivercameron/status/1605280391446904832

I can't wait to see all the new ways ML will change the world, and to see @Cruise scale to serve millions more happy customers in self-driving cars.

A huge thank you to all of @Cruise, our customers, @kvogt, @danielkan, the Product team, and the @voyage team. It was epic! ❤️