Oliver Cameron Profile picture
May 15 19 tweets 11 min read Twitter logo Read on Twitter
I recently left @Cruise after 2 amazing years.

In that time, we launched & scaled cars with no driver to tens of thousands of happy customers. I'm super proud to have worked at such a special place.

What's next after 7 years in self-driving cars & AI, and what did I learn? 👇
7 years ago, self-driving cars were heavily rules-based, with limited use of ML (mostly for object detection).

On a good day, with perfect conditions, this produced impressive demos. But, place the car in a dynamic, complex scenario, and it fell apart.

Rules were insufficient!
So, in 2016, a team of us (including @SebastianThrun) began building an open-source self-driving car.

Thousands of engineers from around the world joined in.

Our goal was to accelerate self-driving car and ML dev, so we open-sourced data and set challenges for the community.
Inspired by work from @nvidia, one challenge we set was to train ML models that could predict the correct steering angle of a car from solely a camera image.

No lidar or radar allowed. Just end-to-end ML.

51 teams contributed, with dozens of encouraging ML models generated.
A few months later, we set another challenge.

We offered a $100K prize for the best-performing object tracker & detector, this time allowing lidar and radar inputs.

Basically, use ML to significantly improve the vision of a self-driving car.

Over 2,000(!) teams participated. Image
After 5 challenges, what was clear to me:

1. ML would dramatically improve the performance of self-driving cars.

2. New, novel NN architectures were now needed.

3. The entire stack—from perception to prediction to planning to controls—should transition from rules to ML models.
In early 2017—impatient with the speed of self-driving car deployment—we started our own startup, @voyage.

We wanted to deliver self-driving cars to those who needed them most, starting with senior citizens.

We raised $5M from @khoslaventures and others to get started. Image
To deliver a self-driving car fast, we started by developing & deploying our fleet in retirement communities.

There's so much demand from the 70+ year-old residents for safe, autonomous transportation.

Plus, retirement communities have quiet and slow (~25 MPH) roads. ImageImageImageImage
Over 4 years, the @voyage team delivered milestone after milestone.

Each milestone increased the performance of our self-driving car, enabling us to move more customers to more places.

Unsurprisingly—you guessed it—our biggest leaps in performance came because of ML.
My favourite example was HQDM, or High-Quality Decision Making, where we replaced our rules-based Behavior Planner with ML.

Basically, we taught our self-driving car to make more human-like decisions in scenarios where rules simply aren't sufficient.

In early 2021, @Cruise was on the march to commercialization, which was our speciality. To supercharge that, they acquired @voyage.

@Cruise had a mind-blowing stack powered almost entirely by—you guessed it again—ML.

We joined forces to rollout fully driverless cars faster. Image
Now inside @Cruise, I saw how broadly ML was applied. Dozens of ML models ran in real-time on each @Cruise car.

This explained why @Cruise's technology was so much better at handling crazy, dynamic traffic in SF.

Watch this for examples.

After 2 years at @Cruise, I've seen ML become even more ingrained in our stack. Think way beyond even perception, planning, prediction, and controls.

For customers, this has resulted in superhuman driving, in both comfort and—most importantly—safety.

I cannot emphasize enough how incredible this tech is and the impact of ML.

7 years ago, rules-based tech struggled to drive a few blocks.

Now, ML has enabled cars to drive by themselves all day long in downtown SF, resulting in a remarkable product.

So, after 7 years in self-driving, what do I now know?

1. The need for humans to manually drive steel boxes is ending.

2. ML was the enabling technology necessary to replace the need for human drivers.

3. ML will eventually replace hand-written code.

The first transition of a complex human task to ML has now happened with self-driving cars.

I am confident we'll now see this same transition occur in other human tasks within all industries: entertainment, finance, defense, education, construction, energy, aerospace, and more.
So, what's next for me?

If ML can outperform humans at a crazy complex & dynamic task like driving, what else is it capable of?

That—as vague as it sounds today—is what I’m passionate about, and what I’m exploring now after leaving @Cruise.

I couldn't be more excited.
I'm also excited to continue to invest in AI startups.

With the recent explosion of LLMs, computer vision, and beyond, it's a special time to build. If you're developing game-changing AI products, please get in touch.

I can't wait to see all the new ways ML will change the world, and to see @Cruise scale to serve millions more happy customers in self-driving cars.

A huge thank you to all of @Cruise, our customers, @kvogt, @danielkan, the Product team, and the @voyage team. It was epic! ❤️ Image

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Oliver Cameron

Oliver Cameron Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @olivercameron

Feb 7
A mistake I often see: people using self-driving car R&D timelines to estimate how long it’ll take to scale self-driving.

Yes, it took awhile to remove the Safety Driver—with a few missed milestones along the way—but scaling is going way faster…
@Cruise was founded in 2013, and it took 7 long years of ML and robotics R&D to unlock 1 fully driverless AV in SF.

But, in just the last 12 months we expanded to 125(!) concurrent AVs, scaled from 1 city to 3 in <90 days, and grew # of rides 4x in the last quarter.
The R&D phase was fundamentally unpredictable.

Massive breakthroughs in CV, prediction, planning, and safety were needed to deliver a generalizable, safe self-driving car.

Each required experimentation and iteration, with failure the default.

Tough to provide accurate dates!
Read 8 tweets
Dec 20, 2022
AI is changing everything, and it's not just ChatGPT or self-driving cars.

You'd be surprised at just how broadly AI is being applied across tons of industries.

I’ve now invested in 50 AI startups, and here are some recent investments using AI in fascinating applications 👇
☄️ @ForgeAstro is building robots to autonomously mine asteroids, returning precious materials to Earth.

Why? One asteroid can contain up to $10,000 quadrillion(!) of rare material.

This is enabled by computer vision, affordable rocket rides, and small sat maturation.
🎭 @WOMBO makes creating images with AI easy, fun, and useful.

With 130+ million downloads on iOS and Android and 2 billion images created, they are likely the largest consumer AI startup around.

My 8-year-old uses @WOMBO like I did Photoshop way back when. So fun.
Read 14 tweets
Dec 24, 2021
3 years ago, I started angel investing in machine learning and autonomous robot startups. I'm now 35 startups in, learning a lot, and having a blast.

The future is going to be incredible. Here are just a few of the startups making it happen 👇
🚗 @AppliedInt builds software tools for automakers and AV companies, significantly accelerating their autonomy efforts. Think world-class simulation, validation, and more.

They are helping save lives at large scale, and just raised $175M at $3.6B.

🏠 @mightybuildings builds beautiful and sustainable homes using robots, advanced materials, and 3D printing. In other words, they are reinventing how we build houses.

They've raised just over $100M from great investors like @khoslaventures.

Read 14 tweets
Dec 18, 2020
Our self-driving technology hit a big milestone recently.

The speed at which computer vision, robotics, machine learning, sensors, and compute is advancing is mind-blowing.

The result? Ride after ride after ride looking like this one 😍
Or this one…
Or this one, with a little rain…
Read 5 tweets
Nov 8, 2020
🚨 It’s that time again!

Let’s dive into the latest release of @Tesla FSD (full self-driving) with a rather eventful trip through midtown Sacramento.

1. Before we get started, here’s a refresher on two important differentiators for FSD.

These differentiators will (eventually) help FSD scale, but they also make the self-driving problem significantly more complex.

In this thread, we’ll explore examples of this.
2. Let’s talk roundabouts.

Roundabouts are not yet consistently handled well by FSD.

See here how FSD appeared to attempt to drive through the center median, requiring intervention.

This is why drivers need to always be alert with their hands on the wheel.
Read 19 tweets
Oct 24, 2020
🚘 Buckle in: it’s time to talk about @Tesla FSD!

In this thread, I’ll share what I find most impressive and the challenges that remain before the “F” in “FSD” is true.

Let’s go!
1. @Tesla FSD is a polarizing topic in AV for a few reasons:

✅ No use of pre-recorded HD maps
✅ Perceiving the world with cameras (no lidar!)

Many think FSD is very different than most other self-driving technologies, but you can boil it down to these two reasons.
2. What @Tesla FSD is thus able to accomplish:

🗺 By generating a map on the fly, instead of pre-loading one recorded earlier, FSD can theoretically drive anywhere

💰 Realizing cost-savings because of fewer sensor modalities
Read 20 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!


0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy


3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!