This is classical self-driving car software stack. Nowadays, all steps in the pipeline are dominated by machine learning.
Read below for details on each step ๐
Sensors ๐ฅ
There are 3 main sensors for environment perception 360ยฐ around the car:
โช๏ธ Cameras
โช๏ธ Lidars
โช๏ธ Radars
Each sensor has different advantages and disadvantages, so combining all 3 is the best strategy to achieve maximum robustness.
Perception ๐ผ๏ธ
The perception module processes all the sensor raw data to detect different objects, drivable space, lane boundaries, measure distance etc.
Fusing the information from different sensors usually increases the quality of the data significantly.
Localization ๐บ๏ธ
Most self-driving cars use a HD map. The map provides information about the geometry of the road, traffic rules and position of interesting objects (e.g. traffic lights).
Localization in the map is done using GPS and landmarks detected by the sensors.
Prediction ๐ฎ
The goal of this module is to predict the actions and trajectories of other traffic participants.
Will the car in front break?
Will the car cut in front of me?
Will the pedestrian cross the street?
This is crucial for the car to plan its own trajectory!
Planning ๐
In this step the car plans its own trajectory and actions. It needs to consider all other traffic participants, their intentions and the surrounding infrastructure.
One of the main objectives when planning is to maximize safety, but also to drive naturally.
Control ๐
The final step is controlling the throttle, breaks and steering so that the car actually drives the planned trajectory.
Here, it is important to have a smooth and natural control and not a steering wheel that twitches all the time.
If you liked this thread and want to read more about self-driving cars and machine learning give me a follow! ๐
I have many more threads like this planned ๐
โข โข โข
Missing some Tweet in this thread? You can try to
force a refresh
Open-Source Self-Driving Car Simulators ๐น๏ธ ๐
You want to play around with self-driving car software and gather some experience? Check out these open-source self-driving car simulators!
Details below ๐
CARLA
CARLA is a great software developed by Intel. You can use it to work on any step of the pipeline, model different sensors, maps, traffic. It also integrates with ROS.
Another great simulator by Voyage - the self-driving company that was recently acuired by Cruise. It is built on the Unreal Engine and supports lots of features.
Useful online courses on self-driving cars ๐ง ๐
Here is a list of useful courses if you want to learn about software for self-driving cars.
Some of the courses are paid, but all platforms offer regular discounts and financial aids if you can't affor them.
Thread ๐
Udacity Self-Driving Car Nanodegree
This program offers hands on experience on all kind of relevant topics like perception, localization, planning and control. It takes a lot of time, but it is worth it.
I plan many threads on self-driving cars and how to get into the industry.
I will link all of the individual threads that will be focused on a particular topic below.
๐งต
I recently gave a talk on AI for self-driving cars for a @DeepLearningAI_ Pie & AI event hosted by @Jeande_d. You can check out the recording on YouTube.
I will be posting threads summarizing the talk and link them below ๐