Self-driving car engineer roles - Big Data Engineer π½
Self-driving cars have lots of cameras, lidars and radars. Waymo currently has 29 cameras on a single vehicle! The cars generate huge amounts of data, easily more than 1 GB/s. This data needs to be processed...
Thread π
Problems to work on π€
The big data engineer needs to design and implement efficient storage and data processing pipelines to handle such large amounts of data.
The data also needs to be made available to the developers in a way that they can efficiently get to what they need.
Data πΎ
Imagine that the self-driving car is recording data at a rate of 1 GB/s. Going on a test drive for 4 hours means that you'll collect more than 14 TB of data!
There are specialized loggers that can handle such rates, like this beast for example: vigem.de/en/content/proβ¦
Data Centers πΏπ
The processing of such amounts of data happens in distributed data centers. Here are some technologies that are commonly used:
βͺοΈ Kubernetes (container orchestration)
βͺοΈ Hadoop (storage and processing)
βͺοΈ Spark (compute)
βͺοΈ Airflow (orchestration)
Required knowledge π
If you have experience with processing of huge amounts of data you have a good chance of entering the industry. Experience with any of the technologies above or similar ones will help you!
Programming language π»
This depends on the stack a particular company is using, but Java and Scala are popular languages for many of the Big Data frameworks. Python is also used for analyzing data. You may also need to use C++ for code that needs to run in an efficient manner.
β’ β’ β’
Missing some Tweet in this thread? You can try to
force a refresh
Self-driving car engineer roles - Computer Vision Engineer π
The camera is one of the most important sensors! It is not always the most accurate one, but it can provide much more data than a lidar or radar. Extracting this data is the job of the CV engineer.
Thread π
Problems to work on π€
Here some typical object classes that need to be detected and classified.
π Traffic signs
π¦ Traffic lights
π Vehicles
πΆ Pedestrians
π¦ Animals
π£οΈ Lane markings
ποΈ Landmarks
π§ Construction zones
𧱠Obstacles
π Police cars
Distance estimation π
Detecting an object is not enough, though. You also want to know how far the object is from the car. While the detection part is dominated by deep learning, the traditional CV methods (e.g. Kalman Filter) are still very useful for distance estimation.
Self-driving car engineer roles - Software Engineer π»
There are many specialized roles in a self-driving car project, like ML or CV engineers. However, every projects needs lots of good software devs - you can enter the industry even without specific knowledge!
Thread π
Problems to work on π€
Some problems that software developers work on to build a self-driving car (the list is not exhaustive):
- HMI
- Operating system
- Logging and tracing
- Communication between ECUs
- Internal frameworks and libraries
- Implementing diagnostic interfaces
Software engineers also work closely with many of the more specialized roles.
For example with Machine Learning engineers to implement models on the ECU or with Vehicle Control engineers to get their algos working efficiently.
Some interesting self-driving news lately:
- Waymo launching test fleet without safety driver
- Tesla launching a beta of their Full Self-Driving
- Mercedes announcing a level 3 traffic jam pilot for 2021
There are 3 very different approaches π
1οΈβ£ "Everything that fits" approach.
This is Waymo's approach, but other companies like Cruise, Argo, Aurora, Uber, Zoox have a similar strategy.
Fit as much sensors as possible on the car, build high-definition maps of the environment and throw in lots of compute power.
Check out some images of these cars - they all have multiple lidars, cameras and radars all around the car. Waymo now has 29 cameras! π²
They are not really integrated in a consumer oriented way, but it should be fine for a robotaxi.
How to become a self-driving car engineer? π»π§ π
Well, there is no easy answer - building a self-driving car requires expertise from many different fields.
The good new is that there are many paths that will lead you to a job in this industry!
Read more details below π
There are many different engineering roles in a self-driving car project:
π» Software Development
π§ Machine Learning
π Computer Vision
ποΈ Vehicle Control
βοΈ Hardware
π½ Big Data
πΉοΈ Simulation
π Mapping
π§ Safety
π Security
π Test and Validation
π¨ Tooling
and more... π
Over the next days I'll share some of my experience working in the industry and describe in more details the different roles and what skills are required to enter the field.
A lot of the technology behind the FSD software was presented at a conference in February. Check out this video, it is really interesting!
Main point is Tesla is using a "Bird's eye view" neural network to predict the layout of the road.
You can see this in action in the car - you see all lines defining the lanes and the road boundaries in the display. They seem to be coming from the vision system and not from a map, because they are very unstable and often not very accurate.