Self-driving car engineer roles - Computer Vision Engineer π
The camera is one of the most important sensors! It is not always the most accurate one, but it can provide much more data than a lidar or radar. Extracting this data is the job of the CV engineer.
Thread π
The camera is one of the most important sensors! It is not always the most accurate one, but it can provide much more data than a lidar or radar. Extracting this data is the job of the CV engineer.
Thread π
Problems to work on π€
Here some typical object classes that need to be detected and classified.
π Traffic signs
π¦ Traffic lights
π Vehicles
πΆ Pedestrians
π¦ Animals
π£οΈ Lane markings
ποΈ Landmarks
π§ Construction zones
𧱠Obstacles
π Police cars
Here some typical object classes that need to be detected and classified.
π Traffic signs
π¦ Traffic lights
π Vehicles
πΆ Pedestrians
π¦ Animals
π£οΈ Lane markings
ποΈ Landmarks
π§ Construction zones
𧱠Obstacles
π Police cars
Distance estimation π
Detecting an object is not enough, though. You also want to know how far the object is from the car. While the detection part is dominated by deep learning, the traditional CV methods (e.g. Kalman Filter) are still very useful for distance estimation.
Detecting an object is not enough, though. You also want to know how far the object is from the car. While the detection part is dominated by deep learning, the traditional CV methods (e.g. Kalman Filter) are still very useful for distance estimation.
Semantic segmentation π¨
Semantic segmentation is useful for detecting objects and areas in the image that don't have a well defined shape, like buildings, trees, the free space on the road and so on. It can also serve as a basis for object detection algorithms.
Semantic segmentation is useful for detecting objects and areas in the image that don't have a well defined shape, like buildings, trees, the free space on the road and so on. It can also serve as a basis for object detection algorithms.
Depth estimation π
A self-driving car needs a good 3D model of the surrounding. While lidar (and to some extent radar) can provide that, the camera can be seen as a redundant sensor.
There are different approaches:
- Stereo cameras
- Structure-from-motion
- Single frame depth
A self-driving car needs a good 3D model of the surrounding. While lidar (and to some extent radar) can provide that, the camera can be seen as a redundant sensor.
There are different approaches:
- Stereo cameras
- Structure-from-motion
- Single frame depth
Relevance βοΈ
Another important aspect is how relevant is an object:
- Is the car in my lane or not?
- Is the red or the green traffic light relevant for my lane?
- Is the pedestrian going to jump on the road in front of the car?
Here, fusion with other sensors can help as well.
Another important aspect is how relevant is an object:
- Is the car in my lane or not?
- Is the red or the green traffic light relevant for my lane?
- Is the pedestrian going to jump on the road in front of the car?
Here, fusion with other sensors can help as well.
Required knowledge π
Nowadays, deep learning and CNNs are a must for a computer vision engineer. However, don't underestimate the importance of traditional CV methods, especially around tracking, distance estimation and depth estimation!
Nowadays, deep learning and CNNs are a must for a computer vision engineer. However, don't underestimate the importance of traditional CV methods, especially around tracking, distance estimation and depth estimation!
Programming language π»
Computer vision is very computationally intensive, so good C++ skills are required. However, for training and evaluation CNNs you usually only need Python!
Computer vision is very computationally intensive, so good C++ skills are required. However, for training and evaluation CNNs you usually only need Python!
Getting started ποΈ
There are again many ways to enter the industry as a CV engineer. Pick one of the problems above and focus on it to gain knowledge and experience!
Example project - benchmark.ini.rub.de/?section=gtsrbβ¦
Check below for some free learning resources π
There are again many ways to enter the industry as a CV engineer. Pick one of the problems above and focus on it to gain knowledge and experience!
Example project - benchmark.ini.rub.de/?section=gtsrbβ¦
Check below for some free learning resources π
Resources π
Check out the following resources to start learning:
- Deep Learning for CV: coursera.org/learn/convolutβ¦
- Classical CV: udacity.com/course/introduβ¦
- TensorFlow: tensorflow.org/resources/learβ¦
- PyTorch: coursera.org/learn/deep-neuβ¦
- OpenCV: docs.opencv.org/master/d9/df8/β¦
Check out the following resources to start learning:
- Deep Learning for CV: coursera.org/learn/convolutβ¦
- Classical CV: udacity.com/course/introduβ¦
- TensorFlow: tensorflow.org/resources/learβ¦
- PyTorch: coursera.org/learn/deep-neuβ¦
- OpenCV: docs.opencv.org/master/d9/df8/β¦
β’ β’ β’
Missing some Tweet in this thread? You can try to
force a refresh
γ
