Thread by @greentheonly

Holiday weekends are great for some fun are not they?

Worked with @rice_fry on the Tesla depth perception NN.

Remember that stuff in ?

Well, the cars actually have a depth perceiving net inside indeed. though unlike what's shown it's a lot lower res

A whopping 160x120 grid (so 1/8th of the native camera resolution) but hey those are actual 3D points in space. I don't know how to make a 3D video, so just visualizing "distance" as "brighter = closer" on the scale of 5 to 62 meters (the actual limit of the output):

Here's the corresponding time-synced video to better understand what you are looking at.

Pretty cool actually despite all the limitations.

you might also notice there's a strange white line in the middle.
What I think is going on is Tesla trained against main + fisheye cams and on the older cars there's a heater element (shown from old screenshot) introducing garbage into the stream.

That's why some think they are retraining with lidar now - because pure optical flow is too hard.

Here's also an example of merged objects shown as one that then corrects into two separate ones later that might be helped with the lidar "groundtruth" as well.

The sky rarely changing might be also contributing to the artifacts at the top.

the above segment with pedestrians in dynamic

Some short city driving

And this one is visualizing another axle of points.

The effect is super nice, reminds me of the "Hedgehog in the fog" somewhat:

visualized single frames of point clouds (do NOT load this in firefox if you can avoid it), scrolable/zoommable/rotating with mouse