Thread by @MarkNeumannnn

If I write a paper one day that is 1/10th as creative and interesting as this one, I will be very happy:
arxiv.org/pdf/1904.11111…

Idea - build a dataset for Monocular Depth Estimation by collecting a set of videos from the "Mannequin Challenge" 1/n

Then use motion parallax to infer the depth of all static points (which is possible because everyone/thing in the video is being helpfully very still). This is already cool, but in addition, they use a 2D segmentation model to mask out the people. 2/n

Why is this cool? Well, for the static videos, the people don't move, so it's easy to predict their depth. But in actual videos, people do move - so if the network is also trained to predict the depth of the masked out humans, it can transfers nicely to a real video 3/n

They show in the paper that you can use this for all kinds of cool visual effects, but the real killer application is Monocular SLAM/3D mapping (monocular depth estimation was hard when the camera is also moving, this looks like a massive step towards fixing that). 4/n

The extensions to this are extremely obvious - use a segmentation model for self-driving cars and slurp up a bunch of data from google maps. Mask out the cars, and boom - you've got the same thing for any moving object that you can accurately segment. Pretty cool. 5/n

I wish in NLP we had such creative datasets! The overwhelming feeling at the moment is "We enslaved a bunch of Turkers who don't care about your task or data..... hopefully this captures all of semantics/syntax/common sense?" (please point me to such datasets if they exist) 6/n

Link to blog post: ai.googleblog.com/2019/05/moving…