New blog post!🕺🥳
This is about a recent paper I enjoyed called "Omnimatte: How to Detect Objects and Their Effects."
I love writing these kind of summary blog posts because I get to learn about such cool papers.
wandb.ai/_scott/omnimat…
1/5
This is about a recent paper I enjoyed called "Omnimatte: How to Detect Objects and Their Effects."
I love writing these kind of summary blog posts because I get to learn about such cool papers.
wandb.ai/_scott/omnimat…
1/5
The model in this paper learns to associate one or more objects to the effects they have on their environment (shadows, reflections, etc.) for a given video and rough segmentation masks of each object. This enables video effects like "background replacement".
2/5
2/5
and "color pop" and a "stroboscopic" effect (in the next tweet):
3/5
3/5
It's trained using self-supervision with a really interesting loss function that I go through in the post.
The main objective is to reconstruct the original video as a composition of each layer (and the background), but there's lots of other aspects that improve the model.
4/5
The main objective is to reconstruct the original video as a composition of each layer (and the background), but there's lots of other aspects that improve the model.
4/5
Hope you enjoy the blog post and check out the original work.
Blog post: wandb.ai/_scott/omnimat…
Paper website: omnimatte.github.io
Paper: arxiv.org/pdf/2009.07833…
5/5
Blog post: wandb.ai/_scott/omnimat…
Paper website: omnimatte.github.io
Paper: arxiv.org/pdf/2009.07833…
5/5
• • •
Missing some Tweet in this thread? You can try to
force a refresh
