My Authors
Read all threads
Introducing RepNet, a model that counts repetitions in videos of *any* action

w @yusufaytar, @JonathanTompson, @psermanet and Andrew Zisserman


#CVPR2020 #computervision #deeplearning
Here's an overview of RepNet's architecture.
An integral part of RepNet is the temporal self-similarity matrix (TSM) which not only makes it easy to count repetitions but also drives generalization to actions and domains not seen during training.
What’s interesting is that we can train RepNet with synthetic data. We generate videos with repetitions by randomly sampling a clip of length N from a video and playing it over and over. This allows us to scale training to a large dataset without human annotation effort.
On the left, we show an example of a synthetic repeating video. These videos are too simple for the model. To make the task more difficult, we add camera motion augmentation to the video (shown on the right).

We train RepNet to predict the period from the video.
After being trained on synthetic videos, RepNet can count repetitions in real videos across many domains and actions.
RepNet can also detect changes in speed of the action.
Here we show an example of RepNet measuring the number of oscillations in a biological process as seen under a microscope. Video from…
We will be presenting this as a part of our paper “Counting Out Time: Class Agnostic Video Repetition Counting in the Wild” at #CVPR2020. Please come by our virtual poster (…) tomorrow 9AM -11 AM PDT or 9PM -11PM PDT for live discussion.
Missing some Tweet in this thread? You can try to force a refresh.

Keep Current with Debidatta Dwibedi

Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Follow Us on Twitter!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!