Tweet

Mario Klingemann

24 Jan, 9 tweets, 3 min read

For an upcoming collaboration I created an algorithm that edits music videos autonomously by selecting the best matching scenes and cuts for a given song. Here's a test with some found glitch material.

Song: Time Traveler (eroded by time mix) by ghosts4hire
CC BY-NC 3.0

Full length version here:

And whilst in the end it does not really matter, just for understanding how this works: the glitches are already in the material and were not added afterward based on the sound.

Here's a sacrilege - "Ballet Mécanique" by Fernand Léger and Dudley Murphy edited to the sound of "The Promo that Got Away (Insert Story Here)" by ghosts4hire

As usual with AI results there are parts where I want to manually polish things, but I here's the raw output.

Not all matches are great in particular towards the end, but I like the brief moments when it seems to do lip syncing.

Song: Duckett Collage (Wired but Disconnected Mix)
by: spinningmerkaba

Same source material, different song. Some strange cuts in there, but overall I feel it doesn't do too bad.

Song: Slow Notes by Aussens@iter
dig.ccmixter.org/files/tobias_w…

Here is some AI-edited retro data-center porn, though some other type of reels also sneaked into the mix.

Song: Hot Mess by R3BEL-1
dig.ccmixter.org/files/rebelspi…

The full 5 minute version is here:

What the hell?

• • •

Missing some Tweet in this thread? You can try to force a refresh

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

More from @quasimondo

Mario Klingemann

@quasimondo

18 Jul 20

Another attempt at a longer piece. An imaginary Jerome K. Jerome writes about Twitter. All I seeded was the title, the author's name and the first "It", the rest is done by #gpt3

Here is the full-length version as a PDF:
drive.google.com/file/d/1qtPa1c…

I am really happy about the spelling of "to-morrow" at the end of page 2.

I made another one in a similar spirit - "On Wireless Fidelity" #gpt3
Complete version here: drive.google.com/file/d/1FW7zki…

Read 9 tweets

Mario Klingemann

@quasimondo

17 Jul 20

It's like programming, but with free text. I don't think a five-year old could do that. #gpt3

It took me three attempts to refine my instructions (on the left) until the model understood what I wanted. Output on the right.

There is more...

Talk about generalization. (And playing a nice trick in the second one - at first I thought it was falling in the classic mix-up trap, but did a great save here.)

Read 5 tweets

Mario Klingemann

@quasimondo

28 Feb 20

Impressions at the super market.

https://twitter.com/Lokoschat/status/1233506824621608961?s=19

Collect them all...

https://twitter.com/Lokoschat/status/1233506824621608961?s=19

https://twitter.com/MissMimiilein/status/1233483403351281665?s=19

https://twitter.com/MissMimiilein/status/1233483403351281665?s=19

Read 114 tweets

Mario Klingemann

@quasimondo

6 Nov 19

"The only good time to be alive is the time after you've died."
-- Anonymous AI #GPT2

"Happiness is like a rose by itself in the garden; when it blooms, the birds will sing and the bees will make honey."
-- Anonymous AI #GPT2

"The difference between the artist and the critic is that the critic looks for a pattern, while the artist recognizes the pattern."
-- Anonymous AI #GPT2

Read 22 tweets

Mario Klingemann

@quasimondo

21 Jan 19

I've created an experimental GAN architecture I call #RecuResGAN or "Recursive-Residual GAN" and I am pretty astonished that:
- it works at all
- how well it works across a pretty wide range of scales.
- it is just 15% the size of a comparable #pix2pixHD model

Of course I did not google the concept of recursive neural networks before I started this experiment and enjoyed the illusion of being very innovative here for a whole day:
en.wikipedia.org/wiki/Recursive…

The principle is pretty simple: in a classic residual architecture you chain several residual blocks behind each other (in #pix2pixHD the default is 9 blocks), what I do in #RecuResGAN is to use a single block, but loop 9 times over it, feeding its output back into its input.

Read 5 tweets