I continued exploring #stablediffusion's latent space over the weekend and oh my; there's still a LOT of treasure to be discovered inside this magnificent neural universe!

Here's a quick thread with some of my personal favorites and how I found them.. The fact that all this visual splendor is compressed in just 4Gb of neural network weights totally blows my mind. Call it compression, call it emergence, it's just 🤯🤯

Getting bored by a StyleGAN model after looking at samples for 20 minutes seems like a very distant past now..

Ok, so first of all, #stablediffusion did not come with code to make videos, so I came up with a way to interpolate between encoded prompt vectors (no worries if you don't know what that means) and thereby create video sequences from prompt sequences (1/n)

https://twitter.com/xsteenbrugge/status/1556099146762919937?t=elwa804pYDGdmmlB8Q-Y6g&s=19

https://twitter.com/LUKEILLUSTRATE/status/1559697309184802819

Next, I had to come up with a visual narrative that would work well with the style of the Diffusion interpolations. You can't just tell any story here: like with any medium, you have to work within the constraints of the technology. (2/n)

"Voyage through Time"
is my first artpiece using #stablediffusion and I am blown away with the possibilities...

We're crossing a threshold where generative AI is no longer just about novel aesthetics, but evolving into an amazing tool to build powerful, human-centered narratives This video was created using 36 consecutive phrases that define the visual narrative.

To find the best possible sequence, I tried over a thousand different prompts and seeds and applied many "prompt engineering" tricks in my code, to figure out what works and what doesn't

I discovered a bug in my own Diffusion + CLIP pipeline and suddenly the samples are unreal.. 🤯
Here's
"Just a liquid reality..."
#AIart #notdalle2 #Diffusion #clip "The magnificent portal of mother Gaia"

This is a "3D-diffusion" video created using a combination of four different AI models🤯

Welcome to the metaverse! 🌌😎

There's such incredible potential here that I want to explain how I made this, so here's a thread! (1/n) The two main models that draw the pixels are a diffusion model guided by a language prompt through @OpenAI's CLIP model.
This idea was introduced by @advadnoun and later refined by many other creatives. My talk at @Kikk_Festival further explains this:

Finally playing around with CLIP + diffusion models.

12 GPU hours in I gotta say I'm pretty impressed with the difference in esthetics compared to VQGAN👌
Big thanks to @RiversHaveWings & @Somnai_dreams for providing great starting code!

"a dystopian city" "The real problem of humanity is that we have Paleolithic emotions, medieval institutions and godlike technology"

Just felt like sharing some beautiful images, these are still hot from the GPU...

"The elder sphere" "The Engine"

Niiice! Hooking this up to CLIP as soon as the weights are released 🤞🤞😋

https://twitter.com/ak92501/status/1445872262771478529

TLDR:
1. Replaces the CNN encoder and decoder with a vision transformer ‘ViT-VQGAN’, leading to significantly better speed-quality tradeoffs compared to CNN-VQGAN

Note to self: don't use default matplotlib colormaps to make digital art🤦‍♂️😅

New samples from my 'color-quantized VQGAN' are looking great!

Here's "𝑨𝒄𝒄𝒐𝒓𝒅𝒊𝒏𝒈 𝒕𝒐 𝑾𝒊𝒕𝒕𝒈𝒆𝒏𝒔𝒕𝒆𝒊𝒏, 𝒂 𝒑𝒊𝒄𝒕𝒖𝒓𝒆 𝒊𝒔 𝒂 𝒎𝒐𝒅𝒆𝒍 𝒐𝒇 𝒓𝒆𝒂𝒍𝒊𝒕𝒚"

#clip #AIart

https://twitter.com/xsteenbrugge/status/1445327569847521281

"𝒎𝒚 𝒉𝒆𝒂𝒅 𝒊𝒔 𝒇𝒖𝒍𝒍 𝒐𝒇 𝒏𝒐𝒊𝒔𝒆"

Inspired by the amazing work of @HvnsLstAngel I've been experimenting with a "color-quantized VQGAN"
Essentially, I introduced a codebook of possible colors and apply quantization in rgb space.

It's always fascinating how removing entropy can make samples more interesting... "Inception"