Share this page!

Maybe Scrolly?
Xander Steenbrugge Profile picture
Twitter logo
Apr 23 β€’ 7 tweets β€’ 3 min read
I discovered a bug in my own Diffusion + CLIP pipeline and suddenly the samples are unreal.. 🀯
Here's
"Just a liquid reality..."
#AIart #notdalle2 #Diffusion #clip Image
"The magnificent portal of mother Gaia" Image
"Framing reality" Image
"Gathering at the great elder sphere" Image
"Why such a rush? It's all twisting and bending anyway" Image
"My hair is a living creature" Image
Caveat: all these pieces are the result of a tremendous amount (months) of code and parameter tuning, careful selection of initialization images, prompt engineering and cherry picking.

#dalle2 is incredible at compositionality and realism, but I haven't seen it do this yetπŸ‘¨β€πŸŽ¨πŸ§™β€β™‚οΈπŸ˜‹

β€’ β€’ β€’

Missing some Tweet in this thread? You can try to force a refresh
γ€€

Keep Current with Xander Steenbrugge

Xander Steenbrugge Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @xsteenbrugge

Xander Steenbrugge Profile picture
Feb 24
This is a "3D-diffusion" video created using a combination of four different AI models🀯

Welcome to the metaverse! 🌌😎

There's such incredible potential here that I want to explain how I made this, so here's a thread! (1/n)
The two main models that draw the pixels are a diffusion model guided by a language prompt through @OpenAI's CLIP model.
This idea was introduced by @advadnoun and later refined by many other creatives. My talk at @Kikk_Festival further explains this:
The diffusion model (I integrated code from @RiversHaveWings and @Somnai_dreams for this) generates images by iteratively denoising noisy-pixel images, every time you run this from different noise, you get a different image, guided by the language prompt:
Read 10 tweets
Xander Steenbrugge Profile picture
Jan 21
Finally playing around with CLIP + diffusion models.

12 GPU hours in I gotta say I'm pretty impressed with the difference in esthetics compared to VQGANπŸ‘Œ
Big thanks to @RiversHaveWings & @Somnai_dreams for providing great starting code!

"a dystopian city"
"The real problem of humanity is that we have Paleolithic emotions, medieval institutions and godlike technology"
"The being"
Read 5 tweets
Xander Steenbrugge Profile picture
Dec 19, 2021
Just felt like sharing some beautiful images, these are still hot from the GPU...

"The elder sphere"
"The Engine"
"We live in golden bubbles of the mind"
Read 4 tweets
Xander Steenbrugge Profile picture
Oct 6, 2021
Niiice! Hooking this up to CLIP as soon as the weights are released πŸ€žπŸ€žπŸ˜‹
TLDR:
1. Replaces the CNN encoder and decoder with a vision transformer β€˜ViT-VQGAN’, leading to significantly better speed-quality tradeoffs compared to CNN-VQGAN
2. Vanilla VQVAE often learns rarily used / β€œdead” codebook vectors leading to wasted capacity. Here, they add a linear projection of the code vectors into a lower dimensional β€œlookup” space. This factorization of embedding / lookup consistently improves reconstruction quality.
Read 4 tweets
Xander Steenbrugge Profile picture
Oct 6, 2021
Note to self: don't use default matplotlib colormaps to make digital artπŸ€¦β€β™‚οΈπŸ˜…

New samples from my 'color-quantized VQGAN' are looking great!

Here's "π‘¨π’„π’„π’π’“π’…π’Šπ’π’ˆ 𝒕𝒐 π‘Ύπ’Šπ’•π’•π’ˆπ’†π’π’”π’•π’†π’Šπ’, 𝒂 π’‘π’Šπ’„π’•π’–π’“π’† π’Šπ’” 𝒂 π’Žπ’π’…π’†π’ 𝒐𝒇 π’“π’†π’‚π’π’Šπ’•π’š"

#clip #AIart
"π’Žπ’š 𝒉𝒆𝒂𝒅 π’Šπ’” 𝒇𝒖𝒍𝒍 𝒐𝒇 π’π’π’Šπ’”π’†"
"π‘΅π’Šπ’ˆπ’‰π’•π’Žπ’‚π’“π’†"
Read 7 tweets
Xander Steenbrugge Profile picture
Oct 5, 2021
Inspired by the amazing work of @HvnsLstAngel I've been experimenting with a "color-quantized VQGAN"
Essentially, I introduced a codebook of possible colors and apply quantization in rgb space.

It's always fascinating how removing entropy can make samples more interesting... ImageImage
"Inception" ImageImage
"The ancient temple of time" ImageImage
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(