Note to self: don't use default matplotlib colormaps to make digital art๐Ÿคฆโ€โ™‚๏ธ๐Ÿ˜…

New samples from my 'color-quantized VQGAN' are looking great!

Here's "๐‘จ๐’„๐’„๐’๐’“๐’…๐’Š๐’๐’ˆ ๐’•๐’ ๐‘พ๐’Š๐’•๐’•๐’ˆ๐’†๐’๐’”๐’•๐’†๐’Š๐’, ๐’‚ ๐’‘๐’Š๐’„๐’•๐’–๐’“๐’† ๐’Š๐’” ๐’‚ ๐’Ž๐’๐’…๐’†๐’ ๐’๐’‡ ๐’“๐’†๐’‚๐’๐’Š๐’•๐’š"

#clip #AIart
"๐’Ž๐’š ๐’‰๐’†๐’‚๐’… ๐’Š๐’” ๐’‡๐’–๐’๐’ ๐’๐’‡ ๐’๐’๐’Š๐’”๐’†"
"๐‘ต๐’Š๐’ˆ๐’‰๐’•๐’Ž๐’‚๐’“๐’†"
"๐‘ฐ๐’๐’„๐’†๐’‘๐’•๐’Š๐’๐’"
"๐’‚ ๐’๐’Š๐’—๐’Š๐’๐’ˆ ๐’…๐’†๐’Ž๐’๐’„๐’“๐’‚๐’„๐’š ๐’ˆ๐’“๐’๐’˜๐’” ๐’๐’Š๐’Œ๐’† ๐’‚ ๐’•๐’“๐’†๐’†, ๐’‡๐’“๐’๐’Ž ๐’•๐’‰๐’† ๐’ƒ๐’๐’•๐’•๐’๐’Ž ๐’–๐’‘"
Quote by @drvandanashiva
"๐‘ฎ๐’๐’๐’…๐’†๐’ ๐’‡๐’‚๐’„๐’†๐’”"
For the collectors: I've minted some of these on my HEC: hicetnunc.xyz/xander

For the techies:
Turns out sending gradients straight through this rgb-quantization is not great for stability, so I'm also minimizing mean(quant_distances) to keep raw img close to quantized one!

โ€ข โ€ข โ€ข

Missing some Tweet in this thread? You can try to force a refresh
ใ€€

Keep Current with Xander Steenbrugge

Xander Steenbrugge Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @xsteenbrugge

6 Oct
Niiice! Hooking this up to CLIP as soon as the weights are released ๐Ÿคž๐Ÿคž๐Ÿ˜‹
TLDR:
1. Replaces the CNN encoder and decoder with a vision transformer โ€˜ViT-VQGANโ€™, leading to significantly better speed-quality tradeoffs compared to CNN-VQGAN
2. Vanilla VQVAE often learns rarily used / โ€œdeadโ€ codebook vectors leading to wasted capacity. Here, they add a linear projection of the code vectors into a lower dimensional โ€œlookupโ€ space. This factorization of embedding / lookup consistently improves reconstruction quality.
Read 4 tweets
5 Oct
Inspired by the amazing work of @HvnsLstAngel I've been experimenting with a "color-quantized VQGAN"
Essentially, I introduced a codebook of possible colors and apply quantization in rgb space.

It's always fascinating how removing entropy can make samples more interesting... ImageImage
"Inception" ImageImage
"The ancient temple of time" ImageImage
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!

:(