I experimented with different ways of asking CLIP+VQGAN for an attractive version of "a herd of sheep grazing on a lush green hillside"
"Award winning national geographic photography" produced impressive scenery but the sheep look like people crawling under green blankets.
Adding "by Bob Ross" to "a herd of sheep grazing on a lush green hillside" did get CLIP+VQGAN to improve the composition dramatically, but gave all the sheep Bob Ross hair.
Adding "by Tim Burton" to "a herd of sheep grazing on a lush green hillside" got CLIP+VQGAN to generate this very cool looking image. Not sure what happened to the sheep though.
I hate that one of the most effective ways to prompt CLIP+VQGAN to generate a realistic and attractive landscape is to ask for this:
"A herd of sheep grazing on a lush green hillside | dramatic atmospheric ultra high definition free desktop wallpaper"
Using the spammy "A herd of sheep grazing on a lush green hillside | dramatic atmospheric ultra high definition free desktop wallpaper" prompt as a starting point leads CLIP+VQGAN to some irritatingly gorgeous places.
Here, I added "cubist cezanne".
I had VQGAN+CLIP generate "A herd of sheep grazing on a lush green hillside | dramatic atmospheric ultra high definition free desktop wallpaper by lisa frank" and got this absolutely apocalyptic landscape.
I think those slippery purple things may be what's become of the sheep.
This experiment illustrates an interesting aspect of generating stuff with big internet-trained models: it's seen a lot of crummy examples of what you're looking for, and those are just as valid to it as the good ones.
It CAN generate the good stuff. But how do you ask for it?
For more technical details on CLIP+VQGAN and other methods of steering CLIP, plus some gorgeous example images, I recommend this post by @sea_snell
No, you're absolutely right, the sheep are uniformly cursed.
Part of why I chose a herd of grazing sheep is image recognition algorithms have historically had trouble with distinguishing the sheep from the landscape: aiweirdness.com/post/171451900…