Discover and read the best of Twitter Threads about #stablediffusion

Most recents (24)

Delighted to share #stablediffusion with Core ML on Apple Silicon built on top of @huggingface diffusers! 🧵
Today's release of macOS Ventura 13.1 Beta 4 and iOS and iPadOS 16.2 Beta 4 include optimizations that let Stable Diffusion run with improved efficiency on the Apple Neural Engine as well as on Apple Silicon GPU
We share sample code for model conversion from PyTorch to Core ML and have example Python pipelines for text-to-image using Core ML models run with coremltools and diffusers
Read 7 tweets
🧵AI-assisted image prompts

1. In the last week, both @stabilityai and @openai have released major updates to their flagship offerings.

In this thread, we use OpenAI's new #ChatGPT model to help talk us through generating prompts for StabilityAI's new #stablediffusion v2 model.
2. ChatGPT3 is a new GPT3-powered model focused on conversing with users.

As with InstructGPT3 (the current default GPT3 model), @openai leveraged Reinforcement Learning from Human Feedback (RLHF) to boost the reliability and desirability of responses.
3. If you create an @openai account, you can currently test ChatGPT for free. We tried it out with the following prompt and were immediately impressed:

"I'm looking to create images of cats using a text-to-image model, similar to DALL-E. Can you suggest some prompts?"
Read 11 tweets
Thread time 🧵

Here's how to precisely design a small building in a game (such as an isometric bunker) by fine-tuning #StableDiffusion

This example was inspired by #RedAlert, which I spent countless hours on (in 96-97 - pls don't call me old 😅)

Style-consistency is paramount when it comes to designing #game assets.

I trained a fine-tune using @Scenario_gg (alpha), using 16 images (below), inspired by the Red Alert/Command & Conquer buildings.

Fun fact: I generated them all in... @midjourney.
I actually generated more than 200 buildings (and even some vehicles), from which I picked a smaller dataset, keeping enough variability within a certain consistent style.

I will keep the remaining data for new trainings in the coming days :)
Read 25 tweets
Fashion 2.0

First try training Stable Diffusion v2.0 (512-base).

Some individual photos + tech details in the🧵

#stablediffusion #ai #portrait #fashion
Fashion 2.0
Fashion 2.0 - 2
Read 8 tweets
Prompting Sucks:
authorized fine-tunings are the future

(a thread for AI-gen developers and enthusiasts)

#AIart #AIArtwork An AI-generated image by #dalle2 + the artist; horses cantorAn AI-generated image by #dalle2 + the artist; a portrait of
This current period of prompt engineering is going to be short lived. The majority are not here for the new niche of prompt design. People use these tools because there are things they want to see and experience in high fidelity, right now.
The minute someone can provide a better experience than:

"brilliant 8K [shot type] [film title], featuring [subject] [action], [environment], [lighting type], directed by [director/production company]"

with truly excellent results, people will come running.
Read 23 tweets
Broken Camera

Selected images from the Rejected pile.

If you're not a fan of cursed images, best not to watch this video.
#stablediffusion #fashion #photos
Broken Camera 1 ImageImageImageImage
Broken Camera 2 ImageImageImageImage
Read 4 tweets
🎉Introducing RoentGen, a generative vision-language foundation model based on #StableDiffusion, fine-tuned on a large chest x-ray and radiology report dataset, and controllable through text prompts!

@PierreChambon6 @Dr_ASChaudhari @curtlanglotz

🧵#Radiology #AI #StanfordAIMI Text-conditioned synthesis of CXR. Each image was hand-picke
#RoentGen is able to generate a wide variety of radiological chest x-ray (CXR) findings with fidelity and high level of detail. Of note, this is without being explicitly trained on class labels. Synthetic images created by prompting a fine-tuned model for
Built on previous work, #RoentGen is a fine-tuned latent diffusion model based on #StableDiffusion. Free-form medical text prompts are used to condition a denoising process, resulting in high-fidelity yet diverse CXR, improving on a typical limitation of GAN-based methods.
Read 13 tweets
Film Camera

Trained a new model for 'realistic photos' - whatever that ever means. It works pretty well.
Imagine where the quality will be in a year.
#stablediffusion #portrait
Film Camera - few individual portraits ImageImageImageImage
Film Camera 2 ImageImageImageImage
Read 7 tweets
I assume you have a stable diffusion set, if not get it from…
Model Anything 3.0…

drop it into directory \Stable Diffusion\models\Stable-diffusion

btw. waifu diffusion is cool too!
Read 11 tweets
I always thought #StableDiffusion prompts needed the right combination of words. But byte-pair encoding can represent anything you can type, including math formulas and emojis. Turns out you don't need any words at all! Here's how and why this works...🧵

Prompt: e=mc^2 Image
Prompts are fed to stable diffusion as binary code, with each letter/symbol represented as several bytes. Then a "tokenizer" looks for commonly occurring spans of adjacent bytes and groups them into a single known "word". Stable diffusion only knows 49408 words.

Here's "🧛🦇🗡️" ImageImage
You might think 49408 is a lot. Well, it's not. Here's the first 1200 words in the vocabulary. They don't get you very far. The words are auto-selected by a simple algorithm and half are junk. And what are the weird "�" entries? We'll get back to them later... Image
Read 10 tweets
1/ A noob level walkthrough on creating #stablediffusion art. At the end, I will have some links to help.

This thread will show I how created this.

Each step. Image
2/ First, you need to play with your text prompts. Start with something simple if you don't have a reference.

"warrior riding a horse, frank-frazetta, frazetta"

If you are using your own Stable Diffusion local copy, use "DPM fast" and low resolution to generate quickly. Image
3/ Be advised, these early rolls often suck. That's okay! Sometimes you get crap with extra legs sticking out of the head, or faces that are horrible. Whatever.

The goal at this stage is to craft the text prompt and settings. So you add details like "realistic" or "airbrushed".
Read 23 tweets
How game artists can explore more creative options with #AI, using their own art, in their own style.

A short (and yet powerful) example with @DanielPlaychain 🧵

#AI #gaming #game #artist #StableDiffusion
.@DanielPlaychain is the Chief Creative Officer @playchain_ (Web3 gaming).

He has posted some awesome game art on his @ArtStationHQ - check it out:
Daniel and I worked on a specific dataset of stylized characters (a style exploration, made a few years ago).

I focused on the characters below (removed the weapons, beast, and badges) and trained a finetune using @Scenario_gg on 7 images.
Read 14 tweets
100 #AI-generated droids, and they're cute.

Created in 15 minutes with @Scenario_gg
Some close-ups examples are below 👇

Like ❤️ if you like it, RT 🔁 if you want more of it 😇.
#AI #StableDiffusion #gaming #GameAssets
Read 5 tweets
Who wants some monstrously cute isometric gingerbread houses? 🏠🍭

I'm cooking a few hundred of them with @Scenario_gg ( and #StableDiffusion.

100% #AI-generated.

Tks @bcjordan for the inspiration.

Some more examples below 👇
Read 4 tweets
121 little characters with pointy ears. A small village.

All 100% designed with #AI.

Men, women, kids, soldiers, wizards, goblins, etc. All created in less than one hour, while keeping a solid style consistency.

Thread 🧵

#StableDiffusion #Dreambooth #Gaming @Scenario_gg
I bought a pack of characters with pointy ears. They made me think of the goblins in Harry Potter (but more human-like)

I trained a finetune using @Scenario_gg (internal alpha) and started prompting (calling them "persons").

At first, you get this, but then the fun begins.👇
A basic prompt, no modifier ("a person")

All are perfectly consistent (the clothes, the ears, the accessories). There are some variations, though.
Read 15 tweets
1/n I made a list of the 13 most mind-bending Stable Diffusion uses I’ve seen so far. 🌈
See🧵for details.

2/n Architectural Sketches
@rostenbach created incredible architectural renderings which made me think of future architecture generated by ai…
3/n Search Engine
@sharifshameem scrapped millions of SD-generated images and turned them into a @LexicaArt search engine for creative assets
Read 20 tweets

I've made some gems too. 100% #AI-generated 🤯

Sorry, I don't have much time for a detailed thread this time... too much work atm. I just wanted to put this out there and show it's really working for SO many game assets!

#StableDiffusion #Dreambooth #Scenario #AI
This is actually a fun one, I'll try sharing the model when I get a chance.
Read 11 tweets
New exploration: AI-generated fantasy game cards.

We're rushing to ship our new product, and it's hard to keep up w. everything (4 threads are waiting!)

This one started from a comment on my latest thread, mentioning #Hearthstone cards (@lewds_ai)

@PlayHearthstone is a free-to-play online digital collectible card game developed and published by Blizzard.

They make beautiful cards, and that's the training dataset I used (let's call this a #fanart experimentation).

20 images x 4000 steps, and I had the finetune 1h later
Now I wonder if I overfitted the model on this one, but there's a highlight. Every card on this dataset has the Hearthstone logo in the middle.

The emblem represents maps, a compass, a spyglass – "everything a real adventurer needs."

That's what I found 👇

Read 26 tweets
¿Te gustaría animar una imagen generada por #stablediffusion – incluso con #Dreambooth – con un video grabado de ti mismo? Aquí tienes otro ejemplo 👉

Pues sigue los pasos que te dejo a continuación que no son pocos :) Let's go! 🧵👇
1/ Carpeta para el proyecto.
2/ Grábate un clip de video con webcam + OBS o tu app favorito. Lo + importante:
- Mínimo HD.
- Cámara que se mueva despacio.
- Que no aparezcan elementos nuevos en el plano: objetos o partes de cuerpo.
- Bien iluminado
- 10 seg. de largo aprox…
3/ Escala a 512px de alto y luego recorta tu video a 512x512px con tu app favorita. También puedes hacerlo online con
3.1/ Guarda el video resultante en tu carpeta de proyecto. Escalar el video a 512px de...Recortar el video a 512x512...
Read 9 tweets
My first article for @returndotlife drops late on a Friday, & is a look at the SBF FTX implosion from a "wat mean?" perspective.…
By way of damage assessment, I start with a prescient December 2021 article and DM exchange from a crypto critic and friend of mine, @matthewstoller. Image
My recommendation is a vibe shift. Now, I know what you're thinking: easy to recommend but hard to pull off. There's good news, tho... Image
Read 6 tweets
I've explored a reliable method to create high-quality, style-consistent #game assets w. #StableDiffusion

We're just scratching the surface here & I believe this can become a very potent creation tool (like a "@procreate on AI-steroids"🤔)

A demonstration with... spellbooks 🧵
Spellbooks are another ubiquitous asset in heroic #fantasy games, tabletop games, D&D, etc.

However (like for potions), the Unity/Unreal asset stores have limited choice, and most assets have a rather simplistic design.

Here's how #AI can solve such a situation.
At first, I generated a set of 38 "random" spellbooks, which I used to train a finetune model of Stable Diffusion (Dreambooth). It took 1 hr.

I then explored a set of possibilities with simple prompts and or limited modifiers (intricate, detailed, beautiful, 3D render...)
Read 24 tweets
I designed the most incredible pack of potions with #AI (and nothing but AI - #StableDiffusion)

Mega-thread 🧵

Follow the exploration below, esp. if you're in the #gaming industry (Game dev, Game Artist, Creative Director, etc.) Content production is about to be transformed 🤯 ImageImage
The gaming industry always needs a lot of new content. 40% of their budget is in the art assets.

Take potions, a ubiquitous prop in RPG games.

Below are some of the “most popular” potion packs available on the Unity asset store (similar sets are on the Unreal marketplace). ImageImageImageImage
And these are some of the potions found on Artstation.

They look much more elaborate, and some designs are highly creative. However, there are only 740 results for the “potion” query, which seems limited. Image
Read 26 tweets
Okay, I believe this is the most INSANE set of assets I've designed so far with #StableDiffusion. Just ready for an excellent #RPG! Who wants some potions?

Take a look at the first selection below... the complete thread + how-to are coming in about 2-3 hours

#AI for #Gaming
Read 12 tweets

Related hashtags

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!