Discover and read the best of Twitter Threads about #img2img

Most recents (12)

I wanted to imagine how we’d better use #stablediffusion for video content / AR.

A major obstacle, why most videos are so flickery, is lack of temporal & viewing angle consistency, so I experimented with an approach to fix this

See 🧵 for process & examples
Ideally you want to learn a single representation of an object across time or different viewing directions to perform a *single* #img2img generation on.

For this I used (2021)
This learns an "atlas" to represent an object and its background across the video.

Regularization losses during training help preserve the original shape, with a result that resembles a usable slightly "unwrapped" version of the object Image
Read 6 tweets
Thread time 🧵

Here's how to precisely design a small building in a game (such as an isometric bunker) by fine-tuning #StableDiffusion

This example was inspired by #RedAlert, which I spent countless hours on (in 96-97 - pls don't call me old 😅)

Style-consistency is paramount when it comes to designing #game assets.

I trained a fine-tune using @Scenario_gg (alpha), using 16 images (below), inspired by the Red Alert/Command & Conquer buildings.

Fun fact: I generated them all in... @midjourney.
I actually generated more than 200 buildings (and even some vehicles), from which I picked a smaller dataset, keeping enough variability within a certain consistent style.

I will keep the remaining data for new trainings in the coming days :)
Read 25 tweets
I've explored a reliable method to create high-quality, style-consistent #game assets w. #StableDiffusion

We're just scratching the surface here & I believe this can become a very potent creation tool (like a "@procreate on AI-steroids"🤔)

A demonstration with... spellbooks 🧵
Spellbooks are another ubiquitous asset in heroic #fantasy games, tabletop games, D&D, etc.

However (like for potions), the Unity/Unreal asset stores have limited choice, and most assets have a rather simplistic design.

Here's how #AI can solve such a situation.
At first, I generated a set of 38 "random" spellbooks, which I used to train a finetune model of Stable Diffusion (Dreambooth). It took 1 hr.

I then explored a set of possibilities with simple prompts and or limited modifiers (intricate, detailed, beautiful, 3D render...)
Read 24 tweets
I designed the most incredible pack of potions with #AI (and nothing but AI - #StableDiffusion)

Mega-thread 🧵

Follow the exploration below, esp. if you're in the #gaming industry (Game dev, Game Artist, Creative Director, etc.) Content production is about to be transformed 🤯 ImageImage
The gaming industry always needs a lot of new content. 40% of their budget is in the art assets.

Take potions, a ubiquitous prop in RPG games.

Below are some of the “most popular” potion packs available on the Unity asset store (similar sets are on the Unreal marketplace). ImageImageImageImage
And these are some of the potions found on Artstation.

They look much more elaborate, and some designs are highly creative. However, there are only 740 results for the “potion” query, which seems limited. Image
Read 26 tweets
Sharing additional thoughts on #StableDiffusion to create #gaming content & #game assets 🧵

My last exploration was on golems; this one is with "Space Marines-like" heavy infantry.

This was a fun creation with lots of learnings 👨‍💻. Feel free to like/RT if you find it useful 🚀
1/ First (as mentioned before - ) > curate a training dataset (such as pictures of figurines) to feed a #Dreambooth finetune.

Once the model is ready, compare different generic prompts (e.g. "low-poly," "3D rendering“, "ultra-detailed," "pixel art," etc.)
2/ Once a prompt looks good, keep the modifiers (in this case: "3D rendering, highly detailed, trending on Artstation").

And start iterating around variations, such as colors (or pose). Don't over-engineer it, to keep the consistency. You should get this:
Read 15 tweets
Let’s keep using #StableDiffusion to design or create #game assets, quickly and consistently.

In 15 min, generated an army of golems with similar shapes and sizes, but various “materials” (lava, rock, water, ice, forest, sand, gold…).

No in-painting is needed. Thread 🧵
Step 1 (as in my previous examples with chests - ), I made a Dreambooth finetune w. just 11 pics.

As soon as the model was trained, the first step was randomly generating a large set of images w. a simple prompt ("golem, detailed, realistic, 3D rendering")
Each golem can be extracted by removing the images' backgrounds (reco: @photoroom_app).

Some of the designs are amazing. However, the golems all look very similar to each other. Let's separate them into categories.
Read 14 tweets
People loved the "treasure chests" #AI experiment I posted yesterday! However, the chests were all closed 🤨.

So I quickly ran a second training on a different dataset, and this time, gold and diamonds are overflowing from the open chests! 🪙🤑💰

I also used #img2img to instantly generate dozens of variants, "inspired“ by a single original photograph.

This provides consistent assets (similar shape, size, or materials) with some slight variations. It's up to the artist/user to select which one looks best.
Same thing here, using #img2img - however, I prompted "steel chest" instead of "wooden chest"

While it's not 100% perfect, there's still more steel in these chests than in the previous ones.

Also, some of the assets are disjoint or show anomalies. Some fine-tuning is necessary
Read 4 tweets
#Dreambooth will change how 2D game assets are being designed or generated.

Today's exploration: treasure chests 💎💰.

I trained Stable Diffusion with some low-poly treasure chests (23 images). 1h later, here's what's possible (🧵): ImageImageImageImage
At first, I randomly generated a set of 64 items, without any specific prompt engineering ("a treasure chest"). Image
Here's a close-up view: Image
Read 11 tweets
Tonight's dreamboothing is about grenades 💣(!).

I made a #dreambooth finetune from a few Sci-Fi grenade illustrations.

Explosive thread 🧵👇

#AI #StableDiffusion ImageImageImageImage
First, some random grenades, no specific adjectives. Image
But they look great in blue too! ImageImageImageImage
Read 14 tweets
1/ I created this with Stable Diffusion using image inpainting and “walking through the latent space”

Without using tweening, every frame is generated by an interpolated embedding and variable denoising strength, so keeping continuity was tricky

See 🧵for process
2/ First off, finding the right combination of prompt, seed and denoising strength for an #img2img in-painting is a roll of the dice

Luckily it is easy to script large batches to cherrypick
3/ The first and last pairs were just regular #img2img ramped through a range of denoising strength of 0 to 0.8
Read 6 tweets
Took a face made in #stablediffusion driven by a video of a #metahuman in #UnrealEngine5 and animated it using Thin-Plate Spline Motion Model & GFPGAN for face fix/upscale. Breakdown follows:1/9 #aiart #ai #aiArtist #MachineLearning #deeplearning #aiartcommunity #aivideo #aifilm
First, here's the original video in #UE5. The base model was actually from @daz3d #daz and I used Unreal's #meshtometahuman tool to make a #metahuman version. 2/9 #aiartprocess
Then I took a single still frame from that video and ran it through #img2img in a local instance of #stablediffusion WebUI. After generating a few options I ended up with this image. 3/9 #aiartprocess Image
Read 9 tweets
イラスト制作におけるStable Diffusionの活用事例
#stablediffusion #img2img ImageImageImageImage
シルエットと大まかなライティング、色合いくらいまで描き込んでStrength 0.6で出力すると、良い感じにニュアンスを汲み取ってくれる
ともに同じプロンプト、同じStrengthで生成しています ImageImage
Read 3 tweets

Related hashtags

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!