Discover and read the best of Twitter Threads about #ControlNet

Most recents (7)

🧠 AI experiment comparing #ControlNet and #Gen1. Video goes in ➡ Minecraft comes out.

Results are wild, and it's only a matter of time till this tech runs at 60fps. Then it'll transform 3D and AR.

How soon until we're channel surfing realities layered on top of the world?🧵
First ControlNet. Wow, this tool makes it very easy to get photorealistic results. I used the HED method for this scene and got some amazing results. I used EbSynth for smoother interpolation between ControlNet keyframes. Check out my prior posts for the end-to-end workflow.
Next up Gen 1: impressive is the word. The star of the show is the temporal consistency. Getting photoreal results is harder than ControlNet IMO. #Gen1 is almost its own stylized thing, so I advise leaning into that. But why does it matter - can't we just type text to get video?
Read 8 tweets
Testing Multi-ControlNet on a scene with extended dialog, a range of facial expressions, and head turning. No EBSynth used. #AImusic from aiva.ai. Breakdown🧵1/15 #aicinema #controlnet #stablediffusion #aiia #aiartcommunity #aiva #machinelearning #deeplearning #ai
Overall flow: pre-process video > img seq > play with prompts > initial controlnet settings > control net batch render > upscale > clean up in post > key out background > deflicker > video post-production 2/15
The approach I used here was figuring out an initial workflow. There’s definitely a lot to play with and improve on. The orig vid is low-res and a little blurry so I started with a pre-process pass by bringing out the edges/brightness & upscaling. 3/15
Read 15 tweets
Will scale pay dividends in diffusion models? Here is Alibaba's Composer with 5B parameters. Dall-e 2 has 3.5B parameters, and StableDiffusion has 890m parameters.
Browsing through the examples, there are only a few things that #stablediffusion with #controlnet cannot do. There aren't any Color palette and intensity ControlNet networks. But certainly this can be trained.
The generality of modular networks appears to trump scale. medium.com/intuitionmachi…
Read 11 tweets
👉#Blender + #ControlNet prototype. Pose any character (I'm using Mixamo) right in Blender, hit F12, Booooooom! You don't have to leave Blender now, using A1111 in the background. #AIart #stablediffusion #b3d #aigirls
ImageImage
just a quick share, will opensource when i finish these messy codes.😉
Read 5 tweets
ControlNet experiment where I'm toggling through different styles of contemporary Indian décor, while keeping a consistent scene layout.

Loving how ControlNet is putting the artists back in control of AI image generation process.

🧵Thread

#ControlNet #StableDiffusion #EbSynth
For the input sequence I use a short animation made from a photogrammetry 3d scan I did a few years back of my parents living room in India.

- Top: Output generated with ControlNet + EbSynth

- Bottom: Input video sequence from my 3D scan
I used the ControlNet depth method for this experiment.

- Left: MiDaS depth map generated from a screen recording of my 3d scan

- Right: Awesome results reskinning the room! Pretty majestic if I say so myself

Next time I want to use the synthetic depth from my 3d scan
Read 6 tweets
#stablediffusion
👁️如何一键安装 #Chilloutmix 到云端电脑?
完全免费!不用下载任何软件到本地?
而且完全在google云端运行🤓
我来教你快速使用Google的Colab网站去安装 #Chilloutmix 在上面生成各种ai美图🫵
1⃣️首先你需要注册谷歌的Colab平台
colab.research.google.com
2⃣️注册好之后点击下面的链接并使用colab打开
drive.google.com/file/d/1dZxzC_…
Read 5 tweets
Turns out #stablediffusion #controlnet openpose model works with a posable figurine. This is going to make getting the right pose super easy. Next step is hooking it up to a webcam!
Another example #stablediffusion #ControlNet
Read 3 tweets

Related hashtags

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!