Thread by @bilawalsidhu

ControlNet experiment where I'm toggling through different styles of contemporary Indian décor, while keeping a consistent scene layout.

Loving how ControlNet is putting the artists back in control of AI image generation process.

🧵Thread

#ControlNet #StableDiffusion #EbSynth

For the input sequence I use a short animation made from a photogrammetry 3d scan I did a few years back of my parents living room in India.

- Top: Output generated with ControlNet + EbSynth

- Bottom: Input video sequence from my 3D scan

I used the ControlNet depth method for this experiment.

- Left: MiDaS depth map generated from a screen recording of my 3d scan

- Right: Awesome results reskinning the room! Pretty majestic if I say so myself

Next time I want to use the synthetic depth from my 3d scan

Different ControlNet methods have their pros/cons:

- Left: Canny does well to keep the wall paintings consistent (thanks to the contrasty edges), but fumbles with the table

- Right: Depth turns walls 🖼 into a flickering mess (since no depth), but crushes it with the furniture

⚔ The best results will come from combining the strengths of various ControlNet methods into one composite, which is my next test.

📝Also, ControlNet deep dive post dropping later this week!

If you enjoyed this thread:

1. Follow me @bilawalsidhu
2. RT the tweet below… twitter.com/i/web/status/1…

You asked, so here is a 7 minute YouTube video deconstructing my workflow. I hope it inspires you to capture spaces you care about and reskin them with AI!