Made this video with iPhone photos I took of my friend Stephanie that I used as keyframes in @LumaLabsAI! With the camera controls I can gen transitions between shots. I also built a custom web app in Next.js to help me speedramp and edit all the clips! Breakdown 🧵(1/18) Basically if you have some photos, throw in a start and end frame, and start your prompt with the camera move, and then stuff like “smooth camera, steadicam” I find minimal prompts work best. And don’t enhance the prompt (that tends to add hand held). (2/18)

Testing how LivePortrait works lip syncing 24fps lyrics on top of slow motion footage. Was curious to see if it might help with music videos. Quick explanation below! 🧵 Started with a clip from an Eminem song and passed it through Adobe Podcast to get the acapella. Passed that through @hedra_labs with a Midjourney portrait for the face animation. Used that as input into LivePortrait using ComfyUI and a slowmo clip from Die Hard.

Made this video (🎶) with a Midjourney v6 image! Started by upscaling/refining with @Magnific_AI, pulled a Marigold Depth Map from that in ComfyUI, then used as a displacement map in Blender where I animated this camera pass with some relighting and narrow depth of field.🧵1/12 Here's the base image and the before/after in @Magnific_AI. Even though MJv6 has an upscaler, Magnific gave me better eyelid and skin details for this case. (Fun fact, this image was from a v4 prompt from summer last year, when MJ had just released a new beta upscaler.) 2/12

Testing LCM LORAs in an AnimateDiff & multi-controlnet workflow in ComfyUI. I was able to process this entire Black Pink music video as a single .mp4 input. The LCM lets me render at 6 steps (vs 20+) on my 4090 and uses up only 10.5 GB of VRAM. Here's a breakdown 🧵[1/11] Entire thing took 81 minutes to render 2,467 frames, so about 2 seconds per frame. This isn't including the time to extract the img sequence from video and gen the ControlNet maps. Used Zoe Depth and Canny ControlNets in SD 1.5 at 910 x 512. [2/11]

Timelapse of using #photoshop’s new generative fill feature to connect two images and build a scene around them using blank prompts. Was inspired by @MatthieuGB’s post doing something similar! Notice how I’m not adding any descriptions, but letting gen fill present options for… twitter.com/i/web/status/1… Here’s the final image! 2/4

Testing Multi-ControlNet on a scene with extended dialog, a range of facial expressions, and head turning. No EBSynth used. #AImusic from aiva.ai. Breakdown🧵1/15 #aicinema #controlnet #stablediffusion #aiia #aiartcommunity #aiva #machinelearning #deeplearning #ai Overall flow: pre-process video > img seq > play with prompts > initial controlnet settings > control net batch render > upscale > clean up in post > key out background > deflicker > video post-production 2/15

Really impressed by the acting you can generate with @elevenlabsio #texttospeech! These are #AIvoices generated from text—dozens of "takes" stitched together. Breakdown thread: 1/8 #syntheticvoices #ainarration #autonarrated #aicinema #aiartcommunity #aiia #ai #MachineLearning I started by having #chatGPT write a few rough drafts of a scene involving a panicked character calling her friend for help from a spaceship. I was going for something that would involve heightened emotions but not be too serious. 2/8

I wonder what the future of UX design (maybe apps in general) may be like if AI really allows us to customize our experience. Not to mention blend programs together through a 3rd party/custom UI if an AI can understand onscreen what's being displayed by the app's GUI. 1/5 Combined with no code platforms of the future and advanced templates, you could probably do weird stuff like Premiere Pro x Unreal Engine x a fighting game template x an anime you like and custom gameify your interface. 2/5

Lately I've been thinking about how much of "reality" is a negotiation with useful illusions and the material world. I think it's safe to say that a portion of how we view things is through shortcuts and narratives. 1/11 To what degree we engage in fictions probably differs from person to person. Some believe the entire thing is a fiction passed on to us from evolution to navigate the food chain. Others think they concretely engage in reality the whole time. 2/11

Used #stablediffusion2 #depth2img model to render a more photoreal layer ontop of a walking animation I made in #UnrealEngine5 with #realtime clothing and hair on a #daz model. Breakdown thread 1/6 @UnrealEngine @daz3d #aiart #MachineLearning #aiartcommunity #aiartprocess #aiia Reused this animation I made from a post from a few months ago. Rendered in #UE5 @UnrealEngine using a @daz3d model with #realtime clothing in the @triMirror plugin. Walk was from the #daz store. Hair was from the Epic Marketplace. 2/6 #aiartprocess

A thought on resistance to change. I recently had a convo with a friend of mine who went thru a serious breakup that left her rattled. She talked about how hard it was to let go of the future she had envisioned for herself; that she felt so sure was going to come. 1/7 I feel like part of the resistance to change isn’t just rooted in the past and present, but also your perception of how you thought the world was going to look like and your place in it. Expectations are set and not met. 2/7

I’m so fascinated by how much of understanding a concept can sometimes just be a language issue. Being able to ask #chatgpt to summarize, expand, rephrase and format explanations in different ways is so refreshing. 1/7 Like here’s #ChatGPT explaining how to cook a steak in pseudo code format. 2/7

Started with a landscape in #midjourney. Used a depth map in @sidefx #houdini to create a scene mesh. Imported to #UnrealEngine5 @UnrealEngine to add grass and animate. Added music. Breakdown thread 1/8 #aiartprocess #MachineLearning #deeplearning #aiartcommunity #genart #aiia I could have created a similar scene in just Unreal Engine and Quixel but I wanted to see what I could do with this landscape image I generated in #midjouney 2/8 #aiartprocess

Created with #deforum #stablediffusion from a walking animation I made in #UnrealEngine5 with #realtime clothing and hair on a #daz model. Combined the passes in Premiere. Breakdown thread 1/8 @UnrealEngine @daz3d #aiart @deforum_art #MachineLearning #aiartcommunity #aiartprocess I started with this animation I rendered in #UE5 @UnrealEngine using a @daz3d model with #realtime clothing using the @triMirror plugin. Walk was from the #daz store. Hair was from the Epic Marketplace. Went for a floaty, underwater vibe with the hair settings. 2/8 #aiartprocess

One thing I learned from working in fashion—what’s considered “good” changes from region to region, country to country. From working in video and photo, “good” changed depending on if the person worked mostly with photographers or filmmakers. 1/10 Then there’s differences in genre, branding strategy, or individual ppl. There’s overlap of course, what we might consider the transcendant or timeless qualities of certain aesthetics, but the differences in taste always had relevant impact on budgets, rates, and networking. 2/10

Took a face made in #stablediffusion driven by a video of a #metahuman in #UnrealEngine5 and animated it using Thin-Plate Spline Motion Model & GFPGAN for face fix/upscale. Breakdown follows:1/9 #aiart #ai #aiArtist #MachineLearning #deeplearning #aiartcommunity #aivideo #aifilm First, here's the original video in #UE5. The base model was actually from @daz3d #daz and I used Unreal's #meshtometahuman tool to make a #metahuman version. 2/9 #aiartprocess

If you have a style or consistent aesthetic you’re going for with AI art, it might be a good idea to figure out how to get to that look with different tools, starting points, and process pathways. 1/8 We already see how fast these tools are changing. And as we iterate, as the tool space evolves, we can’t know how backwards compatible changes will be. Maybe we have more control with local SD, but maybe 2/8

Using X/Y Plot in a local #stablediffusion WebUI to create contact sheets exploring the latent space for my previous post. As a photo/video person I'm trying to bring the #aiartprocess closer to the workflows I use with clients.1/7 #aiphotography #aifashion #aiart #aiartcommunity I imagine if you're coming to AI art from a different set of creative fields and client expectations, you'd approach the latent space from a different collection of frameworks and processes. Very curious how different a UI/UX catering to painters could look. 2/7 #aiartprocess