1/ Filmmaker's guide to GEN-2

Generate videos with nothing but words
Access to GEN-2 will start rolling out this week @runwayml

I put together this guide to help with your experiments for when you play with it
#gen2 @runwayml 2/ Filmmaker's guide to GEN-2

It gets interesting when you add movement

It works best with simple movements btw, and struggles with more complex or intricate movements

Some of my early experiments with @runwayml GEN-1

It’s still early days of AI video - this tech will only get better. A whole new generation of filmmakers is gonna be able to make whatever they want on zero budget

See below for my process @runwayml Early access to GEN-1 is available via waitlist here: research.runwayml.com/gen1

I'll be sharing more tutorials once access opens up to more people

Microsoft just announced its new Bing with @OpenAI integration - using a model more powerful than ChatGPT

I’m on campus right now with access. Tweet reply what you’d like to test and I’ll try your query - Access is currently available via waitlist
I’m told it will roll out to millions of people within a few weeks

bing.com/new

Using AI for design inspiration

We used Stable Diffusion Depth to Image to get the consistency - collab with @justLV

See below for our process

#ArtificialIntelligence #stablediffusion #interiordesign @justLV Any image will work for this - you could use pictures of your real furniture too

But we thought it'd be fun to use dollhouse miniatures since they'd be easier to manipulate

NeRF update: Dollyzoom is now possible using @LumaLabsAI
I shot this on my phone. NeRF is gonna empower so many people to get cinematic level shots
Tutorial below -

-
#NeRF #NeuralRadianceFields #artificialintelligence #LumaAI 2/ First, stand still. Have a friend take a quick video of you
Capture the area in front of you and behind you

Testing out Stable Diffusion inpainting on a video
- by @justLV
See below for his process
#inpainting #stablediffusion #artificialintelligence @justLV

https://twitter.com/justLV/status/1587510992786714624

1/ Now you can create "drone" shots from your phone footage, thanks to NeRF
Collab with @justLV / software @nerfstudioteam / see below for our process
#NeRF #neuralrendering #artificialintelligence #instantNeRF 2/ Here is the raw footage input (sped up 5x here) which we fed into NeRF Studio

1/ Using AI to generate fashion

After a bunch of experimentation I finally got DALL-E to work for video by combining it with a few other AI tools

See below for my workflow -

#dalle2 #dalle #AIart #ArtificialIntelligence #digitalfashion #virtualfashion 2/ First I used DALL-E to generate outfits. I did this by erasing parts of my existing outfit and inpainting over it

Btw when I erased the entire outfit, the results didn't look as good. By keeping parts of the original, DALL-E was able to better match color and lighting

1/ Using AI to imagine alternate worlds🪞
collab w/ 🍃@MichaelCarychao 🦾Sagans

#discodiffusion #aiart #artificialintelligence @MichaelCarychao 2/ Here's the raw footage. I started off by creating a simple AR filter of two empty white rectangles using Adobe Aero. Shot by @AustinGumban

1/ I used DALL-E to imagine beyond this painting @OpenAI Inspired by @literallydenis #dalle #dalle2 #artificialintelligence #art @OpenAI @literallydenis 2/ Something that AI headlines don't always capture is that as a human, you actually have a lot of artistic input in what the AI paints. The AI didn't draw all this automatically - I prompted it to draw certain elements

1/ Using AI to smooth my footage 😌 with @jperldev. We’re using DAIN, an AI technique that smoothly interpolates frames. #stopmotion #artificialintelligence 2/ For every frame you see, it generates 8 frames in between with incredibly smoothness and accuracy. Its main use case is to create artificial (and very convincing) slow motion on clips, but I thought it'd be interesting to apply it to stop motion to create "impossible" movement

1/ We used NeRF to get this shot. This entire thing was shot with a phone 🤳 No drones were used @jperldev #photogrammetry #instantNeRF #neuralrendering #artificialintelligence 2/ Here's how - walked around me in circles a few times, pointing his phone at me. We fed the video into Instant-NGP which created a NeRF out of the footage.

I used @OpenAI #dalle2 to create the first ever AI-generated magazine cover for @Cosmopolitan!! The prompt I used is at the end of the video #dalle 1/ For something like this, there was a TON of human involvement and decision-making. While each attempt takes only 20 seconds to generate, it took hundreds of attempts.

Hours and hours of prompt generating and refining before getting the perfect image