Trist Profile picture
Mar 21 11 tweets 5 min read
AI text-to-video is now available as an open-source model 🚀

It's still in its early stages & nowhere near as mature as #Midjourney, but it's definitely worth exploring.

Here's how it works & how you can experiment with it.

🧵
#Modelscope #text2video #AIArt #AiArtCommunity
The model is simply called “text-to-video synthesis”. A brief summary:

- 1.7 billion parameters 🔥
- training data includes public datasets like LAION5B (5,85 billion image-text pairs), ImageNet (14 million images) & WebVid (10 million video-caption pairs) 🌎
- open source 💪
Text-to-video synthesis consists of three sub-networks that work together to produce short MP4 video clips:

- a text feature extraction,
- a text feature-to-video diffusion model,
- and a video-to-video diffusion model.

🚀🚀🚀
If you just want to try it out briefly, you can go to @huggingface and use their publicly available free demo: huggingface.co/spaces/damo-vi…
If you want to dive deeper into experimenting with the model, set up and run your own instance of the model at @huggingface.

1. Get an account: huggingface.co
2. Navigate here: huggingface.co/spaces/damo-vi… and "Duplicate Space"
3. Give your new Space a name and set it to “Private”. After that, once again hit “Duplicate Space”. Your new Space will then be created automatically.
4. Go to the “Settings” of your Space and choose a sufficiently powerful setup. I used T4 medium for $0.90/hour to save time when generating, but T4 small might also work?
5. Set inactivity timeout and confirm🔥
More about AI & creativity:
medium.com/@tristwolff

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Trist

Trist Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @tristwolff

Mar 21
New paper on #GPT4 & "Labor Market Impact" is out.

It's an early evaluation, but creatives should be paying attention to this ... 👀

🧵 1/5 Image
All the jobs that show 0 exposure to being automated by LLMs are defined by

- blue-collar, manual labor
- outdoor or non-office
- lower educational requirements
- use of specialized tools or equipment

As we'll see: jobs /w higher educational requirements ➡️ high exposure 🚨 2/5 Image
☝️In fact, if your job is defined by

- creative or intellectual labor
- office or indoor work
- high educational requirements

it's much more exposed to being replaced by #GPT (alpha), or #GPT-based apps (beta & theta):
3/5 Image
Read 8 tweets
Mar 16
Some #Midjourney V5 findings 🔥
Finally! Full-blown cinematic aspect ratios!

2.4:1 widescreen cinema standard ✅
2.76:1 ultra panavision 70 ✅

you can go up to 14:1 (!) but: letterboxing seems to set in roughly at ultra widescreen (3.6:1)

#midjourney5 🧵
However, 3.6:1 (and higher ratios) seems to work better if you drop the cinematic prefixes (cinematic shot, film still, etc.) 🤷‍♂️🤔
Here it's just scene & style description. 3.6:1, no letterboxing.
despite the letterboxing, exploring 4:1 is fun ... 🍄🎉
#MidjourneyV5
Read 10 tweets
Mar 15
Built this in 1.5 hrs w/ #GPT4 & #Midjourney.
I only have basic JS skills.

The possibilities of 🤖 &🧠 co-creation are mind-boggling. 🎉

Graphics: 🤖 images &🧠 curation
Code: 🤖 HTML/CSS/JS &🧠 co-debugging
Music: 🤖 Samples &🧠 sequencing

Here's how: 🧵

@RedSquidProject
I generated the images for the game with #midjourney
Prompted for basic 1:1 images, using the tried and tested combination of "white background" and "--no background" to prep transparent PNGs
Then I used #GPT4 via #ChatGPT & gave it the basic story for the game: Squid-Spaceship having to collect little baby squid astronauts, etc...

Asked the AI to come up with HTML/JS code & CSS, then helped it to debug & adjust mechanics (smooth flight paths, etc..)

And done. ✅ 🤯
Read 4 tweets
Feb 10
With #midjourney V4 supporting aspect ratios up to "2:1", here's how you can use them effectively in your #AICinema storytelling. 🎥🔥

🧵👇
Besides being technical requirements for screen sizes, ratios can also be used to create an aesthetic backdrop for your stories. 🎬

E.g. 4:3 (or "1.33:1" in cinema slang) is the "classic ratio" of TV monitors & perfect for scenes that are meant to evoke an "analog TV" vibe
"11:8" ("1.375:1" in cinema slang) also called "Academy ratio" is a standard introduced in 1932.

It's outdated, but still in use as an aesthetic tool. E.g. Wes Anderson used it to differentiate timelines in "Grand Budapest Hotel":
Read 11 tweets
Dec 2, 2022
#AIArt is about to revolutionize #screenwriting! 🤯 As AI tools become more intuitive and easy to use, some think production companies might prioritize profit over storytelling & we'll be binging stunningly beautiful visuals paired with incredibly boring plots.. 🤔

A thread 🧵
1) We could be in for a wild ride here:
As with all technology, there is a risk of it being used in a way that strictly prioritizes profit. AI material might be used to create visually appealing stories that lack depth and originality
2) Yet AI will likely revolutionize screenwriting in a similar way it's currently enhancing visual expression and #AIArt!
Training models with dramaturgical data and narrative finesse is key. And it's already underway 💪 E.g. Dramatron, check @korymath's ImageImage
Read 9 tweets
Dec 2, 2022
Here's a technique for movie concept creation I currently experiment with:

Using #screenwriting devices to prompt #GPT3 for basic story ideas & plots and then using those to generate #midjourney #aiart images depicting key scenes, characters & settings...
Not only is this a huge time saver for #writers when brainstorming or trying out different dramatic scenarios, but you also get images that further spark your imagination...
... and this inspiration goes both ways: images create new approaches to dramaturgical twists or character development, and storytelling devices literally create new perspectives on the story 🤯
Read 4 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(