Thread by @tristwolff

AI text-to-video is now available as an open-source model 🚀

It's still in its early stages & nowhere near as mature as #Midjourney, but it's definitely worth exploring.

Here's how it works & how you can experiment with it.

🧵
#Modelscope #text2video #AIArt #AiArtCommunity

The model is simply called “text-to-video synthesis”. A brief summary:

- 1.7 billion parameters 🔥
- training data includes public datasets like LAION5B (5,85 billion image-text pairs), ImageNet (14 million images) & WebVid (10 million video-caption pairs) 🌎
- open source 💪

Text-to-video synthesis consists of three sub-networks that work together to produce short MP4 video clips:

- a text feature extraction,
- a text feature-to-video diffusion model,
- and a video-to-video diffusion model.

🚀🚀🚀

If you just want to try it out briefly, you can go to @huggingface and use their publicly available free demo: huggingface.co/spaces/damo-vi…

If you want to dive deeper into experimenting with the model, set up and run your own instance of the model at @huggingface.

1. Get an account: huggingface.co
2. Navigate here: huggingface.co/spaces/damo-vi… and "Duplicate Space"

3. Give your new Space a name and set it to “Private”. After that, once again hit “Duplicate Space”. Your new Space will then be created automatically.

4. Go to the “Settings” of your Space and choose a sufficiently powerful setup. I used T4 medium for $0.90/hour to save time when generating, but T4 small might also work?

5. Set inactivity timeout and confirm🔥

https://twitter.com/tristwolff/status/1637640996191125505?s=20

Original repo: modelscope.cn/models/damo/te…

More about AI & creativity:
medium.com/@tristwolff