1/ @lishali88 and @spring_stream joined us to talk about building Rosebud.ai.
Rosebud.ai's @tokkingheads turns portraits into animated avatars that read text you provide. It's fun to play around with!
Here are some challenges they faced building it:
Rosebud.ai's @tokkingheads turns portraits into animated avatars that read text you provide. It's fun to play around with!
Here are some challenges they faced building it:
2/ A scalable model training platform was key to experimenting quickly enough to build talkingheads.rosebud.ai.
They built theirs on Kubernetes and take advantage of spot instances to keep costs down.
More on their training infra here: blog.rosebud.ai/cost-efficient…
They built theirs on Kubernetes and take advantage of spot instances to keep costs down.
More on their training infra here: blog.rosebud.ai/cost-efficient…
3/ Model quality is key to their product, so Rosebud prioritizes that over performance.
They're looking into model compression techniques to make big models faster (and more cost effective).
They're looking into model compression techniques to make big models faster (and more cost effective).
4/ @tokkingheads is built on GANs. Rosebud is excited to try VAEs and autoregressive models as well, but they're not convinced the quality is quite there.
GANs are also sample-efficient to train and relatively cheap to run inference on.
GANs are also sample-efficient to train and relatively cheap to run inference on.
5/ The Rosebud team thinks that the interplay between users and models will be a key to success for their product.
* How can you enable users to intuitively maneuver in the GANs latent space?
* What is the right UX for users to guide the model's output?
* How can you enable users to intuitively maneuver in the GANs latent space?
* What is the right UX for users to guide the model's output?
6/ Next up for Rosebud - automated video creation composing synthetic media assets (virtual beings, backgrounds)!
Users can scale up and make frictionless high-quality digital media storytelling.
(They're hiring an intern to help)
Users can scale up and make frictionless high-quality digital media storytelling.
(They're hiring an intern to help)
https://twitter.com/lishali88/status/1332487889754714115
7/ There’s been press recently about misuse of generative AI (e.g., DeepFakes).
Rosebud's tool isn't meant to be photo-realistic. Its users want to do creative things with synthetic art.
Misuse is a concern, but they think marketing to the right customers will help mitigate it.
Rosebud's tool isn't meant to be photo-realistic. Its users want to do creative things with synthetic art.
Misuse is a concern, but they think marketing to the right customers will help mitigate it.
8/ Final takeaways:
- Early on, spend minimal time building models. Instead, look for ways to get the product to users quickly.
- Build your technical stack for quick iteration
- Invest in parts of your stack that will help for many use cases (since you'll pivot!)
- Early on, spend minimal time building models. Instead, look for ways to get the product to users quickly.
- Build your technical stack for quick iteration
- Invest in parts of your stack that will help for many use cases (since you'll pivot!)
• • •
Missing some Tweet in this thread? You can try to
force a refresh
