Generative models are finally good enough to make talking characters with AI.
Combining a voice with a face has unlocked countless use cases - from sales reps for enterprises to consumer animation.
Our market map + insights 👇
@a16z I tested 20+ products to make AI characters.
In case you haven't tried them yourself, I've included some of the outputs below (from @hedra_labs, @getcaptionsapp, and @arcads_ai)
As someone who has been trialing these tools for years, I'm blown away by what we can now create.
@a16z @hedra_labs @getcaptionsapp @arcads_ai Why is this such a hard problem?
Talking face models aren't just generating an image or video. They have to:
1/ understand phoneme-to-viseme mapping (so lips match the sound) 2/ maintain character consistency between frames 3/ generate believable expressions + body movement
I think I've uncovered the next Turing test for AI video: writing.
I've tried this prompt dozens of times on every model: "man writes 'hi' in chalk on blackboard"
None can do it. Veo 2 (below) gets the closest. It's actually frustrating to watch!
The Sora attempts are quite funny because the characters often look like they're having some kind of existential crisis about their inability to write.
These guys are both going through something...
Hailuo (Minimax) often gets the correct letters and adds some flair to the chalkboard.
But the letters just appear, it still can't "write."