I think that replacing a content creator with AI will be possible shortly. Having worked for years with Unreal Engine/3D and deepfakes, this thread is just me spitballing ideas on how this could be made possible today:
While tools like synthesia.io exist, they're very fake and simply wouldn't fly with the audience. Your digital avatar using a tool like this would lack authenticity.
The best way to make an authentic avatar is to use deepfakes (trained on the content creator's own face/video): github.com/iperov/DeepFac…
You could also do voice reconstruction and Descript surprisingly has a pretty good tool for Voiceovers that we use often:
You could also do voice reconstruction and Descript surprisingly has a pretty good tool for Voiceovers that we use often:
The problem with deepfakes is that you STILL need another person to mimic your personality and look approximately like you. A stunt so to say. Even if you do it near-perfectly, the trained eye will still be able to tell the difference unless the base actor is your twin.
For example we deepfaked Ranbir Kapoor but you can still tell the difference:
Voice is the easier problem:
In my opinion, in 2 years nobody can trust phone call recordings and they won't be admissible in courts either. This is a radical change that Indian courts especially aren't ready for.
In my opinion, in 2 years nobody can trust phone call recordings and they won't be admissible in courts either. This is a radical change that Indian courts especially aren't ready for.
Another approach (free spitballing here) is to create a 3D avatar of the creator and use a library of pose animations. Someone will have to still input what pose the creator will use at what timeframe but that's an easier job than shooting a video. Unreal Metahuman is great.
I've done something similar in Unreal many years ago (way before Metahuman): (watch from 15mins)
This one is much lower fidelity than Metahuman.
This one is much lower fidelity than Metahuman.
There's another crazy combination I saw a while ago: Unreal Metahuman + deepfake.
This one is realtime but anyone that's made a deepfake knows that 1024 res takes lots of TIME to train. Impressive for realtime. My 3070Ti took 48h for the Ranbir one.
This one is realtime but anyone that's made a deepfake knows that 1024 res takes lots of TIME to train. Impressive for realtime. My 3070Ti took 48h for the Ranbir one.
A further combination could be Metahuman + deepfake + RTX + $2k Rokoko mocap suit for real animations: rokoko.com
Now you can use any actor: Easy to outsource your job to your cousin for cheap :P
Now you can use any actor: Easy to outsource your job to your cousin for cheap :P
You can do Mocap for nearly free with deepmotion.com but I find their AI and output to be clunky. Audience can tell.
Eventually some combination of trained data from creator mocap + Lyrebird type voice reconstruction + Metahuman + deepfake + GPT-3/GPT-4 (feed transcripts of creator's content so you can generate scripts given prompt) == Near total exit from your job
IMO creator-AI replacement is ready for shorts/reels (someone just needs to run an experiment) because you don't have too long to display personality.
In 2014 I freelanced for a guy from MIT who was making a "digital will": people close to death were supposed to write lots of content, and the company would create an AI-chatbot for their children to converse with post death. I think this will be real one day.
We've seen this before, in Superman. Superman's father Jor-El packs an AI-version of himself that can converse/help his son posthumously. It is mentioned in the comics that this version of the AI does not "feel" like a human being, it's just a mimic like chatbots are today. 
Interesting times.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
