Hey #VirtualProduction enthusiasts, time for a #UE4 tweetorial.
Surely you have seen the new Live Link Face App. You need a character with a set of blendshapes that match with the 51 produced by ARKit.
What happens when you don't have as many blendshapes, or with different poses.
Surely you have seen the new Live Link Face App. You need a character with a set of blendshapes that match with the 51 produced by ARKit.
What happens when you don't have as many blendshapes, or with different poses.
2- Into the Permanent Free Collection in Unreal Marketplace you can find the Stylized Character Kit: Casual 01. A great asset but with a limitation. Instead of 51 morph targets only has 7. And not based on specific muscle movements but on emotion poses
3-You can read the documentation to know how to configure and use Live Link Face App.
But in THIS CASE the 1rt problem is how to recognize what "Blend Shape" should move my few morph targets?
There is a plugin: Live Link Curve DebugUI, you can use it to view the curves and values
But in THIS CASE the 1rt problem is how to recognize what "Blend Shape" should move my few morph targets?
There is a plugin: Live Link Curve DebugUI, you can use it to view the curves and values
4- Now check the grimace of the morph targets(MT) and test which curve is higher by mimicking the face in the Live Link Debugger.
In some cases, expression-based morph targets separate eyes, brows, and mouth. This is not the case, so focus on the curves of the mouth and the blink
In some cases, expression-based morph targets separate eyes, brows, and mouth. This is not the case, so focus on the curves of the mouth and the blink
5- If your character doesn't use exactly the same names as the ARKit blend shapes it will not work.
But we can create an LiveLinkRemap and overwrite a function called GetRemappedCurveName to translate the names of the blend shapes to those used by your character.
But we can create an LiveLinkRemap and overwrite a function called GetRemappedCurveName to translate the names of the blend shapes to those used by your character.
6- Since this mesh only blinks both eyes at the same time, I connected both EyeBlink to Wink.
This is how the LiveLinkRemap asset would look using this mesh.
This is how the LiveLinkRemap asset would look using this mesh.
7- Now we just retarget the new Live Link Remap asset to the Live Link Pose and that's it. Our character has expressions.
Yes, I know, this has limited expressions, but achieving it with 7 MT is a challenge. Try to bring your mesh closer to the existing 51.
Yes, I know, this has limited expressions, but achieving it with 7 MT is a challenge. Try to bring your mesh closer to the existing 51.
If you copypaste from FaceAR Sample Project the nodes that allow head rotation you noticed that it doesn't work.
You will need to remap 3 curves associated with the head from ARKit but our character no longer has Morph Targets available (there were only 7).
Solution in next tweet
You will need to remap 3 curves associated with the head from ARKit but our character no longer has Morph Targets available (there were only 7).
Solution in next tweet
8-Just open any animation sequence associated with the skeletal mesh and add 3 new curves to it, use the name of those curves to add them to the ReMap asset and override the names of the head-moving curves in ARKit.
Then replace the curve name in the Anim blueprint, and Done!
Then replace the curve name in the Anim blueprint, and Done!
For those interested in trying the same free asset, I also made some adjustments to the nodes that control head and neck so that the rotation improves. This is the screenshot.
I hope it will help you in your future projects!
I hope it will help you in your future projects!
