Thread by @robcupisz

Here it is! Just released:

Enemies

A short film we’ve been working on for a while now on the Demo Team, here at Unity.

Some notes on rendering in the 🧵

#unity #gamedev

With ultra settings, it runs at 4k 30fps (average of 40fps) on an i7 cpu and rtx 3090.

We've prepared other quality settings to run it on lower hw targets, including consoles. 4k is upscaled - with DLSS on supported hw.

Together with Unity R&D we’ve been improving the engine and HDRP during the production. All improvements have either already been released, or are in PRs that are about to land in the coming weeks.

In other words the project runs on a vanilla editor and HDRP.

Lighting.

Sun, and mostly area lights later on.

For indirect: Adaptive Probe Volumes (screenshots), SSGI to add contribution from smaller shapes, and AO (raytraced and SS).

No lightmaps - would've been a nightmare on this kind of geometry.

Apart from raytracing AO, Sun shadows and reflections are raytraced as well.

SSR couldn’t handle all these reflective bits swirling around in the background, disoccluding, etc.

Raytraced reflection denoising does a surprisingly good job in these conditions.

For area light shadows we needed both extra large penumbrae, and precise and sharp contacts. Getting these wrong is one of the worst artifacts you can introduce on a face.

I'm quite happy with our improvements here. (PR about to land in HDRP)

Hair is a big one.

It's a real-time strand-based simulation, which tries to maintain the initial groom and clumping, while making hair as dynamic as possible.

Mesh collisions via SDF with a real-time mesh-to-SDF renderer.

Shading in texture space on a limited number of points along a strand, then interpolated during actual strand rendering.

Strand LODding for simulation, shadowing and final rendering. Segment culling.

We'll release the hair solution as a package in Q2, stay tuned.

Facial animation is still a 4D capture. We've improved our workflow and quality, but 4D would still be impractical for any larger production.

The future of our productions is Ziva, where we only train the system with 4D, but final performance is captured even with and HMC.

Ziva allows for full body performance while recording facial performance with HMC - in our old workflow matching 4D with the body is hard, and the actor performs differently with their head fixed.

It is also effectively an efficient compression, allows for interactivity, etc.

We will migrate Enemies from 4D to Ziva in the coming weeks.

Skin tension.

Calculates edge length relative to the rest pose, to identify stretching and contraction, which allows us to drive wrinkle and blood flow maps.

In The Heretic we needed a facial rig just for that. This is much more convenient.

Skin attachments.

Eyebrows, eyelashes, tearline, peach fuzz/vellus hair(!), etc. need to follow skin deformation.

We've developed a system for handling that through jobs for The Heretic, now moved to gpu.

Some attachments still resolved on cpu, like the teeth occlusion polygon

Improvements to skin shading, hair shading (Marschner), eye caustics, tearline (the wet interface between the eyelids and the eye).

Most of these already shipping in HDRP. Remaining digital human features mentioned earlier shipping in Q2 as an updated Digital Human package.

Lookdev scene for comparing against light stage photos

Random screenshot intermission

Smoke and flame.

Transparencies don't play nice with DoF in a rasterization renderer, and here we have close-ups of in-focus smoke/flame over out-of-focus face, and vice versa.

To get around that, we're rendering them into an off-screen buffer, but still depth-test against

the main depth buffer, to get occlusion from the bishop and the hand.

Then a blur with a simplified DoF kernel with a fixed CoC size, and composite with the main colour buffer - after the regular DoF (and moblur) pass.

Easy with HDRP's custom pass and custom post-process.

Team intro!

We're a small team, and kinda like it that way :) On this production these were our roles:

@vesseff as usual, as a writer and director, also on cinematography and lighting.

@tlaedre as a tech director.

Yours truly👋, Mikko Alaluusua and @codeverses on programming.

@vesseff @tlaedre @codeverses .
@Paco_q2 art director and 3D artist

@KNechevski anim director

@ChrisKardach tech anim

@adrianlazar3d tech art

@dominas64 more programming

Petter Flood - more 3D art

Jörn Großhans - additional art consulting

Aleksander Karshikoff - producer

Silvia Rasheva - manager

Special thanks to:

The entire Unity R&D team, for actually making the tools we use!

@matt_ostertag (@Tuatara_VFX) for the smoke and flame vfx art :)

@8Infinite8 for his 4D scanning studio and expertise.

Everyone else who contributed! Full credits:
unity.com/demos/enemies#…