Thread by @phyronnaz

I've been digging into @UnrealEngine 5's Lumen yesterday to make it work with my voxels.

Here's what I learned along the way, if anyone's curious

(if I say something wrong please correct me :) )

I'll use the following scene as an example: a sphere with a colored emissive material, a white plane and no light whatsoever

For reference, here's how it looks without Lumen

The first thing we can do is disable screen space traces using r.Lumen.ScreenProbeGather.ScreenTraces 0

As you can see, the GI looks a lot worse with only the voxel grid (?) contribution

To know the mesh color/emissive/normal without sampling the material all the time, Lumen uses a surface cache system.

You can see the values stored in that cache using Show -> Visualize -> Lumen Scene

That cache works by rendering the mesh under different angles to an atlas. Each angle is defined using a "card": a rectangle onto which the mesh is rendered.

You can view these using r.Lumen.Visualize.CardPlacement 1

Below: the -X and +Z cards

The atlases are stored in Lumen.SceneAlbedo/SceneDepth/SceneNormal etc

You can use RenderDoc to view these, or the vis Lumen.SceneAlbedo command in UE

The cards are updated when the camera moves, as their resolutions are based on the distance from it.

You can use r.LumenScene.RecaptureEveryFrame 1 to force a recapture of all the cards in every frame

In our scene, the plane is rendered once and the sphere 6 times as expected

To find the color at a position, all we have to do now is find the best cards representing that position, project it onto them & read the values.

Lumen also reads the Depth atlas to ignore cards that would be occluded (I think 😄 )

All this is done here: github.com/EpicGames/Unre…

As you can guess, cards won't work well with meshes with interiors as they'd self occlude when rendering to the cards.

This is why Epic recommends to split meshes as much as possible when working with Lumen: don't import a house mesh, import wall/floor/roof meshes

So now we know how Lumen finds the color of a hit. But how to find the hit itself?

As far as I can tell, Lumen uses distance fields to do that!

To visualize distance fields, you can do Show -> Visualize -> Mesh Distance Fields

In UE4, distance fields were stored in a single 3D buffer per mesh.

In UE5, Lumen needed a much higher DF resolution, so Epic switched to a sparse storage.

This is done with an indirection table, storing indices to 8x8x8 bricks, or 0xFFFFFFFF if empty.

Bricks are stored in DistanceFields.DistanceFieldBrickTexture - you can see parts the sphere shape below :)

DF sampling: github.com/EpicGames/Unre…

3 mips are stored in that format. The distances are stored in 8 bit, and are only stored (accurately) up to 4 voxels from the surface.

That's all I have for now :)

I have yet to look into how the actual GI is done, how reflections work, and to dig into all the changes Epic already made in ue5-main 😄

lmk if you'd like to see more engine breakdowns like that, where I talk about stuff I barely understand!

Correction: cards can also be placed _inside_ meshes! This means Lumen could support single meshes with interiors :)

https://twitter.com/knarkowicz/status/1401227651969323010