Thread by @StylizedStation

Ever wondered how clouds work in video games?

They may be an afterthought for gamers, but creating clouds is one of the most exciting and complex topics in game art.

We have a lot fo talk about, so lets get right into it.

Another massive 🧵

Back in the day, there weren’t really clouds in video games.

Sure, you could see clouds, but these were just handpainted photoshopped images projected onto something called a skybox - a method of creating backgrounds to make a video game level appear larger than it really is.

When a skybox is used, the level is enclosed in a cuboid.

The distant mountains, distant buildings, and other unreachable objects like clouds are projected onto the cube's faces (using a technique called cube mapping), thus creating the illusion of distant three-dimensional surroundings.

And this worked well for a while, but As technology improved, developers and artists started using some really interesting shader techniques to add movement to their skybox clouds.

One of these techniques is called flow mapping, which is a method that allows developers to create movement in their skybox, by storing directional information into a texture, called a flow map

The Stormpoint map in Apex legends is a great example of excellent use of flow maps.

This storm looks alive, but this skybox is just a simple matte painting with a flowmap and distortion shaders applied to it to give the illusion of cloud movement. There's a lot more going on in this, but that's the basic idea.

So if most clouds are actually 2d, how do the clouds in genshin impact create these shadows on the ground from clouds passing overhead?

Well, they dont - and you’ve just fallen for one of my favorite tricks in-game art.

By projecting a massive dark noise texture over the entire map artists can fake cloud shadows by slowly scrolling that texture over the landscape to mimic clouds passing overhead.

It's simple, effective, and much cheaper than using dynamic shadows created by clouds in a skybox.

I can prove this is happening by looking up at the sky when a cloud shadow passes over my character, the sun should be somewhat obscured by a cloud if I’m seeing a shadow, but we see nothing but a clear sky around the sun - it's just a texture.

So, animated skyboxes are very powerful, but what about the clouds in Sea of Thieves that you can actually see in 3d? How is that done?

By creating clouds with 3d geometry and running it through filters, SoT are able to create clouds of any shape and control everything from the lighting, shape, and subsurface scattering of the clouds, making the clouds look full and fluffy.

Creating filters for 3d geometry in real-time is very resource-intensive, so to avoid this, the cloud geometry and filters are calculated off-screen in a buffer. A camera-facing quad samples the buffer target and composites the clouds with the scene to create one cohesive sky.

The final result is surprisingly performant, painterly-looking clouds that can take any shape or form, like a skull, or even another pirate ship.

Cloud technology is being pushed even further every day, and lots of modern games like Horizon Zero Dawn and Ghosts of Tsushima, and even RdR2 -

- are moving past simple skyboxes and skydomes and feature real-time volumetric clouds - a cloud system that uses a three-dimensional volume texture that is ray-marched to represent cloud layers in real-time.

The creators of the Horizon series, guerilla games, even went as far as to create their own in-house volumetric cloud and weather system called Nubis, that is capable of creating realistic cloud systems and massive swirling storms.

(Yes, this was recorded in game. Crazy)

For the first time ever, with volumetric clouds players are able to climb a mountain above the clouds in open-world games or soar through the clouds in flight simulators - forever changing the way we experience games.

That's right, the introduction of volumetric cloud technology in games was a game-changer, taking the in-game environment's realism to an entirely new level.

And it's not just for the aesthetics, but volumetric clouds also interact dynamically with light sources, which can result in beautiful light scattering effects, like god rays breaking through a cloudy sky, or the majestic setting sun getting partially obscured by clouds.

Let's take Red Dead Redemption 2 for instance. It uses a remarkable combination of volumetric fog and volumetric clouds to create stunning weather systems that dynamically change over time.

The dense fogs that roll over the game's mountains at dawn, the fluffy clouds during a sunny day, and even the menacing dark clouds during a storm, all of these are achieved using volumetric rendering.

The volumetric clouds in RDR2 are made up of multiple layers of 3D noise textures. The different layers allow for a variety of cloud types to be represented - lower stratus clouds, towering cumulus clouds, and thin wispy cirrus clouds high in the sky.

The 3D noise texture is sampled during the rendering process, and by adjusting the threshold value, artists can control the cloud density at any given point. (See gif above)

But wait, there's more. These clouds in RDR2 are not just a static entity, they actually evolve over time. By slightly shifting the 3D noise texture used for the clouds over time, the game creates the illusion of cloud evolution, making the sky look incredibly alive and dynamic.

Another example would be Flight Simulator 2020, developed by Asobo Studio, which has set a new bar in terms of realistic cloud rendering.

As you might expect from a flight simulator game, clouds play an essential role in creating an immersive flight experience.

The game uses real-world weather data to render the sky, and it creates breathtakingly beautiful and accurate representations of cloud formations.

The use of volumetric rendering allows the game to create an incredibly varied and dynamic range of weather conditions, from clear skies and fluffy cumulus clouds to dense fog and violent thunderstorms.

And let's not forget about how the developers use the lighting system in combination with volumetric clouds.

As the light from the sun interacts with the cloud particles, it creates stunning lighting effects such as crepuscular rays, also known as god rays, diffusing through the clouds, and beautiful hues during the sunrise and sunset.

There is so much more to talk about, but that will have to be for another time.

If you want to learn more about game art, feel free to check out my courses:
courses.stylizedstation.com

Thanks for reading!