Thread by @delaneykingrox

A quick #gamedev question about texture arrays...

So texture arrays are brilliant for reducing draw calls and state changes by gathering up a bunch of textures in one go in the shader...

https://twitter.com/gomboc04/status/1505082156812541953

Texture coordinates work from 0,0 (left,bottom) to 1,1 (right, top).

Note this is not based on texture resolution. So the middle of a 256x256 and the middle of a 1024x1024 image is the same... it's always (0.5, 0.5).

When you go outside of the zero to one space, what happens? Well, the default behaviour is called "tiling" the texture repeats off into infinity and beyond in all directions.

If you set your texture to "clamp" any UV that goes outside the zero to one space is black.

So this allows us to do something very cool. If I superimpose two pictures ontop of each other by adding them, and have both set to clamp, then I offset one of textures one space to the right (I add 1.0 to the horizontal) then rather than superimpose, they both add to black.

This lays the two textures side by side. Essentially appending the two textures.

We call this a UDIM- because when this is done in film we tend to use the U (horizontal) dimension to describe the texture.

The first is UDIM 0 (the UVs go from 0-1), the second UDIM 1 (1-2)

This is the key design feature of Mari- it was designed by this awesome guy Jack Greasley so they could texture King Kong.

Each map gets a number based on the U dimension. So 0,1,2,3,4 is the horizontal tile which the texture sits.

So what is the benefit of texture arrays?

Well, firstly your polygons can tile over them. This effectively means you can have a whole mess of textures flowing along, say, a road left to right with no break.

Second cool thing? Each texture can be any resolution and it still works. You can tack on a 64x64 texture onto a 2048x2048 texture.

Third- it let's you draw all of this at once. The graphics card doesn't have to keep fetching textures for each material change.

So let's say I have a shop front asset to make. It has windows, signs, wood, the step, the door, maybe some objects in the window.

They all need to high resolution because the player can get their nose right up in this business.

Okay, enter texture arrays!

I bolt together all my textures for the shop front into, say, four 2048x2048 texture maps. In my material, I gather these up and lay them out in a 2x2 square.
This gives me essentially 4096 x 4096 in texture space to do what I do.

Nice.

But there is another benefit...

The street has fifty shops.

Yikes.

But never fear, because I am smart and I laid out my UVs in a square.

For my first level of detail model, I scale my UV maps by half and fit them back onto the 0-1 texture range.

I then take all my textures and in photoshop scale them down.

And I stick them together into one texture map.

So as I walk away from the shop, we switch out the material with the array of four 2048 maps, to one with a simple single 256x256 map.

Now for the advanced stuff.

Ready? Deep breath.

You can use texture arrays in the 0-1 range using masks.

So say I need four textures. If I scale the texture coordinates by 0.5, I get four repeats of the same texture in the 0-1 square...

I can then create a mask with a square for each texture, and clip them to that mask. Voila. Now I can fit textures of any resolution into other textures of any resolution.

You know you can use vertex colors as masks, right?

;)

A great use of this technique is customisable characters.

Let's say I have a customisable character, and she has texture options for eyes, teeth, tattoos and hair colour.

Rather than rendering all those materials or cramming that all into one big map, I can array.

So I UV map the characters face to within the 0 to 1 UV space.

I then create a mask with a value for each part I want to get from my texture array.

I plug in my eye texture, and it tiles them say, 10 times, but only draws the eye on the pixels with the mask.

Voila! I have one material with one shader that takes high res textures for skin, teeth, eyes, eyebrows and tattoos and composites them together. I now can switch out any of those textures at any resolution and end result is the character renders in one go.

Again, I can be smart here and have the game render the results to render textures when the character enters the scene to keep in memory as a LOD texture- as for the most part you won't be close enough to see all the fine texture detail.

Arrays of textures are powerful.

So an atlas is a single texture that several objects share. This is fast as you load one texture for everything and render one material for everything.

A texture atlas is multiple textures that are blended together at run time, so they share one material, render as one but...

...the down side is more shader instructions per pixel of your screen (each pixel of your screen is called a "fragment" in shader terms).

Now mega textures work using texture arrays that stream intelligently as your camera moves around the environment.

Basically the place I am standing determines which U,V space I am in. The UV spaces around me are populated by lower resolution textures automatically.

To picture this, say I divide up my environment up into four zones.
The village, the market, the alley and the graveyard.
In texture space, the village fits into the texture space of 0 to 9, giving me an array of 10 textures of any resolution.
The market goes from 10 to 19, etc

As I wander around the four spaces, the arrays crunch down or open up. Effectively everything I am close to is drawn in one go, using a bunch of high res textures. Everything in the distance is using tiny textures.
The materials are updated by script using my position.

If you are confused about texture arrays, picture a big assed pinboard.

On the board you pin up a picture of Owen Wilson, of the This Is Fine Dog and a note to get milk. These are your textures.

In the middle of your pinboard is a pin, this is 0,0 in UV space.

Now if you stick a pin at random anywhere on the board it can be identified by a horizontal (U) and a vertical (V) position.

What your pin hits is the pixel used by the shader.

It could be Owen Wilson's eye, the letter M of milk or a flame from This Is Fine.

That it.

I hope this makes you go wow.

You can support my free gamedev lessons by tossing a few bucks at me via Ko-Fi.

And I have a book on stuff like this in the works. Be sure to grab a copy. Like hits and submit... er... like and subscribe.

Or whatever.

X

ko-fi.com/dellak

Quick addendum- texture arrays are now being supported in hardware and engines, and these do require all textures to be the same scale. This is because they are mapped to one chunk of memory and directly accessed as if one big texture under the hood... this reduces instructions.

...but means the texture is one big chunk of memory.

1 Texture, UV range zero to one (0,0 to 1,1).

1 texture, UVs sampling from a small part of the area

One texture wrapping (repeating) as the sample area is outside the 0,1 range.

Wrap mode off. Texture does not repeat.

This is fine texture added to Owen Wilson texture.

This is fine texture offset up 1. Voila. This is essentially a texture array. Note that the This is Fine is 512x512 and Owen is 256x256.

So this polygon is actually drawing two textures at two different resolutions on the same triangles.

Now if I introduce a mask, and modify Owen's UV coordinates a bit I get this...

Which lets me do this...

If i turn off filtering and make the This Is Fine image 256x256 you can see that I am, essentially, combining textures of vastly different resolutions and offsets.

if I turn wrapping on for This is fine texture, but not Owen Wilson, you can see I get a version with and without that beautiful human being.

So I can do this...

Like this...

By mixing textures together in the shader, you reduce the state changes and draw calls, but also it lets you mess around with super crisp texture areas that are tiled over a non tiled area.

This is fine.

So am I.

Support my free gamedev lessons with a coffee or... better, composite many coffees over each other.

ko-fi.com/dellak

wow.

I CANNOT BE STOPPED

[WILSON RESOLUTION INTENSIFIES]

WHERE IS YOUR GOD NOW?

(ahem)

Maybe too much coffee this morning.

Man that is some NFT shit right there.

Did I do good? Can I go now?

For a more practical example, this is how one could lay out four textures together to render a character in one go.

And when you do one of your LOD model swaps, you can put all of these into one texture and scale down the UVs of the model... like so.

A hacky way to do this in something like UNITY is set up a camera and a quad, and render the quad to a texture for each channel. This can be done once when the character enters the game. Great for custom character systems where a character may comprise of multiple textures

Poly polys can have an array of texture partners. ;)