GLSL Data Tricks: TexelFetch, Index vs 2D co-ordinate

In my personal experience, lately I’ve been harping on something along the lines of “You’re all too obsessed with GLSL!!” as I shake my fist at the sky. Even with that said, I know the value of GLSL and it’s high demand in our industry, so I thought I’d share and explain two techniques that are useful in modern GLSL workflows and provide details on both. First up: texelFetch.

TexelFetch

In the modern GLSL pipeline, sending large datasets inside of a texture is pretty common. Everything from point clouds, particle systems, physics simulation data, and even just regular old data passed between applications can live inside of a texture. This makes accessing this data inside of a shader a relatively common activity you’ll do. You’re probably familiar with the texture() function. This is a function that takes two arguments as the input, the first being the sampler2D (fancy word for texture) and the second is the UV co-ordinate to sample. You’ve probably used it successfully till now, so why change?

There’s 2 reasons you might consider changing over to using texelFetch() instead of texture(). The first is that texture() is infinitely interpolating. What this means that if you don’t aim to sample dead in the centre of the pixel, you will get a possibly unexpected value. Seeing as we’re trying to sample data, we almost certainly do not want bleeding between the pixels. You’ve probably seen the fix to this which is to offset your sampling by half of the sample step…but do you really want to always have a workaround just so that something works the way you want it?

Enter texelFetch(). Without going too deep on the techniques behind it, one of the main reasons you should consider using it instead of texture is that it doesn’t have any filtering or interpolation built into it. It was designed for a workflow where you’re sampling data from a texture, as opposed to sampling image contents from an image.

The second benefit of texelFetch() is that you can use non-normalized texture coordinates. No more long strings of +’s and -‘s and sampleStep calculating or any of that. Whether you’re using a single index to look up your values (more on that later) or you’re sampling a position from sensor input, it’s easy to just take that position and pick the exact pixel or pixels you want to sample with an integer co-ordinate. I find this invariably makes code cleaner, less likely to contain little bugs, and just easier to deal with all around.

Remember there are two types of texture coordinates: normalized texture coordinates (ranging from 0 to 1) and texel space (ranging from 0 to the size of the texture). When using texelFetch(), we use texel space coordinates.

Two things you need to remember when switching to texelFetch() for the first time:

1. The second input needs to be an ivec2(), so when you’re passing your co-ordinates pass them as ivec2(xPositionVariable, yPositionVariable)
2. There’s a 3rd argument to texelFetch called level of detail, you can almost always just pass the number 0, which means it doesn’t do anything

So your first texelFetch() will probably look like:

vec4 someData = texelFetch(samplerName, ivec2(xPosition, yPosition), 0)

Trust me, you’ll love it and never look back

Index vs 2D co-ordinate

This is such a common struggle for many beginners of GLSL, I’m surprised there aren’t a ton of tutorials already just about this. The situation I’m talking about is where you have have an index (like point count) and you need to convert that into an XY co-ordinate on a grid or vice versa.

You’ll often encounter this situation when for example you have a grid of 400 points and you want to work with it in a shader. Maybe you have a 20×20 noise texture you’re going to sample to offset the points on a grid, but then a completely different data texture is 400×1 texture which has colour data inside of it. You need to conform these two data points together and sample them at the same time. Often what you’ll have is something like a point index value that says “I’m on point 250 of 400 now” and the rest is up to you. Lucky for us, smarter people than I have figured out the easy math for this.

If you ever need to go from a 1-dimensional index (like 250 out of 400) to a 2D co-ordinate in a regular grid (such as pixel 7 on x axis and pixel 8 on y axis) you can use the below:

x = index % width
y = index / width

In this case you’re processing the index in two different ways and getting the co-ordinate on the grid (which is pretty common thing to have to find in TouchDesigner). The width in this case is the width of your grid or texture that you need to wrap around.

The first line uses the % sign which is modulo, or the mod() function in GLSL. When you modulo something, it does division but returns the remainder not the result (like regular division). This means if you have a texture that is 20×20, as your index counts up starting from 0, the modulo of your index and 20 will return which pixels column you are inside. Then when it gets back to 20 (or any multiple of 20 in this example) the remainder is 0, so then you know you’re back at column 0 again. A little hard to wrap your head around, but try taking a bunch of numbers and doing modulo and you’ll start to visualize it better in your head.

The second line does a regular old division. Not much explain here. When you take the index and divide it by the width you’re sampling, you can figure out which pixels row you’re on. Combine these two quick math lines and you went from index -> XY position.

The inverse is just as easy if you have

index = xPosition + ( yPosition * width )

So whichever you get stuck with, finding the other can be a quick and painless process.

Wrap up

What always surprises me about GLSL is that the hangups are never exciting! The hang ups are often bugs caused by messy code, like having to a bunch of fiddling around with sample steps with texture(), or dealing with weird ways of sampling textures that aren’t all the exact same layout, like having to sample both a 20×20 texture and a 400×1 texture for the same data point. Hopefully this post helps you get through these few hurdles and sets you up to get to the more fun parts of GLSL!