Graphics in Plain Language

Parts of a frame

by baldurk

In this part - as with most of the time - we're going to look at just one frame, and talk about the building blocks that a game uses to create the final frame. Also, there are several new shiny pictures.

There are a few ways in which a frame is constructed out of smaller building blocks. The final image you see is not immediately drawn. This used to be the case many years ago, but modern graphics engines will almost always do some kind of early processing. The graphics engine will draw many intermediate images of different types to help it to calculate the final image, before finally layering up the frame you see on your screen.

These images will differ a lot from engine to engine and it will depend on what techniques the graphics programmer is trying to accomplish. For example she will want to make sure the sunlight casts proper shadows - this will require one type of image for shadows. Elsewhere she might want to have proper reflections on the car the player is driving, and this will require another image for reflections.

Some of the intermediate images used when constructing our Watch Dogs frame.

In this series I'm not going to necessarily cover every single image used in the Watch Dogs frame, but I'll touch on the main ones that can teach us something. This is the kind of area where graphics research and new techniques are being developed all the time. There is innovation and new work happening at finer grained levels, but when you hear marketing talking up some great new feature - this is usually the kind of thing they will be talking about.

---

Within these intermediate images, each is built out of yet smaller pieces. Each object in the scene, or perhaps a group of related objects, will be built separately as a Textured model. When building a game, there are artists who are constructing these models in a 3D modelling package, and creating all the resources needed. Then these models will be placed into the world in a level editor to slowly build up the virtual city.

This is reasonably common knowledge, and especially if you've seen realtime 3D graphics grow from its infancy 20 years ago you will know just how much more sophisticated today's models are. Indeed in the very early days of graphics, textures were expensive and wherever possible would be skipped to just paint simple colours on the objects. Textures would be reserved for things like eyes or faces that really needed the detail.

A 3D model is entirely made out of interlocking triangles that make up the shape of the object. Each triangle has three points, called vertices - and since the triangles interlock, the vertices will be shared between several triangles. We'll be coming back to these later, as vertices and triangles are quite important. It's also worth remembering that some objects like characters or trees will be animated before being drawn. The model is created in a default static shape, and the animations are applied each frame - again we will come back to this.

This is the 3D model for Aiden Pearce's head, after being animated. The triangles are visible as they are drawn flat, instead of being visually smoothed out as would be typical.

To give more detail to the 3D model, textures are applied. Textures are just normal flat image files, usually square or with simple dimensions like 2:1 rectangles. The textures are applied onto the 3D model via a more complex process which I can go into more detail of later, but you can conceptually think of it like very carefully wrapping a present. Instead of a simple repeating wrapping-paper pattern, the image is made precisely for that wrapping. If you've seen papercraft models in flat form that construct and glue together, this is the same principle.

This analogy is actually more apt than you might think, as the way these textures are usually created is by first 'unwrapping' the 3D model into a flat blueprint, like you would with a papercraft model, and then the texture is drawn on top. This unwrapping is often done automatically, but for particularly complex objects it might be done by hand.

This is the texture corresponding to the above model for Aiden Pearce's head. Parts are used for the teeth and tongue. Note the area above his forehead that isn't textured, as it will always be covered by Aiden Pearce's Iconic Cap™.

It's common to talk about different 'skins' for models, especially in a customisable character. These days, even what is called a skin will sometimes involve some small changes to the model - a belt changed or a different hat - but the term originally comes from when the exact same model would be used, but the texture (or skin) would be changed to produce a different looking character. Even now, a lot of variation between NPCs or objects can happen with these textures, and it saves time and cost in creating lots of unique 3D models. The different outfits Aiden can wear are for the most part just different textures on the same model.

Here's a short rotation of the 3D model head with only the texture applied, nothing else.

In the frame we're looking at, there are about 1700 objects being drawn in the main rendering portion. Some of those will be identical models - things like potted plants or bins are never uniquely made for each one, but the same model or handful of models will be placed in several areas. However the rough number of objects drawn overall to complete the frame is closer to 4700 - this gives you an idea of how much extra work there needs to happen, beyond simply drawing all of the models.

Let's take a look at another example of an object, the cap that Aiden is wearing.

This is the texture and the textured model of the cap. You can see that the texture is built up in different sections, which will all come together on the model.

All the same ideas that we saw for Aiden's head still apply to the cap. In fact apart from the specific texture and the numbers involved in computing the unwrapping, it works in precisely the same way.

To show the kind of thing that can go wrong in development, and also to show the kind of fun you can have with graphics programming, we can do a little experiment. Since most textures are a standard square size, and since the way that we wrap and unwrap textures onto models is also the same, what if we tweak it a bit? What would happen if we were to apply Aiden's head texture onto the cap model?

Aiden Pearce's Not So Iconic Cap (trademark pending).

Where the logo on his cap was on the cap texture, the head texture has the ear and teeth. Where the brim of the cap was on the cap texture, the head texture has just some hair. The mapping has applied just the same, but the wrong texture is being used. This example would of course be a bug, but think about what you could do if you could perhaps animate or flicker the texture - games use this kind of thing for several different effects that you might be able to spot now. In particular Saint's Row 4 uses this a lot for some of the "simulation" special effects.

It's also useful to think over the implications that this has - the game is being very careful in pairing up the right model with the right texture at every point, which by and large means that most unique models will have their own unique textures to match.

This isn't an absolute rule of course - in some cases to save some space, a texture will be a generic tiling pattern that can be used on many objects. Or perhaps a related series of objects - e.g. newspaper stands - will all share the same texture for newspaper covers with each using only a small section of the texture.

Still though, this means that if you want to be able to draw many images to build up your final frame you have to have all of the building blocks together that you need, and that can add up to a lot of required models and textures. Next time we're going to look at why some things like reflections are really hard to do properly, and some of the ways games will perform little tricks to save time and resources.