Graphics in Plain Language

Introduction

by baldurk

Hi, I'm baldurk. I've worked as a graphics programmer for a few years now, so while I'm by no means an expert I'd like to think I understand a fair bit of what's involved in graphics work.

The idea for this series of posts has been in the back of my mind for a while, and it recently resurfaced reading an interesting article breaking down the new Deus Ex.

I think graphics - especially how complex it gets in modern games - is an interesting topic. Few people are going to really dive in to learn all the intricacies, but I think there are many topics which are interesting to anyone. I imagine most people who've played games have been curious about how some effect is achieved, or seen a new game with graphics that blew them away and wondered what technology goes into putting it all together.

There's a lot that goes into even making a simple 3D game, let alone a game like Watch Dogs.

I have a vague idea of what I'm going to cover in this series, but it will probably also depend on what sparks interest. The main idea though is to outline the kind of things that happen inside a modern game without losing anyone - I won't be assuming any math or programming knowledge. If you know the difference between a CPU and a graphics card, and you know that memory isn't the same as a hard drive, that's enough and I'll explain anything else on top of that.

These posts will be going along with Chip & Ironicus's Let's Play of Watch Dogs to give myself a little structure. It's a game well known for its graphics (for better or worse depending on who you ask), and it has a lot of things we can look at for specific examples. If there's a particular demand I can talk about other games.

I'll start off covering the basics which are fairly similar from game to game but I'll look at some of Watch Dogs' techniques and visual effects as well.

I'll be using a tool I wrote in my spare time called RenderDoc which is used for debugging graphics problems - it helps you deconstruct a graphics frame, and this will let us peek at how it's put together.

This animation shows part of the frame being built up over time, as the graphics card draws each part.

Most people already know that computer graphics (and any other video really) is made up of a series of still frames, shown for fractions of a second each. Film has traditionally been 24 frames per second (FPS), TV has been similar, around 24-30. For games the FPS can be variable as a lot of work has to happen for each frame. Dropping below 30 is undesirable, although it often happens. 60 FPS tends to be an upper limit for console games, with games aiming for 30 or 60 depending on their goals. On PCs if you have a higher-rate display you can go for 90, 120 or above. The reason for these particular numbers has to do with vsync, which is something we can cover later.

From our perspective, we need to flip this problem on its head - instead of looking at how high the FPS is, we look at how short that makes the time allotted for a single frame. If we want to run at 30 FPS, we only have 33 milliseconds to do everything needed for a frame. At 60 FPS it's half that, about 17 milliseconds. Even for a computer, that's not a lot of time considering everything that has to happen. To put it in perspective, a back of the envelope calculation says a bullet will travel about 1 metre per millisecond.

We'll be mostly talking about PCs here since the platform is public and I can't talk about specifics without worrying about breaking NDAs from consoles. For the most part, I'm not going to talk about anything that would be different on consoles anyway - if I do then I'll probably call it out. As for mobile platforms, most of the differences between PC/console hardware and mobile hardware is out of the scope of this series.

Honestly this picture is just here to look shiny so that this page isn't all words.

This is the task we're interested in - we don't have to worry about getting the AI to do all its calculations, or doing the physics simulation to figure out where things have moved to. The boundary of what you'd call "graphics programming" is a bit fuzzy, but I am going to make the distinction that graphics programming starts when you have all of the information you need to make a frame: You know what things are happening, you have the textures and models in memory (not on disk), the animations have been animated, the physics have been physiced, and we now have to draw the final frame that will be displayed.

As a side note, I will be focussing on a 3D game with fairly conventional rendering like Watch Dogs - a lot of the core principles apply to 2D games, but it's a little harder to illustrate the concepts. Also a disclaimer, especially to any other graphics programmers, I'm aiming for understanding so I'll probably include some very dubious explanations that are only barely true if it aids that goal!

So if learning (roughly speaking) how that is accomplished in and seeing everything that we need to worry about piques your interest, then these posts might be for you! If you have any feedback, requests for further explanation or if you think there's something in particular you want me to look at just let me know via twitter or email.