Archive for the 'Uncategorized' Category


“Great idea. Please create a tracker.”

Sometimes my job is pretty calm. Nobody really needs anything. My tools seem to be functioning pretty smoothly. I have time to work on my own ideas and a normal and sane pace. Other times, it’s like juggling rabid beavers in the middle of a kindergarten playroom. Fur is flying, jaws are snapping, timing is essential, and I’m going to have to deal with a lot of angry parents if I let something drop. Now is one of those times.

Great idea. Please create a tracker.

I used to know this phrase by heart. I can’t believe I forgot it. It’s so incredibly necessary.


Inlining and #includes

I’m on build coverage tonight, meaning I’m working until 2AM. Not the most glamorous aspect of the game dev industry, but when you’re trying to hit beta, making sure we have solid builds for QA every day is pretty important. So at least I’m important. Right?

So, the incremental build failed on me in an interesting way this evening. A header failed to compile because there was a class method defined (inlined) in it that used a function that wasn’t declared in any of the included headers. This, in and of itself, isn’t a particularly interesting failure. In fact, it’s a fairly typical, easy-to-catch-on-your-own-machine failure, one that you can usually shoot your fellow devs with nerf darts for causing. What made this particular instance interesting is that the check-in that broke the build wasn’t the one that broke the code.

Of course, it’s fairly common for a perfectly valid piece of code to expose a flaw in another piece of code, but I just couldn’t seem to figure out how the original code, which clearly called a function that wasn’t declared in any of its included headers, had ever compiled. My first thought was that maybe before this check-in, the header had included another header that had included the necessary header, but when I checked, none of the included headers had changed. It turns out the issue was that the new code changed the method from private to public.

Previously, the method had been private, meaning that it was guaranteed not to be used by any other translation units, so, even though the header was included by other files, that particular method was expanded. In the source file for the defined class, the header include was preceded by an include for another header which did declare the needed function. Once the method was made public, that method became expandable in other translation units, ones that may not have had the missing header in their includes.

This fix for this particular issue is incredibly simple — add the necessary header to the includes. But what’s the moral here?

  1. Always include all of the necessary headers for your code, and don’t rely on compilation to tell you whether you’ve done so. If you’re worried about build times or violating the one definition rule, use include guards. Actually, always use include guards. It’s a tiny amount of effort and extremely low-risk for a potentially epic payoff. (If you name your guards after the namespace and class that they guard, your chance of naming collisions is slim.)
  2. Please, for the love of all that is good, compile your code before you check it in.


Some people don’t like threads, and with good reason. They complicate a program and introduce a whole new class of potential bugs that you don’t see in unthreaded software. However, while I’m the first to espouse the KISS principle, I think there are certain situations (like trying to render 60 fps on an Xbox 360), where the benefits outweigh the dangers.

That said, I always feel really embarrassed when I introduce a deadlock into the codebase, especially an easily avoidable one, like I did a few days ago. Once they’re in, deadlocks are devious little bastards and managing to reproduce them with a debugger attached is a royal pain. Assuming you’re lucky enough to discover that they exist before your game ships. That’s why you have to catch them before they make it into your code repository. So how do we avoid this terrible scourge? With this one simple rule:

Always grab mutexes in the same order.

It really is that simple. If you know exactly what mutexes are being acquired, in what order, by every part of your code, you’ll be absolutely fine! Of course, this is akin to someone telling me to eat right and exercise. Sure, it’s simple and effective, but with that kind of effort involved, I’m more likely to go for the tube of cookie dough and some miracle drug featured in 3AM infomercials. But, just as every dieter has their own tips and tricks for avoiding brownies and hitting the treadmill, I have a small but growing collection of guidelines for writing thread-safe code.

  1. Minimize the scope of your mutex grab. This one is easy. The greater the number of lines of code executed while you’ve acquired a mutex, the greater the chance that one of them acquires a different mutex. The more mutexes acquired, the greater the chance of deadlock. So restrict the scope of your mutex to what it is supposed to protect. No more, no less. If you haven’t yet subscribed to the Resource Acquisition Is Initialization philosophy, now would be a good time to start, and apply it to your mutexes.
  2. Know what your mutex meant to protect. This goes hand-in-hand with the previous guideline. If you don’t know what you’re trying to keep consistent, you can’t know what the scope of your mutex should be.
  3. (Try to) know the order of mutex acquisition in the codepath you’re modifying. This one is a bit trickier. The advice I’ve always heard is to designate a mutex order from the start and always abide by it, but in practice, I’ve yet to see that happen, and honestly, I can’t think of a good way to document and communicate that to new coders, especially with a changing codebase. Instead, you usually have to determine the order by reading the code. Going down the chain is pretty straightforward (if sometimes a bit tedious), but determining  all of the clients of your code and what order they grab mutexes can be nearly impossible, especially if you’re working in a large codebase or your mutex is embedded in low-level code (like, say, new or alloc). Theoretically, it seems like it would be pretty easy to make a tool that crawls your code (like test coverage tools do) and writes out an ordering, though I’ve never done it.
That’s all I’ve got. It’s not perfect — even if you do know the order of mutex acquisition in your codepath, it still might be different in another codepath. But I think these guidelines make some of these nasty bugs a bit more avoidable. The bug I fixed today, for example, could probably have been avoided if I had not assumed that the code it called didn’t acquire any mutexes. If I’d noticed that, I might have then gone on to see where else that mutex gets acquired, and noticed that I was introducing an out-of-order mutex grab. Or maybe not. But at the very least, I’d have been aware of the order for the next time I modified the code, something that has saved me some pain multiple times already.

Yes (/WX)

About a month ago, I realized that I was learning a lot and not writing it down. So I decided to start writing it down. A month later, I don’t remember what I learned that was so important, but I figured I should just start writing. Some of it will be technical. Some of it will be common sense. Most of it, you’ll probably have heard before somewhere. I’ll start with something that I tend to re-learn every project.

Treat warnings as errors: Yes (/WX)

I know some of you are saying, “But I plan to use that unused variable later!” or “Hey, I *know* that variable gets initialized in one of the branches of my if case, so it’s okay.” To put it bluntly, you’re wrong.

If you’re not using the variable now, replace it with a TODO comment, which has at least some chance of reminding you that you were supposed to do something with it. (No, it’s not a good reminder, but trust me, one more warning in the inevitable sea of warnings will not be any better.) And as for that variable that you know is going to be initialized in one of those branches, when was the last time you touched a piece of code and it never changed ever? That never happened? That’s what I thought.

But why should you care that your cup overfloweth with warnings every time you build your solution? Well, for one thing, they’re warnings, not beer. They’re an indication that something is behaving not quite as intended. It may mean a little bit of code bloat, which is not such a big deal. Or it could mean that you wrote this code when you were really tired, and you didn’t actually use the variable you thought you were using. But if it’s only a warning, amid a sea of warnings, you’re not going to notice until your code doesn’t do what you thought it should do, and it’s the middle of crunch and you’ve just forced QA to fail the build that was supposed to go to cert.

Or, more importantly, you might find that some other project that you didn’t build locally does treat warnings as errors and then your change fails on the incremental build, and everyone will point at you and laugh.


My name is Maitland Lederer, and I’m a video game developer. I learn stuff you probably already knew and have opinions you've probably already heard. I figured it might be a good idea for me to start writing down the stuff I've learned so I don't have to relearn it. It's not, like, great wisdom or anything. It's just things I happened to learn, usually today.

Header photo by D Sharon Pruitt, used under a Creative Commons License.