30
Jul
11

ThreadHead

Some people don’t like threads, and with good reason. They complicate a program and introduce a whole new class of potential bugs that you don’t see in unthreaded software. However, while I’m the first to espouse the KISS principle, I think there are certain situations (like trying to render 60 fps on an Xbox 360), where the benefits outweigh the dangers.

That said, I always feel really embarrassed when I introduce a deadlock into the codebase, especially an easily avoidable one, like I did a few days ago. Once they’re in, deadlocks are devious little bastards and managing to reproduce them with a debugger attached is a royal pain. Assuming you’re lucky enough to discover that they exist before your game ships. That’s why you have to catch them before they make it into your code repository. So how do we avoid this terrible scourge? With this one simple rule:

Always grab mutexes in the same order.

It really is that simple. If you know exactly what mutexes are being acquired, in what order, by every part of your code, you’ll be absolutely fine! Of course, this is akin to someone telling me to eat right and exercise. Sure, it’s simple and effective, but with that kind of effort involved, I’m more likely to go for the tube of cookie dough and some miracle drug featured in 3AM infomercials. But, just as every dieter has their own tips and tricks for avoiding brownies and hitting the treadmill, I have a small but growing collection of guidelines for writing thread-safe code.

  1. Minimize the scope of your mutex grab. This one is easy. The greater the number of lines of code executed while you’ve acquired a mutex, the greater the chance that one of them acquires a different mutex. The more mutexes acquired, the greater the chance of deadlock. So restrict the scope of your mutex to what it is supposed to protect. No more, no less. If you haven’t yet subscribed to the Resource Acquisition Is Initialization philosophy, now would be a good time to start, and apply it to your mutexes.
  2. Know what your mutex meant to protect. This goes hand-in-hand with the previous guideline. If you don’t know what you’re trying to keep consistent, you can’t know what the scope of your mutex should be.
  3. (Try to) know the order of mutex acquisition in the codepath you’re modifying. This one is a bit trickier. The advice I’ve always heard is to designate a mutex order from the start and always abide by it, but in practice, I’ve yet to see that happen, and honestly, I can’t think of a good way to document and communicate that to new coders, especially with a changing codebase. Instead, you usually have to determine the order by reading the code. Going down the chain is pretty straightforward (if sometimes a bit tedious), but determining  all of the clients of your code and what order they grab mutexes can be nearly impossible, especially if you’re working in a large codebase or your mutex is embedded in low-level code (like, say, new or alloc). Theoretically, it seems like it would be pretty easy to make a tool that crawls your code (like test coverage tools do) and writes out an ordering, though I’ve never done it.
That’s all I’ve got. It’s not perfect — even if you do know the order of mutex acquisition in your codepath, it still might be different in another codepath. But I think these guidelines make some of these nasty bugs a bit more avoidable. The bug I fixed today, for example, could probably have been avoided if I had not assumed that the code it called didn’t acquire any mutexes. If I’d noticed that, I might have then gone on to see where else that mutex gets acquired, and noticed that I was introducing an out-of-order mutex grab. Or maybe not. But at the very least, I’d have been aware of the order for the next time I modified the code, something that has saved me some pain multiple times already.
Advertisements

0 Responses to “ThreadHead”



  1. Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


About

My name is Maitland Lederer, and I’m a video game developer. I learn stuff you probably already knew and have opinions you've probably already heard. I figured it might be a good idea for me to start writing down the stuff I've learned so I don't have to relearn it. It's not, like, great wisdom or anything. It's just things I happened to learn, usually today.

Header photo by D Sharon Pruitt, used under a Creative Commons License.


%d bloggers like this: