CTF Design 101
I’m frequently asked how I go about building a successful CTF and while I’ve tried to answer this in the past (generally on Twitter), I’ve never actually gone in-depth on it. So today we’re going to talk about what makes a CTF successful.
A word of warning: I’m writing this entirely from my own experience as a CTF builder, not as much of a CTF player. While I’ve participated in a couple over the years, I’m actually quite bad at them in most cases and need significantly more practice (… and free time) to change that. Additionally, I’m only going to be talking about “jeopardy-style” CTFs (a term I absolutely despise!) wherein tasks are laid out and the players win by completing those tasks. With that warning out of the way…
So You Want To Build A CTF #
Before you write a single line of code or start designing challenges, you need to ask yourself one question: when the players are done with your CTF (win or lose), what do you want them to take away from it? That might be any combination of:
- New knowledge
- Enjoyment
- Prizes
- Engagement
- Awareness of your/a sponsor’s brand
- <Insert other takeaway here>
If you have a single goal, congratulations! That makes your life a lot easier. However, in most cases you will be trying to achieve more than one goal with your CTF; for instance, the Hacker101 CTF was designed primarily for new knowledge, with enjoyment and prizes (specifically, private invitations on H1) being secondary goals. Enjoyment really needs to be on every CTF designer’s list; if people don’t have fun, they’re not going to learn/win prizes/buy your stuff.
You don’t have to assign concrete numbers to this, but you should loosely prioritize them in your mind.
Themes #
In order to provide a cohesive experience, it’s really important to consider what the themes of your CTF will be. This has two major parts:
Explicit Story #
Often you’ll be designing with some specific setting and story in mind. This isn’t necessary, but it can help with engagement and enjoyment, which often act as a trojan horse for other goals. For instance, I designed the 2019 44con CTF while at HackerOne. That year’s theme for the con was Blade Runner, which gave me a fantastic setting in which to tell a story. I imagined the hackers sitting down at one of the terminals and breaking into Tyrell Corporation, going from ‘room’ to ‘room’. Take this shot from the movie:
Then look at the opening text you get when you telnet into the CTF machine:
I wanted to elicit that nostalgic feeling while providing something new. That concept was a huge contributor to the success of the CTF.
Implicit Story #
No matter the medium, communication always tells a story. Whether you’re painting dogs playing poker, giving a sales pitch, debating the merits of pineapple on pizza, or putting challenges together for a CTF, you are telling a story. If you don’t think about what story you’re telling, the likelihood that it comes through clearly is pretty low.
What do I mean by story in this context? I mean the actual, functional narrative your participants are going to experience, ignoring the explicit themes or flavor text. For instance, if I want to succeed in a CTF I probably need to first get access, then sign up, then find and solve challenges in some order. That’s a story as much as dogs playing poker, and it leads to the most important aspect of CTF design.
Play More Games #
CTFs should be first and foremost about the game design and storytelling (implicit and explicit), and secondarily about technical aspects. If I want to sell you on Final Fantasy 7, I’m not going to say “oh man, it’s got this really awesome transition from the field to the battle!” I’m going to say “it has the unarguably best villain of any game ever made.”
Your players should be able to work through where to go without you explicitly pointing them in a given direction. This applies to the CTF infrastructure (e.g. registration, accessing levels, etc) and the path to and from the flags. The good thing here is that there’s an entire industry focused on this task: game development.
There’s so, so much you can learn from existing games and their developers. A great example is the “tutorial” in the original Portal by Valve:
There’s also a lot to be learned from tabletop games, but that’s a story for another day.
Design Your Difficulty Curve #
Every game has a difficulty curve that it aims for. This is a graph with intended difficulty on the Y axis and time on the X axis:
That graph is from a fantastic article on the subject, which you 100% should read: The idea of a difficulty curve is all wrong It’s all from the perspective of a video game designer, but there’s absolutely no reason you couldn’t (or shouldn’t) apply the exact same ideas to your CTF designs.
This one is hugely dependent on your goals as a designer, as well as your audience. If your target audience is hardcore CTF players and your goal is to get awareness of your brand, your best bet is a difficulty curve that looks more like Everest than a gentle climb. If your audience is brand new hackers, you should probably follow a more traditional game design curve. If you’re giving away prizes, you probably want the end to tick up pretty considerably, to make them really earn it. I could go on and on, but I think you get the point: this is something you should decide for yourself, because every CTF will be different.
Assessing Difficulty #
For me, this is the hardest part of building CTFs. I may not play many CTFs, but I have been hacking for a long time and my perspective on these things is going to be vastly different from most. As such, figuring out how difficult a given challenge will be is a struggle.
The only way I’ve really found to consistently and accurately measure difficulty is relatively large-scale playtesting: I try to get at least 10 people to work through the level and give detailed feedback. In the future, I actually plan on giving people access to a box where they can do the CTF and I record their entire session, to get better data.
In lieu of this, you can simply guess and hope for the best. To be honest, in most cases this is probably fine, even if not optimal. The reason this is usually okay is the next item on the list:
Always Give Players An Out #
Players should never feel like they absolutely have to do the next challenge, where the alternative is just quitting outright. If you allow players the freedom to take a break from one challenge by working on another, you achieve a few important goals:
First and foremost, you keep them engaged. The worst thing a CTF designer can do is make a player quit outright. If you get them to go over to another level, it’s much more likely that they’ll come back and finish the one that kicked their ass.
Second, they’ll often think of the solution when they’ve switched their brain over to another challenge. But for this to work, they really need to be able to downshift; if the only thing they can do is go to an even harder level, that won’t work. Consider this: if you’re playing an RPG and you get owned by a tough boss, do you go kill an even tougher one? No, you go get some better gear and kill some low-level enemies to build your strength, then you go back and kill the big baddy.
Third and most importantly, you give them an opportunity to get a win (assuming you gave them that opportunity to downshift). It can be incredibly demoralizing to hit a brick wall when solving a challenge, but if they get a win elsewhere it will completely reverse that. Every one of those negative feelings from the hard challenge will suddenly turn into a drive to win.
Be Mean, But Not Too Mean #
My CTF players often curse my name, to put it mildly. They do this because I make some of my challenges very hard; much harder than I think I should, sometimes. But if there’s one thing I’ve learned, it’s the the sense of accomplishment you get is exponentially higher than the difficulty you overcame. I always provide my players with an out, so that I can push them past their existing limits, while not making them quit (usually).
This is where it’s really important to know your audience and study game design closely, because figuring out that balance is critical to your success. If you make your CTF too easy, your players won’t get a real sense of accomplishment, but if you make it too hard (without giving an out) then they’re liable to quit; in either case, you won’t have achieved your goals, whether that’s for them to learn, for you to be able to sell them something later, or otherwise.
Push Yourself #
Some of the best CTFs I’ve ever built were using/about technologies I knew absolutely nothing about beforehand. I built the CTFs as part of my own learning, and that exploration mapped to a fantastic narrative for the players.
A good example of this is the GraphQL levels on the Hacker101 CTF. Before I built the first level, I had only very briefly touched it, and certainly never built an application using it. I had so much fun with that level, I ended up making two more and releasing them all as a “themed week”. They’ve consistently been a favorite amongst participants.
Conclusion #
I hope this gave you some insight into how I go about designing a successful CTF. There are a million things I could tell you about designing specific challenges, but at the end of the day, that’s something you need to learn for yourself.
Your first few CTFs will be very bad. The next few will be bad. Then after that maybe you’ll start getting a good one here and there. Then occasionally a great one. Like anything else, this is a skill that must be practiced consistently! If someone told you that you could pick up a violin for the first time and start playing Ravel, you’d tell them that they’re on crack; why would it be any different with building CTFs? The key thing here, though, is that you can do this.
Good luck and happy hacking,
- Cody Brocious (Daeken)