Auto-dynamic difficulty

There is a zone beyond that which is known to game designers. It is a place as vast as space and as timeless as infinity. It is the middle ground between boredom and frustration, between easy and hard, and it lies between the tips of our fingers, and the sparks within our CPUs. This is the dimension of pleasure. It is an area which we call... THE FUN ZONE.

(Forgive me, Rod!)

Okay, that out of the way, I'm going to start off with an actual, complete email I sent to the project leader of Max Payne a few months before the initial release of that game in mid-2001, almost three years ago:

From: "Scott Miller"
To: "....... ......."
Sent: Thursday, April 12, 2001 1:54 PM
Subject: Re: Max Payne Build 125

Here's the reasoning behind not having an easy skill level: We do not want players to finish the game and not have a satisfying feeling of accomplishment. Too many players will pick the easy mode just to get through the game, and end up finishing and think, "Well, that was nothin'! Where were the enemies!? Where's the challenge?!" In other words, if we give players a chance to shoot their own foot, some will.

Diablo 2 obviously didn't suffer from not having a skill level choice -- it's one of the three best selling games ever, with 2M and still selling.

Also, my view is that as developers, it should be *our job* to properly play balance the game, not the player's choice. In other words, skill levels are an easy way out for developers too lazy or incompetent to properly play balance their own games. (Okay, a bit harsh, but I'm saying this as someone who used to be a strong proponent of skill levels, too, so I was part of that incompetent developer group! *grin*)

Finally, a skill level screen is just another screen getting in the way of enjoying the game itself, like those boring as hell logo screens that I hope to g.o.d. we can get rid of in all future games we release!

Anyway, the perfect game will adjust its difficulty based on the player's personal ability. This was done in arcade games going back to Atari's Xevious in 1982. It should be really easy to do: Simply have a few variables that rate the player's ability, and the player's rating (completely internal to the game) determines the damage that both the player's weapon delivers, and the enemies' weapons deliver against the player. You do not want to adjust the health benefit of pain killers because that's something the player might notice being adjusted as the game goes along because it's represented by a visible bar.

A few ideas for variables that determine the damage ratings of weapons might include:

o Max's average health (i.e. if it averages around 25%, then the weapon power variables need to be adjusted to favor Max slightly, but if Max's health averages 80% or higher, then up the difficulty).

o Kills made per level, vs number of possible kills per level. A good player will leave few left standing, if any at all.

o Number of times Max dies per level -- this is a good indication of how good the player is.

o Note: I wouldn't keep track of shots fired or number of saved games--those can be misleading either way.

Finally, never make a change during a level, only between levels.

At the end of the game, via a cheat code, you could plot a graph of the ups and downs as the player played the game, which would be a very interesting graph to compare with other players!

Scott Miller
3D Realms Entertainment
www.3DRealms.com

As can be gathered in this email, it's my opinion that games should only rarely allow players to set their own difficulty level. Afterall, it should be the designer's job is to insure the proper play experience, not the player's. The idea behind auto-dynamic difficulty (ADD) is to keep the player in the sweet spot, where the game remains challenging, but never impossible. A perfect game experience is one that constantly tests the player's skill, without being so hard they want to give up. Completing such a game gives a player an elated feeling of accomplishment, because they know they passed a tough test of their ability.

One of the key challenges developers must overcome in broadening the appeal of our games is making our games less hardcore in nature. We need to improve in many areas, like interface, content, and adding emotional involvement for story-based games. ADD should become one of our best tactics to help us achieve this important goal.

One of the most common ways games sabotage their potential to appeal to larger numbers of players is by being too difficult (or too easy, but that's much less common). Practically everyone designing games nowadays is a hardcore player with elite skills. It's therefore easy for game designers to misjudge the difficulty of their own games, making them too hard for average or new players.

Sequels, especially, are a breeding ground for ramped up game difficulty. The natural tendency with sequels is to add more features, leading to more controls. And higher game difficulty. Also, with sequels designers naturally assume that players have all the skills learned from the previous game, and so the challenges are made tougher for new installments. Not exactly an inviting situation for anyone but the hardcore gamer.

ADD provides the near-perfect solution for most problems dealing with game difficulty. At least if implemented well, but that's the case with any facet of design. There are two primary issues with implementing ADD:

[1] The first issue when implementing ADD is to make it invisible to players. It should not be obvious when the game is self-adjusting its difficulty level, as this can hurt the game's immersion.

[2] The other issue is to make it so that's it's hard for player's to manipulate the system, taking advantage of it. For example, a player might play very poorly for the level leading up to the boss, just to make the boss a breeze to beat. While clever players will always be able to manipulate games with ADD, there are ways to make manipulation more trouble than it's worth. And, the bottom-line is that players who do this are only cheating themselves, so perhaps this is not too great of an issue.

One last point that developers should consider: If a player completes your game, they are much more likely to buzz about, spreading the word that it was a great game. We all love to brag about feats of accomplishment, and beating a game, for a gamer, is near the top of the list. A game with a well implemented ADD system is more likely to allow players of wide ranging abilities to complete your game.

I think Max Payne benefited greatly from its implementation of ADD. And I think this should become a standard feature for most games. At least, if broadening our customer base is of any concern.

Comments

[Collin van Ginkel]

Interesting article I must say!

I'm sure you look at things from a console/PC perspective, but we've had a similar experience while balancing Toki Tori ( http://www.tokitori.com ) for Game Boy Color. It's a pretty simple-looking game but it's one of the hardest games around on GBC to complete.

Because Toki Tori levels are tightly designed puzzles for the player to complete we couldn't really adjust any game settings dynamically. We knew we were creating a tough game, so we didn't make it mandatory to complete all the levels in the game. You only have to complete 40 of the 60 levels in the game to see the ending, and when players actually finish all the levels they are rewarded with a special gift.

If even 1% of all the buyers completed all the levels I'd be surprised. But still, the reactions we get from that 1% are totally worth the extra effort we put into it.

[Brad Renfro]

I am all for ADD, and even other forms of intelligent adapting dynamic gameplay systems. I thought it worked really well in both Max Paynes. But here are a few more issues I came up with.
- Sometimes, I feel like breezing through a game just to see its story. Then I want to manually pick the easiest setting.
- Sometimes, I feel like I am a badass because I specifically choose to play games on the hardest difficulty. I want the choice and honor of picking nightmare difficulty. The elite difficulty modes in the Max Paynes and Diablos worked well but had to be unlocked.
- Even though it's dynamic, the ADD designer still has to concretely determine what constitutes easy difficulty and hard difficulty. It still involves setting down tweaked variables. If a designer would improperly balance a non-ADD game, what guarantees that this designer would properly balance an ADD game? Even worse, if the balance of an ADD game fails, the player is stuck in this unbalanced game with no difficulty choice.
- ADD is a system that needs as much tweaking as normal difficulty level tweaking, if not more. In fact, it requires significant extra code and testing. The normal reasons that cause developers to ship unbalanced/unplaytested games apply to an ADD implementation as well (lack of time, lack of manpower and budget, poor design principles). If anything, the extra work involved means that games could be even worse if poor ADD implementations become the norm.

I guess even with ADD, it still comes down to design talent and time/budget to allow design polish.

[JP]

"Number of times Max dies per level -- this is a good indication of how good the player is."

Actually, in the case of Max Payne, it's not really. The game frequently kills / damages the player with no recourse because the player has no information on what's going to be in the next room and the weapons are frequently lethal enough to whittle a player down from full or near-full health to zero in a short time. Ask anyone who played the game and they will all say they had to use quicksave to scout rooms at least a few times. Jump through door (because of the way the bullet time economy works there is never a reason not to use it), get blasted by the customary two-guys-with-uzi + one-guy-with-shotgun, quickload, do the same thing but ace the room now that you know how to handle the situation. This is called Teaching with Death, and anyone who does it needs to go back to game design school. Making your ADD algorithm aware of how many times the player has died is throwing good effort after bad.

Also, the citation of Diablo2 at the beginning of the argument - a completely different game in a rather unrelated genre that just happens to have sold really well, a game that *does* actually offer a skill level choice once you beat the game once and has no ADD - is pretty much useless and goes to show how rudderless your game design practices are if sales are your only quality metric.

Also, what exactly does it mean for something to be "auto-dynamic"?

[Jeff Mackintosh]

If a designer would improperly balance a non-ADD game, what guarantees that this designer would properly balance an ADD game?
-----
I think Brad brings up an excellent point. While I think an ADD game is the right choice (I think it makes a game better in virtually every way), I also think that a weak designer using difficulty levels as a crutch will do a poor job designing an ADD game. If they need the crutch to make a good game, that almost ensures they will screw up a game without the benefit of that crutch.

Of course, in a perfect world, all game designers would be good at their job and not require crutches, but we know we don't live in that perfect world.

Well, not all of us at any rate. :)

Anyhow, that was another excellent and thought-provocking post. Keep up the great work.

[Gestalt]

Dynamic difficulty settings are a nice idea in theory, but as has been pointed out already - 1) players don't always want the game to be challenging but not quite impossible, and 2) Max Payne isn't a great example because its dynamic difficulty settings didn't always work very well.

Personally I found some of the later stages of Max Payne virtually impossible. I'm guessing this was because you didn't use frequency of saving and reloading as a factor in the balance equation. The harder the game got, the more often I would quick save and quick load, often not waiting for Max to actually die but simply reloading if I was badly injured in a fight and there weren't any painkillers immediately to hand. Judging from your e-mail above, this might actually have made the game harder instead of easier! Which would make me reload more often, which would make the game even harder, which... ;)

Until games can accurately judge how tough I'm finding the game and whether I'm having fun, dynamic difficulty systems shouldn't adjust the difficulty too drastically, or they might make it completely unplayable. And they should start from a base easy / medium / hard difficulty setting that players can pick manually because no game, however clever the coding behind it, can tell how hard I *want* the game to be.

[Brian Gladman]

I agree with the earlier posters, in that MP (particularly MP2) was an exercise in QuickLoad/QuickSaving, more than anything else. Enjoyable nonetheless, but still required a QuickSave after every battle. This is not good design, IMO.

The best example of "ADD" I can think of is Sly Cooper. This is a game I could play with my 4 year old son, and it would adjust accordingly, being more generous with gold coins/extra lives when he was playing (and dying!), and more stringent with the goods when I was breezing through it.

[Scott Miller]

All: I'm not saying that Max Payne had a perfect ADD system in place, because I know it didn't. But, if more developers started using ADD, we'd see gradual improvement over time. Most games do not use ADD, especially shooters, so Max was on the leading edge within its game category, and leaders seldom get everything right.

The key point is that ADD is to the benefit of players, and therefore the industry. It allows games to better cope with a wider range of player abilities. But in its own it will not correct overly difficult level design, tough jumping puzzles, and so on. Level designers must still do their jobs well for ADD to shine.

Re: Diablo, true it didn't have ADD, but I thought it made for a good example, due to its success, of a game that didn't allow players to select a difficulty level. ADD caries this one step further.

Also, the technique we used in Max Payne allows players to select a harder difficulty level AFTER they've finished the game, as at this point it doesn't matter, and players often desire a harder challenge if they replay the game.

[Benjamin Graner]

Interesting read, and I can see ADD being useful in balancing the needs of hardcore players while keeping the game accessible to the newbies. I just want to point out my biggest problem with ADD and why I feel it should be used with a bit of caution.

In a sense, ADD basically says to the gamer 'you are going to beat this game no matter what'. If the game adjusts it's level of difficulty so that the game is essentially always beatable by that player, where is the satisfaction in finishing?

It's like the game of poker I played last night with my friends. At the end of the night I was down $5 and my friend was sitting on a huge pot. I guess we all looked glum, because he tried to give back the money he'd rightfully won. None of us accepted. If we let him, what was the point of playing at all? Poker is exciting because you have something to lose. When you win, there is satisfaction because you might not have been so lucky, and you were willing to put something on the line for a chance to play & win.

ADD, if it's opaque to the player is the same as my friend giving back our money...it softens the blow of defeat, but also takes away from the satisfaction of winning. Why play a game you know you're going to beat? Where is the fun in that?

Personally, I'm biased toward the Final Fantasy solution: Make intermediate enemies plentiful and easy to beat, and ramp up the difficulty for boss's and out of the beaten path areas.

So you want that BFG? You're going to have to go through the uber-mutant which is guarding it.

I guess attention deficit disorder is great if you can tell a gamer is really having a hard time and needs some help, otherwise I would personally throw lots of easy enemies at a player with one or two major obstacles.

[Scott Miller]

-- "In a sense, ADD basically says to the gamer 'you are going to beat this game no matter what'."

Great point, Ben, but is this really so bad. As long as the road is difficult, I *want* people to finish the games I help make. And even with a perfectly tuned ADD system, it's still up to the player to beat the game -- it doesn't become a walk-thru.

[Nathan Peterson]

I'm also assuming that Scott ment there would be a 'floor' to the difficulty level, so the game doesnt become so easy, that anyone who plays it the first time, can simply walk thru it?

I would be a fan of more hybrid approach, where ADD could change the basics of gameplay (such as thuglike enemies behavior) and then for intermediate bosses or level bosses, to have a set difficulty, or at the least, a much smaller scale ADD difficulty scale depending on the players performance in that level.

That way, obsticals are still obsticals, and a player has to really work to move foreward, and to advance their skills in the game.

hence making the ADD function more later, when the player starts to use more skill against tougher enemies, or bosses.

just a thought, on how ADD could work, and still provide a satisfying 'completion' feeling.

-nathan

[Factory]

Hmm another way of doing ADD is done by roguelikes, which is to put a series of gradually increasing in difficulty levels, and let the player go down as fast as they like. The player sets their own difficulty level by adjusting the number of levels they down to. The incentive to go down is that you get more loot the further down you go. This system has the problem that one can easily go down too fast, and get yourself in trouble, and also that the speed at which a character descends is not generally linear, some levels can be descended faster than others.
Diablo has a similar system, althought the end of level bosses tend to act as obsticles that must be passed, while a roguelike like Angband will not be as strict. 'Boss' monsters will generally hang around a set of levels and be an annoyance to the player. (if a Boss is too hard, it's generally easy to flee past it)

[Jim Vessella]

I really love the comment Ben made about side quests. From my experience, most people who play RPGs will first play to beat the game and see the story. Afterwards they're go through and try to collect all the items, beat all the monsters, and explore every inch of land until its painted with your dominating presence.

While there is no actual difficulty setting at the beginning of the game, the player can choose their difficulty by the actions they make. Some games now give you a "percentage completion" meter that reflects how much stuff you've done in the game. A great example is Metroid Prime, where at the end the player is rewarded with a different cinematic depending on how much they completed. Not only does this provide a total esteem boost when you complete the game with say 90%, but it gives you that extra incentive to go back and complete 95 or 100% if you're super hardcore.

[Cebrian]

ADD introduces an element of variability at the rules that the player can easily identify as cheating. Players usually want a "fair game", one with a fixed set of rules they can feel and understand: a bullet to the head will kill, three bullets will surely kill this particular enemy, and so on. It also hurts the immersion and the solidity of the game world. A main premise of the ADD is that the player won't know it exists. As a game mechanic, penalizes good play while rewarding bad players. It just doesn't sound right. Maybe it can work, but I'm unconvinced.

[Gestalt]

Cebrian - "ADD introduces an element of variability at the rules that the player can easily identify as cheating"

Good point. Also, adjusting the damage weapons cause is a pretty blunt way of adjusting the difficulty. Why is it that most games just throw more enemies at the player, make their weapons cause more damage or give them god-like aiming on higher difficulty settings, rather than (say) adjusting the way the AI behaves? I can see that the AI programmers might want players to always see their creations at their best, but that's no fun if they keep killing you! Wouldn't it be cool if enemies on higher difficulty levels made more use of cover and were more coordinated with each other, rather than just having bigger guns or more reinforcements?

[wormstrangler]

"Wouldn't it be cool if enemies on higher difficulty levels made more use of cover and were more coordinated with each other, rather than just having bigger guns or more reinforcements?"

Would you like to pay for the extra coding time? :P

It just seems like a big hassle but one which would have good benefits for the player.

I hate how AI on a high difficulty is just given an accuracy of 99%.

So these types of implementations all comes down to time and the AI coders experience. AI programming has been lagging behind for a while. Maybe a 3rd party should start making AI geared towards one or two types of genres (FPS generally) and then license their technology out, like physics and even engines in general.

I'd like to be a start up in that industry, but it ain't going to happen, for me.

[Roger Taylor]

You say that skill levels are a lazy and easy way out of properly play balancing, that players should play the experience that the designer provides and imply that ADD is a way to properly play balance a game. When I read this in the e-mail and the text following it, my mind can't help but wander back to the first piece of text on the page that reads to me as "designers don't understand fun."

When I read your blog entry, I have this picture in my mind of having to adjust my game playing to change the difficulty to best match my idea of fun. I see the difficulty curve that ADD allows for in a game and think that by intentionally or subconsciously retarding how I play the game to make the game more fun for me, I'm intentionally making how I play the game more of an obstacle in order to make how the game plays less of an obstacle. And then theres no guarantee that the fun I can get out of the game by doing so is anywhere near what it could have been given the game engine and game design unconstrained by the ADD. Skill levels on the other hand allow me to adjust the difficulty to suit my idea of fun without necessarily having to retard how I play the game. Which isn't to say I wouldn't have to retard how I play the game, but that I guess I would have to retard it less and might gain an order of magnitude more fun out of the game because of it.

Like most other things, it probably comes back to how well the thing was done, rather than just the approach taken. Perhaps ADD can be done well. I could see it becoming am lazy and easy way out of properly play balancing as well.

What really ruins the immersion and immediately sucks the fun out of a game is the point where it becomes obvious that the difficulty is a pointless obstacle that serves only to slow down my progress in the game. Suddenly a game I was really enjoying feels like a shallow game, like dungeon siege or a bad space invaders clone, endless waves of targets I am forced to pointlessly and tediously kill. The beautiful environment that is the game becomes tainted by the bitter memory of how much fun it used to be before the illusion was destroyed. With skill levels I can choose to make this less likely to happen. Can ADD recognise this? I would be surprised if it could. Although perhaps its more often a case of bad design rather than unsuitable difficulty that causes this problem.

I haven't played Max Payne 1 or 2 and cannot say how well ADD worked for me in those games.

[Scott Miller]

Roger, ADD doesn't promise to cure cancer. And you're right that it comes down to implementation, and even if every game designer began to use ADD, it would take a few years before it would be done well on a regular basis. ADD presents similar problems that we find in creating good AI, and how many games have good AI-driven characters? Not many.

Still, with a little effort, ADD can be pulled off reasonably well in today's games, as was demonstrated in Max Payne, even with its imperfections.

-- "ADD introduces an element of variability at the rules that the player can easily identify as cheating."

Cebrian, there are numerous variable that can be changed that are less obvious to players. Also, as we did with Max, all variables had a min and max level, to ensure that they stay reasonable and less noticeable, such as the damage taken by enemies.

However, it'd be easy to avoid recognizable variability with a little extra work on the designer's part. For example, if the player is proving to be very skilled, then a harder class of enemy is used. This way it doesn't look like enemies that took three shots to kill a few levels ago now take five shots -- the game simply replaces the three-shot bad guy with the five-shot baddie.

Another tactic is to make changes slowly, nor jarringly. And boosting AI intelligence makes enemies harder in a hard-to-recognize way, such as making them better at diving for cover or better coordinated, as Gestalt suggested.

Another thing I'm all for is having a cheat available that overrides ADD, allowing hardcore players to set their own skill level if they prefer.

[Blake Grant]

I agree that ADD would work well in some games, but I still believe that there should be some level of difficulty that is adjustable by the player. I personally don't have as much time to play games as I would like, and as a result, prefer to play them on an easier difficulty so I can get through more of them. As you said, it should be about making the game fun for the player, and not everybody's idea of fun is the same. To one guy its being as challenging as possible without being impossible, to another its being essentially impossible, but pushing on anyways, and to another, its just going through and seeing the story, not wanting to be getting your ass kicked left and right.

I've played both Max Paynes (didn't finish the first one because I reinstalled Windows and didn't realize the savegames were in the My Documents folder!?). Loved #2. My favorite game of last year. In that case, I think ADD was probably working well. I didn't find it overly difficult. Its MUCH easier than the first one, and I blew through the game in about 6 hours. For me, the perfect experience. For many others however, they are annoyed the game is so short. As i said, everyone has a different idea of fun. Letting them decide for themselves how they want to play it is ideal.

One other note. This is alot like the old "should I let them save anywhere, or have checkpoints" issue. Games that only have checkpoints drive me crazy (I'm looking at you, XIII). The developer argues that people quicksaving and quickloading are ruining the experience by making it easier for themselves. So what? Let the player play it how he/she wants to. They put their money down, they want to have fun, not be frustrated. I see the same issue by removing selectable difficulty levels.

Just my 2 cents.

[Greg Findlay]

I think one thing everyone seems to be forgetting is that as players we expect the next level we play to be harder then the last one. So if the ADD system makes it more difficult to kill a guy on the next level, we would probably rack it up to the levels getting harder. This has been done in the past, and although I don't think it's the best solution, it works (using bars and not number to represent an enemies health helps this dillusion).

btw, Diablo and Diablo 2 both have rudementary ADD systems, it just assumes that if you're a higher level, your a better player and adjusts the monster difficulty level. Diablo 2 also adjusts to the number of players in the game. If you played either and didn't notice, then I guess ADD systems can work then eh? ;)

[JP]

In Diablo 2, monster levels for a given area are fixed, but the monsters you see in that area are randomly chosen from a short list. The only thing that varies depending on your level is how much EXP you get for killing those monsters.

Bumping up the monster levels in a game with >1 players has nothing to do with dynamic difficulty in a single player game... it's a simple, linear multiplayer game balancing equation.

Still wondering what it means for something to be "auto-dynamic"!

[Scott Miller]

JP, checking the dictionary, "dynamic" means changing (or able to change), and "auto" modifies this such that it can change by its own means. That said, I am not married to this name, so if you have a better one let's hear it. I often call this feature "self-adjusting difficulty," too. Perhaps I should have gone with SAD. ;-)

[JP]

The industry standard term for it is "Dynamic Difficulty Adjustment", or DDA. I've heard that term used by developers for at least ten years now.

[Tom]

I'm surprised that noone has mentioned Halo yet, or "The Illusion of Intelligence" presentation from the Bungie designers.
http://halo.bungie.org/misc/gdc.2002.haloai/talk.html

Halo is IMHO a very fine example of how to implement difficulty levels in a game: Easy allows the casual audience to complete the game, and Normal already provides a good experience with a few challenging parts. Heroic is pretty tough even for the hardcore FPS player, and then there's Legendary for the extra fun. Halo also proves that "Teaching by death" can work if used wisely, and thanks to the checkpoint based save system, the penalty isn't critical either. Players just get more psyched to clean out those enemies and will try different tactics to succeed, as intended by the designers.

The presentation also has a lot of nice thoughts on what players mainly associate with difficult enemies (mostly hit points), intelligent enemies, and how they tuned the game to have the fun (30 second scope). Halo is generaly agreed to be the best FPS with double emphasis on Shooter: combat is just pure fun and a lot of people I know played through the game several times just because of it.

The concepts outlined in the presentation seem to work, also supported by the ~3 million Xbox copies sold and as far as I know the PC version is a hit as well, even 2 years after the release of the original game.

[Sully]

My vote is for SARS.
Self-adjusting Response System.

[Brad Renfro]

Tweaking enemy health and attack damage fundamentally changes the game. If these variable changes are drastic enough, it warrants different playstyles and strategies, especially in the face of fixed enemy counts/placement and fixed ammoclip counts. It would be unfortunate if Max Payne's ADD caused players to play the game as a timid and careful hard-boiled cop :)

There could be other things you can tweak that will impact gameplay-style even less but still achieve results. Placement of medical kits may be the simplest. Players may be less prone to feel cheated with changing medkits. While a weapon's attack damage is something that is expected to be constant, changing placement of medkits isn't as predictable.

All of this reminds me of a virtual dungeonmaster, working in the background to give the player the best experience as possible. I think eventually, it could expand to cover more concepts than difficulty.