Auto-dynamic difficulty

There is a zone beyond that which is known to game designers. It is a place as vast as space and as timeless as infinity. It is the middle ground between boredom and frustration, between easy and hard, and it lies between the tips of our fingers, and the sparks within our CPUs. This is the dimension of pleasure. It is an area which we call... THE FUN ZONE.

(Forgive me, Rod!)

Okay, that out of the way, I'm going to start off with an actual, complete email I sent to the project leader of Max Payne a few months before the initial release of that game in mid-2001, almost three years ago:

From: "Scott Miller"
To: "....... ......."
Sent: Thursday, April 12, 2001 1:54 PM
Subject: Re: Max Payne Build 125

Here's the reasoning behind not having an easy skill level: We do not want players to finish the game and not have a satisfying feeling of accomplishment. Too many players will pick the easy mode just to get through the game, and end up finishing and think, "Well, that was nothin'! Where were the enemies!? Where's the challenge?!" In other words, if we give players a chance to shoot their own foot, some will.

Diablo 2 obviously didn't suffer from not having a skill level choice -- it's one of the three best selling games ever, with 2M and still selling.

Also, my view is that as developers, it should be *our job* to properly play balance the game, not the player's choice. In other words, skill levels are an easy way out for developers too lazy or incompetent to properly play balance their own games. (Okay, a bit harsh, but I'm saying this as someone who used to be a strong proponent of skill levels, too, so I was part of that incompetent developer group! *grin*)

Finally, a skill level screen is just another screen getting in the way of enjoying the game itself, like those boring as hell logo screens that I hope to g.o.d. we can get rid of in all future games we release!

Anyway, the perfect game will adjust its difficulty based on the player's personal ability. This was done in arcade games going back to Atari's Xevious in 1982. It should be really easy to do: Simply have a few variables that rate the player's ability, and the player's rating (completely internal to the game) determines the damage that both the player's weapon delivers, and the enemies' weapons deliver against the player. You do not want to adjust the health benefit of pain killers because that's something the player might notice being adjusted as the game goes along because it's represented by a visible bar.

A few ideas for variables that determine the damage ratings of weapons might include:

o Max's average health (i.e. if it averages around 25%, then the weapon power variables need to be adjusted to favor Max slightly, but if Max's health averages 80% or higher, then up the difficulty).

o Kills made per level, vs number of possible kills per level. A good player will leave few left standing, if any at all.

o Number of times Max dies per level -- this is a good indication of how good the player is.

o Note: I wouldn't keep track of shots fired or number of saved games--those can be misleading either way.

Finally, never make a change during a level, only between levels.

At the end of the game, via a cheat code, you could plot a graph of the ups and downs as the player played the game, which would be a very interesting graph to compare with other players!

Scott Miller
3D Realms Entertainment
www.3DRealms.com

As can be gathered in this email, it's my opinion that games should only rarely allow players to set their own difficulty level. Afterall, it should be the designer's job is to insure the proper play experience, not the player's. The idea behind auto-dynamic difficulty (ADD) is to keep the player in the sweet spot, where the game remains challenging, but never impossible. A perfect game experience is one that constantly tests the player's skill, without being so hard they want to give up. Completing such a game gives a player an elated feeling of accomplishment, because they know they passed a tough test of their ability.

One of the key challenges developers must overcome in broadening the appeal of our games is making our games less hardcore in nature. We need to improve in many areas, like interface, content, and adding emotional involvement for story-based games. ADD should become one of our best tactics to help us achieve this important goal.

One of the most common ways games sabotage their potential to appeal to larger numbers of players is by being too difficult (or too easy, but that's much less common). Practically everyone designing games nowadays is a hardcore player with elite skills. It's therefore easy for game designers to misjudge the difficulty of their own games, making them too hard for average or new players.

Sequels, especially, are a breeding ground for ramped up game difficulty. The natural tendency with sequels is to add more features, leading to more controls. And higher game difficulty. Also, with sequels designers naturally assume that players have all the skills learned from the previous game, and so the challenges are made tougher for new installments. Not exactly an inviting situation for anyone but the hardcore gamer.

ADD provides the near-perfect solution for most problems dealing with game difficulty. At least if implemented well, but that's the case with any facet of design. There are two primary issues with implementing ADD:

[1] The first issue when implementing ADD is to make it invisible to players. It should not be obvious when the game is self-adjusting its difficulty level, as this can hurt the game's immersion.

[2] The other issue is to make it so that's it's hard for player's to manipulate the system, taking advantage of it. For example, a player might play very poorly for the level leading up to the boss, just to make the boss a breeze to beat. While clever players will always be able to manipulate games with ADD, there are ways to make manipulation more trouble than it's worth. And, the bottom-line is that players who do this are only cheating themselves, so perhaps this is not too great of an issue.

One last point that developers should consider: If a player completes your game, they are much more likely to buzz about, spreading the word that it was a great game. We all love to brag about feats of accomplishment, and beating a game, for a gamer, is near the top of the list. A game with a well implemented ADD system is more likely to allow players of wide ranging abilities to complete your game.

I think Max Payne benefited greatly from its implementation of ADD. And I think this should become a standard feature for most games. At least, if broadening our customer base is of any concern.

Comments

[Scott Miller]

Yup, over 3000 hits today so far! www.ActionTrip.com also covered this site.

[ragmana]

ADD made my day in Max Payne. It is probably my single favorite advancement in games in the last five years.

[Pedro Faria]

Greg Findlay:
"The health and damage that enemies do in Diablo 2 change based on the level of the character in the game, not just the number."

This is simply not true. The monster's health are fixed within certain ranges. In D2 they DO increase with the number of players in the game though, but that is it.

"The "boss modifiers" also change based on character levels."

Not true either. They are fixed or randomly picked from a pool of possible ones.

"The sets of monsters that spawn in also changes based on level. D2 does more then that too but you get the idea :). "

Not true either. Each area in the game has a certain set of monsters form which the game pick randomly (plus some areas having fixed monsters on top of that). The selection is not affected by player level at all.

All the above holds true for both Diablo 1 and Diablo 2. And yes, I consider myself to have good knowledge how the games work both, including the inner workings of them. There is almost nothing in the game of Diablo that is affected by the character level when it comes to difficulty or how the enemies are generated or behave. The few things I can think of is that items sold in town get better when your character raises in level. That is, as a low level character, you will be shown daggers with low level mods but as a high level character you will be shown might swords with super mods. There are a few other similar effects, None have anything to do about difficulty though.

The way a player can regulate the difficulty in those games are by choosing in what area or dungeon to play. One can pick a hard one (with high level monsters and such) or an easy one, (with low level monsters) and that is basically it. In addition, I would not say that Diablo have sold way due to NOT having a difficulty setting (it really don't need it as you can decide were to play and thus adjust your difficulty) or that it would have sold better/worse if it had had one. Actually, to be completely true it DO have three difficulty settings, normal, nightmare and hell difficulty, although for most players it is not really a true difficulty setting since they are meant to play once you manage to play the whole previous difficulty at ease since your character level is too high. They need them due to the fact that the game do NOT adjust itself (thank good).

Speaking of the main subject, I would have to say that I hate it when there is no difficulty setting in a game. I think one might perhaps need to distinguish two game types. The ones you play through once or twice and "finish" and the ones you play over and over and over. Difficulty settings is far more important in the last category. If for no other reason than for me wanting different game experience at different times. If I play a RTS or FPS type of game, at times I want to just fool around, and having an easy time, crushing the computer or perhaps mostly minding my own business and not having to worry about the computer players much. Other times I want to really test my skills and just see how long I can handle it. I want different difficulties at different times. I even want to be able to adjust individual aspects of the game individually (not common) instead of just an "easy, medium, hard" setting. One of my all time favourite games was Civilization. I would have loved to set up the difficulty of the game for the city handling part of the game, but lower it for the military/combat part of the game versus the computer. That was not possible.

For the games that you "finish", I could possibly see a reason to have it, yet, I ultimately things that the player is the one who can best judge how they get the most out of the game and if they want an easy ride or a hard one. Removing that option and thinking the players are to stupid to handle it and that the game designers know the best is just insulting your customers.

[ADoomedMarine]

Nice article Scott.

I agree that ADD was a very nice implementation into the Max Payne games and I really hope to see it used alot more in future games.

I agree with you, the difficulty select screen is only a blockade between you and gameplay, you should be able to just jump straight in :)

[Dan M]

Sounds like Unreal Tournament. It had auto adjusting bots. If you set the game to auto adjust and got on a roll, then the next thing you know you playing on the godlike setting.

The problem with ADD is the effect it would have on linear (story driven) single player only games. If there are no levels, then once you've beaten the game, there is a much more limited replay value for the gamer. Whereas if they beat it on "normal", you may be tempted to try to beat it again on "nightmare".

[Scott Miller]

Dan, this is easily solved by having more difficult levels open up after you've beaten the game, as was done in Max Payne. Also, for hardcore players who want to bypass the system and adjust the difficulty level as they prefer, cheat codes can handle this.

So far, I've not seen a single concern verses ADD that does not have a sensible solution (and this includes the many messages in the Slashdot comments section where this topic was covered). The mindset of some people is to try to find faults with this sort of thing, rather than think in terms of solutions. These people would not make good game developers.

[Dan M]

I agree with you that elements of ADD should be used in as many games as possible as long as it adds to the experience. I wouldn't want it to replace the skill level screen. Ultimately the gamer should make the choice of what type of experience they will have no matter how lame it may be. If you look at the trend of some of the latest sports games, they now give you the ability to adjust almost all aspects of the game.

What I'd like to see is that the gamer still picks a level and then ADD kicks in to adjust the gameplay defined within the selected level. So if "easy" is picked on an FPS then the usual increased size healths/more ammo are available to the gamer. Yet unseen to the player if they're playing well then the bot's awareness, reaction time and accuracy slowly increase to a maximum point which would equal the bottom on "normal".

If say you were playing a RTS game, it would have a multilayered approach. If the gamer is building slowly, then the computer players may slow down its building to stay within a given range of the player based on the selected level. Then after each game a log file for the players profile should be updated with mini analysis of the game. That way the next time it plays you it can now "remember" that you always build tanks but never build rocket launchers.

What's the difference between having a gamer go to the option screen to set a level and opening the console and typing a cheat code in?

[Sha]

While ADD is a great idea, I would already be satisfied if you could change the difficulty setting during the game. Most of the FPS I've played don't let you do this, meaning you have to start all over again if the difficulty setting you chose proves too hard mid-game.

[Anon]

I posted much of this at Slashdot just now, but I'll repeat it here:

Good: The idea of dynamic situations. That a level can be different on multiple plays. This isn't tied to the game automatically getting easier or harder, but to it being different.

Bad:
Player abuse. Doing badly on purpose in an unimportant area in order to make the game easier in the important places. Worse, the possiblity that gaming the difficulty system may be required for a player to win the game.

The possibility of "Mario Kart 64 syndrome," that the player is penalized for doing well, perhaps enough so that victory is impossible if you play as well as you can as long as you can.

The idea that the game is a "test" of the player's skill, something to be overcome, is lost. Saying you beat a game carries less meaning when it could be because the game's been coddling you. Furthermore, it's a game, not a movie. I've been playing a fair amount of Viewtiful Joe lately, and although the difficulty is brutal for new players I've still stuck with it. What I'm saying is, a well-designed game should be hard, but allow the player to realize it's possible to proceed if he just does a little better. The player wants to improve his skills, not have the game give him a pass.

However, it may be possible to design an auto-adjustment system that satisfies my concerns. I'd say the player should be notified of the level of adjustment at the end of the game, where it could make for a kind of final score.

[Nathan Cushman]

I'm not sure if anyone reads these posts anymore, but I have important things to say.

First, I'd like to say that Scott Miller's original e-mail was brilliant, and his suggestions should be the standard for story driven games like Max Payne. The player should NOT know that ADD is working in the background, at least not until the end of the game. Scores at the end of levels would take away from the immersion.

I'm not completely sure about the idea that ADD should only work between levels. It is then possible for someone to do very well on an easy level, but go insane when the already difficult next level has it's difficulty ramped up. Ideally this shouldn't happen, but we've all seen that difficulty in games isn't strictly a linear progression, rather it is a wavy upward trend.

The problem with changing difficulty mid-level is that players are far more likely to notice, so some work would have to be done to make it less noticible. The first solution that comes to mind is to add random elements which are given a higher probabilty of aiding the player if they are stuck on a level. This way the player will just feel lucky that they got extra painkillers, a better gun, or that they caught the guard sleeping.

Last, for all those who want to know exactly how well they did, and are seeking a greater challenge, I'll expand on Miller's graph idea. Rather than needing a cheat code to see the graph, I would suggest that a new option opens up on the main menu after you beat the game. This would let you see all the numbers recorded for each level by the ADD system (on various graphs), and allow you to pick any level to replay it, and try to improve your ratings, then you could try the next level on the new higher difficulty.

As Scott Miller said, "So far, I've not seen a single concern verses ADD that does not have a sensible solution."

[Dustin Westphal]

"The first solution that comes to mind is to add random elements which are given a higher probabilty of aiding the player if they are stuck on a level. This way the player will just feel lucky that they got extra painkillers, a better gun, or that they caught the guard sleeping."

That would be a perfect way of handling ADD without changing damage levels or enemy intelligence. Having a guard that is caught sleeping if the player is performing poorly would add variability to the gameplay without having the player feel that the game is dumbing itself down. Increasing ammo drop rates would be another way. If a player is having a hard time killing enemies, increase the number of grenades that each one drops. Once the kill count rises, start lowering the number of grenades available. If you notice the player relying too much on weapon you could also vary the amount of ammo available for that gun.

Changing the amount of damage or health, or the level of enemy intelligence should be a last ditch option, as it might be too noticeable by the player.

[Alex Saveliev]

I want to bring up the way ADD is implemented in Half-Life: if you get hurt badly you’re starting to crush wooden boxes obsessively to find extra painkillers, at least that's what I did :)
It takes time, but you are able to restore your health and move on to the next area. If you are skilled player you won’t spend your time smashing all boxes around. So we have a kind of auto-adaptation here without too much extra coding.

BTW, the boxes contain not only painkillers, but extra ammo too.

[Tim Wallace]

An old thread but if someone reads this they may get something out of what I will briefly say. Have the ADD system that changes frequency of beneficial item drops, adjusts accuracy/damage of enemies, tweaks the AI. Have a skill level as well. Make EASY only provide you with tweaks that make the game easier, NORMAL adjust the game to be easier/harder, and HARD to only make the game harder through tweaks. I would also encourage game designers to make it a bit more random than sweeping changes to all enemies though, the game should play as if the computer is restricted to the same rules as the player

[Anon]

Also for ADD some games will likely lend themselves to adjusting monster ai/difficulty the first time the monster/npc is introduced. For example lets take doom3. If the player is doing well the first time they encounter the mancubus then the mancubus could prove to have extra hp/dmg. All mancubuses after that would keep the same stats. Changes would be moderate to prevent players falling into too much trouble but would prevent players from noticing ADD since the monster would only change previous to the player's first encounter with it.

[Cool Guy]

I say just skip ADD in games and stay with the conventional easy, medium, hard mode. ADD doesn't work very well to begin with. Even in the conventional modes the AI causes another problem in itself. Adding ADD doesn't necessarily improve the AI of the game.

[Saturday Night Live DVD]

>Saturday Night Live DVD is the Emmy Award-winning late-night comedy showcase that has launched the careers of many of the brightest performers in entertainment. It blends the best of the classic and the new in the two largest entertainment genres - comedy and music. Get http://www.saturdaynightlivedvd.com/ >Saturday Night Live DVD at http://www.saturdaynightlivedvd.com/