The Future of AI in Games

By: Laura Parker, GameSpot AU - Posted on November 11, 2010

Artificial intelligence in games has matured significantly in the past decade. Creating effective AI systems has now become as important for game developers as creating solid gameplay and striking visuals. Studios have begun to assign dedicated programming teams to AI development from the onset of a game's design cycle, spending more time and resources on trying to build varied, capable, and consistent non-player characters (NPCs). More developers are also using advances in AI to help their games stand out in what has already become a very crowded marketplace, spawning a slowly growing discussion in the industry about redefining game genres. Think tanks and roundtables on advances in game AI have become prominent at the annual Game Developers Conference (GDC), while smaller AI-dedicated conferences such as the annual Paris Game AI Conference and developer-run online hubs such as AiGameDev.com are garnering a big industry and community following. While industry awareness about the significance of AI in games continues to grow, GameSpot prompted Matthew Titelbaum from Monolith Games, Remco Straatman from Guerrilla Games, and AiGameDev.com founder Alex J. Champandard to share their thoughts on the future and growth of game AI.

The Halo franchise is recognised as a leader in the field of game AI.

Unlocking new possibilities

While faulty AI is easily recognised, an AI system that is doing its job often goes unnoticed. No one stops halfway through a level to admire the idiosyncrasies displayed by NPCs unless they are doing something completely out of character--the more unremarkable, the better the AI system. While achieving this result is still a priority for game developers, making games with an AI system that stands out for being good is a relatively new concept: few studios want to dedicate costly man-hours to chasing innovation in a highly technical field that, for the most part, is likely to go unnoticed. However, there are some exceptions. In 2007, AiGameDev.com launched its annual game AI awards, nominated and voted by the site's community. The purpose of the awards was to spotlight the games that showed promise in the field of AI, either by trying something different or exhibiting technical proficiency. In 2009, the Best Combat AI and the overall Best Game AI awards were won by the same studio--Guerrilla Games for Killzone 2. Remco Straatman, lead AI programmer at Guerrilla, says a lot has changed in game AI in the last five to 10 years, with more developers trading low-level scripting for more advanced NPC decision systems.

"In general, I think game AI has gone from the stage where it was an achievement if it did not stand out negatively to the point where AI in most big games is solid, and some titles are using innovative new ideas," Straatman says. "More development teams have also moved from simple state machines to behaviour trees and using planners in NPC AI systems describing knowledge of the world around the NPCs have improved with better knowledge for navigation over changing terrain, and more knowledge about strategic properties of the world such as cover. I also think advances in animation systems with better ways to combine various animations and physics have become available, which now allows for more realistic movement and responses to being hit [in combat AI]. Most of these systems were not around 10 years ago or simply could not run on the hardware available."

Creating a solid game AI system involves successfully networking smaller systems together. For example, a system that deals with the problem-solving capabilities of individual NPCs goes hand in hand with a system that makes sense of the gameworld and its parameters and helps NPCs make relevant decisions. Thankfully, developers don't have to build these systems from scratch: they use specific planners that generate increasingly complex networks.

"At the moment [Guerrilla Games] is using a specific type of planner for our NPCs called Hierarchical Task Network (HTN)," Straatman says. "This is capable of generating more complex plans than what we had before Killzone 2. We also keep on improving things like the CPU performance, which means we can support more NPCs in Killzone 3 than we could in Killzone 2. The terrain-reasoning systems we generate have also evolved over our various titles. We are now able to deal with much more dynamic terrain (like obstacles moving around or changing shape) than ever before. Our data on where there is cover has also become more detailed, something that allows NPCs to deal with more complex environments such as multistory buildings, etc."

Killzone 2's lead AI programmer Remco Straatman believes the industry is still struggling to make NPCs as human as possible.

Back when Straatman and Guerrilla began work on Killzone and Shellshock, the team’s goal was to make the AI system as capable of making its own decisions as possible, realising this would make things all the more fun for players. However, doing this in a consistent way proved to be a lot more work than the team anticipated, particularly when dealing with combat AI. While the goal of normal AI is to emulate the real-life behaviour of a particular nature (for example, doctor, civilian, or shopkeeper), combat AI works very differently. Firstly, its main objective is to be as entertaining as possible. In some cases this means being efficient at killing players; in other cases, it's more about making intentional mistakes and "overacting" by way of signalling to players what is about to happen.

"Where normal AI tries to emulate an expert medical specialist or world champion chess player, game combat AI is more like emulating an actor," Straatman says. "At the end of Killzone 2 we found ourselves looking at the NPCs doing things that we did not expect, and this surprised us positively. Reviews and forum feedback confirmed we had at least partly achieved the vision we had so many years back, and people playing the game recognised and appreciated it."

One of Killzone 2's most commended features in the field of AI was the game's skirmish mode. Because this mode is more team-based and tactical than the single-player campaign, the AI bots in this part of the game need to do more than simply run around and kill one another. Guerrilla based the skirmish AI in Killzone 2 on the real-time strategy model, building two levels of AI on each individual bot. The first is a commander AI, which controls overall strategic decisions; the second is a squad AI, which translates the commander AI's orders into orders for the individual bots. The team then taught the bots how to use the in-game badges as part of the order given to them by the squad. For example, if an engineer bot is ordered to defend an area, he will first build a turret at a tactical position before starting to patrol. While some might argue that AI bots no longer play as important a role in multiplayer games--given that most gamers now play online--Straatman says bots improve gameplay and give players a chance to test out multiplayer strategies before going up against other human players.

"They give people a testing ground for real multiplayer--getting to know the maps and the game modes in a game against human players can be too much to start with."

According to Straatman, the area that needs most improvement in the game AI field is buddy AI. Because buddy AI systems often have contradictory constraints, getting this system right is often a big challenge: the buddies should be visible and close to the player but not get in his line of fire; they should stay close and respond to the player movement but not move around all the time; and so on. Buddy AI is also much closer in view to players than enemy AI, making any errors easier to spot.

"Enemy NPCs know what other NPCs of the same faction are going to do because they are all computer-controlled and can tell each other what they will do next. However, players are much harder to predict--if you would look at movement patterns of players, you will see they are quite strange at times. This is made worse by the fact that player turn rates, movement speeds, and acceleration are very high. The last point is the expectation of the player: enemies are only supposed to shoot at you, whereas buddies are supposed to fight and interact with you in a sensible way. We are working hard to make the buddies work better, because we feel that they can add a lot to the player experience when done right."

The AI director in the Left 4 Dead games is an example of how developers can use AI to reach beyond traditional individual NPC behaviour.

Straatman believes the struggle to make NPCs as human as possible is still very much at the top of the list for many AI programmers, with the future set to change the way we think about in-game interaction.

"The ideal is always to immerse the player in the game: the NPCs should feel like they are living and breathing creatures, and this illusion should not be spoiled anywhere. Within the relatively limited interaction you have in a game, it may be achievable to make the distinction very small. I think human behaviour is so interesting, and yet subtle interactions such as conversations are still out of reach of autonomous AI; games rely on clever scripting or cutscenes to get that across. If we as a field will master these types of interactions, more parts of the game can be interactive, and possibly whole new game genres may become feasible."

"I think this will make games more approachable and immersive. If we are able to maintain the immersion by having realistic behaviour in the interactive parts of the game, you will get a seamless experience from cutscenes to combat. I also think we are ready to use AI for more than just individual NPCs--the director system in Left 4 Dead is one interesting first step in that direction. We probably will see more combinations of AI systems that before were limited to one type of game: RTS games will have unit AI that will come closer to what you now see in first-person shooter games. MMOs could also start using more elaborate AI, potentially even to command hordes of NPCs. I hope we will see some brave studios try to create these new systems that are now becoming possible." Click on the Next Page link to see the rest of the feature!

Getting the most out of multiplayer

Alex J. Champandard, the brains behind AiGameDev.com, has his finger in every game AI pie in town. Following his work as senior AI programmer for Rockstar Games developing RAGE (Rockstar Advances Game Engine), Champandard moved on to contract for Guerrilla Games’ Killzone 2, where he developed the strategic AI for the multiplayer bots before starting up AiGameDev.com, the online hub of the game AI community. In-between running the site, Champandard continues his AI consulting work with studios like 2K Czech and Crytek, as well as co-organising the aforementioned Paris Game AI Conference.

Citing Left 4 Dead, the Killzone games, the Halo franchise, Grand Theft Auto, Assassin's Creed, and Far Cry 2 as exemplary candidates in the game AI field, Champandard says the last decade has brought a deeper understanding on the part of game developers about how to best fix age-old AI problems.

"We've realised that just borrowing techniques from traditional artificial intelligence doesn't work, and it requires a significant know-how to get something fun and believable out of those algorithms," Champandard says. "A large part of figuring this out has been to think about the impact on the player and the in-game results. Thinking about creating NPCs as 'computational behaviour' instead of 'game AI' sums it up perfectly: it's not about the intelligence under the hood, it's about what the behaviour turns out like: adaptive, credible, entertaining."

Champandard lists the Assassin's Creed franchise as exemplary candidates in the game AI field.

This recent progress means game AI is no longer the weakest link in game development, as was the case 10 years ago. While some studios see AI as nothing more than a necessity, others are trying to innovate in the field. Champandard says the best example of this is AI Directors in sandbox games.

"The entire concept of a sandbox game is impossible without AI. The idea that you can do anything in the world and its inhabitants will react to you would not be possible without AI to power those NPCs. I think the industry has already discovered that you need AI Directors to make sandbox games really fun. Otherwise you may end up with situations that emerge out of the simulation that are just plain boring. Using AI programs that 'direct' the game helps make sure you're seeing the best of what the game has to offer, as planned by the designer. This kind of technology opens the doors to new types of games, where the story is generated as you play. The progress is slow, however, which means it may take a few years before this becomes mainstream."

Champandard believes the future of game AI lies in more solid multiplayer experiences, as AI systems slowly improve.

"I see more and more games providing bots as a way to improve the multiplayer experience. Playing online can be very unpredictable if you're not with your friends, and statistically players tend to prefer playing against bots than random people. On the technology side, AI is now helping out in all the other disciplines of game development, slowly revolutionising software engineering, and improving techniques and algorithms that are applied. It's an exciting time to be in the field."

Expanding the AI domain

Like Straatman, Monolith Games' Matthew Titelbaum believes future AI systems will become more immersive and allow for a new kind of gaming experience. However, quoting his experience working on the F.E.A.R. franchise, Titelbaum does not believe that giving NPCs more human-like behaviour is the way to achieve this. To him, it is not humanity that will advance game AI, but rather, more unpredictable behaviour.

"Most games take the player on a journey from point A to point B. Along that journey, the game presents the player with a series of puzzles to solve. Some may be navigational (i.e. how do I get across that gap?), some may be organisational (i.e. what order should I build things in?), but most of them rely on some sort of other characters' intent on destroying or rescuing the player. Without a series of interesting interactions with these characters, the journey can become fairly tedious," Titelbaum says. "It used to be acceptable for AI to have perfect knowledge of the environment. Now we have stimulus and sensor systems to more accurately model what an AI can conceivably know about. We’re also using planners, hierarchical state machines, and behaviour trees to map this out. The bar keeps getting raised as new concepts from the academic world steadily find their way into commercial releases."

F.E.A.R. 2: Project Origin made use of Monolith's goal-based action planner AI, which helped produce context-sensitive behaviours in the game.

Like Guerrilla, Monolith employs several subsystems for both normal and combat AI. The common link between the two is the goal-based action planner, which shipped in F.E.A.R. and F.E.A.R. 2: Project Origin, which helped produce the context-sensitive behaviours in the games. The first step came from the team’s level designers, who annotated each level with information that the AI could understand. For example, marking a table as a good spot to take cover and noting that the table must be flipped over before being used. After this marking is completed, it is up to the AI to properly interpret the annotations and integrate its own context: when the AI decides to go for cover and it chooses that table, the annotation tells it to flip it over. These systems work together to give the AI more knowledge about the world, and give players the impression that NPCs see their movements and correctly judge their intentions.

So if the goal of NPCs is to fool players into believing they think and act just like other players, why does Titelbaum think that striving to make NPCs correctly copy human behaviour in future AI systems is not a good idea?

"In general, I think human-like AI in traditional narrative player versus environment (PvE) games doesn't really make a lot of sense. The AI characters are there to play certain roles and be part of the puzzles along the way. They can be challenging, they can be unpredictable, they can even adapt to the player, but, above all, they have to be fun to solve. I don't see humanness and fun being directly correlated. I suspect Miyamoto-san doesn't spend a whole lot of time thinking how to make Goombas more human-like.

"That said, I think there are some game genres and styles where it may make sense to have more human-like AI. In a purely player versus player (PvP) experience, or in the real-time strategy genre where all player and non-player entities have the same set of choices, I can totally see the desire to have AI that behaves like humans. But, even then, do we want human-like AI or do we want challenging, engaging, and unpredictable AI?"

Titelbaum’s answer is to expand the domain to which AI is applied in games. For example, if AI can stand in for humans playing the game, this means developers should be able to find ways to allow AI to stand in for humans making the game.

"A good portion of level designers' time on F.E.A.R. was spent annotating all the places AI could hide; it's possible, using terrain analysis techniques, that much of that nitty-gritty work can be automated. Using automation removes the need to redo hand-done work when there are major changes to the level layout. The more parts of level creation we can automate, the more time level designers have to create and polish rich experiences."

Games like Far Cry 2 push the boundaries of game AI.

Titelbaum believes that as games mature and players' objectives begin to push past the 'kill everything in sight' trajectory, developers will create AI that is able to engage in richer, more immersive behaviour that will allow NPCs to express motivation and emotion not just through dialogue, but also through interactions with the player, other NPCs, and the environment.

"I don't think it's about AI acting just like humans. Possible or impossible, I just don't think that's really the desired outcome. Should AI engineers really be working on human-like behaviours and button-mashing behaviours? Is that really what players want to play against?

"I think it's more about AI always acting in plausible ways, given the role within the game they fulfil. There can be a lot of variation in what is plausible at any given time, and this is what makes for compelling interactions. They may have the capability to do what a human could do, but they don't necessarily need to do it when and how a human would." Where do you think the future of game AI is headed? Let us know by leaving your comments below!