The gameplay of real-time strategy games can be divided into macromanagement and micromanagement. Several researchers have studied automated learning for macromanagement, using a case-based reasoning/reinforcement learning architecture to defeat both static and dynamic opponents. Unlike the previous research, we present the Unit Priority Artificial Intelligence (UPAI). UPAI is a case-based reasoning/reinforcement learning system for learning the micromanagement task of prioritizing which enemy units to attack in different game situations, through unsupervised learning from experience. We discuss different case representations, as well as the exploration vs exploitation aspect of reinforcement learning in UPAI. Our research demonstrates that UPAI can learn to improve its micromanagement decisions, by defeating both static and dynamic opponents in a micromanagement setting.

A CBR/RL system for learning micromanagement in real-time strategy games

: Computer Generated Forces (CGF) are used as units or people in military training and simulation. The use of CGF significantly reduces the time and money required for effective training. Real Time Strategy (RTS) games place players in control of a large force whose goal is to defeat the opponent. The military setting of RTS games makes them an excellent platform for the development and testing of CGF. While significant research has been done into RTS agent development, most of the developed agents are only able to exhibit good tactical behavior. By analyzing prior games played by an opposing agent, an RTS agent could determine the opponent's strengths and weaknesses and develop a strategy which neutralizes the strengths and capitalizes on the weaknesses. It could then execute this strategy in an RTS game. This research develops the Killer Bee Artificial Intelligence (KBAI). KBAI takes a classifier for the RTS domain, uses it to generate an effective counter-strategy, and executes the tactics required for the strategy.

/pdf/developing-an-effective-and-efficient-real-time-strategy-apz84mnsf1.pdf

Developing an Effective and Efficient Real Time Strategy Agent for Use as a Computer Generated Force

We consider real-time planning problems in which some states are unsolvable, i.e., have no path to a goal. Such problems are difficult for real-time planning algorithms such as RTA* in which all states must be solvable.

We identify a property called k-safeness, in which the consequences of a bad choice become apparent within k moves after the choice is made. When k is not too large, this makes it possible to identify unsolvable states in real time.

We provide a modified version of RTA* that is provably complete on all k-safe problems. We derive k-safeness conditions for real-time deterministic versions of the well-known Tire-world and Racetrack domains, and provide experimental results showing that our modified version of RTA* works quite well in these domains.

/pdf/thinking-ahead-in-real-time-search-1ow9rey3wc.pdf

Thinking ahead in real-time search

The general purpose of this research is to investigate the possibilities offered for the use of Artificial Intelligence theory and methods in advanced game environments. The real-time strategy (RTS) game genre is investigated in detail, and an architecture and solutions to some common issues are presented. An RTS AI controlled opponent named KAI is implemented for the TA Spring game engine in order to advance the state of the art in usin AI techniques in games and to gain some insight into the strengths and weaknesses of AI Controlled Player (AI CP) architectures. A goal was to create an AI with behavior that gave the impression of intelligence to the human player, by taking on certain aspects of the style in which human players play the game. Another goal for the benefit of the TA Spring development community was to create an AI which played with sufficient skill to provide experienced players with resistance, without using obvious means of cheating such as getting free resources or military assets. Several common techniques were used, among others Rule-based decision making, path planning and path replanning, influence maps, and a variant of the A* search algorithm was used for searches of various kinds. The AI also has an approach to micromanagement of units that are fighting in combination with influence maps. The AI CP program was repeatedly tested against human players and other AI CP programs in various settings throughout development. The availability of testing by the community but the sometimes sketchy feedback lead to the production of consistent behavior for tester and developer alike in order to progress. One obstacle that was met was that the rule-based approach to combat behavior resulted in high complexity. The architecture of the RTS AI CP is designed to emerge a strategy from separate agents that were situation aware. Both the actions of the enemy and the properties of the environment are taken into account. The overall approach is to strengthen the AI CP through better economic and military decisions. Micromanagement and frequent updates for moving units is an important part of improving military decisions in this architecture. This thesis goes into the topics of RTS strategies, tactics, economic decisions and military decisions and how they may be made by AI in an informed way. Direct attempts at calculation and prediction rather than having the AI learn from experience resulted in behavior that was superior to most AI CPs and many human players without a learning period. However, having support for all of the game types for TA Spring resulted in extra development time. Keywords: computer science information technology RTS real time strategy game artificial intelligence architecture emergent strategy emergence humanlike behavior situation situational aware awareness combat behavior micro micromanagement pathfinder pathfinding path planning replanning influence maps threat DPS iterative algorithm algorithms defense placement terrain analysis attack defense military control artificial intelligence controlled player computer opponent game games gaming environmental awareness autonomous action actions agent hierarchy KAI TA Spring Total Annihilation

Sindre Berg Stene

Papers

Artificial Intelligence Techniques in Real-Time Strategy Games - Architecture and Combat Behavior