Short Term Decision Making with Fuzzy Logic And Long Term Decision Making with Neural Networks In Real-Time Strategy Games

Abstract

Real-time strategy is one of the most important game genre from the beginning of the computer games history and one of the biggest inner market in computer games market, in spite of that fact there are only a few strategy games have successful AIs. This paper will try to improve real-time strategy games’ AI by improving long term decisions such as; resource management, determining correct buildings to build, achieving correct upgrades/technologies with neural networks as well as to short term decisions such as; creating correct troops against opponents’ troops, determination of attacking or fleeing, attacking to correct enemy troops. Also this paper will try to give examples from strategy games such as Command and Conquer: Tiberium Wars, Age of Empires.

Key Words: Artificial intelligence in real-time strategy games, fuzzy logic, neural networks, short term decision making in strategy games, long term decision making in strategy games.

1.     Definition of Fuzzy Logic

The best definition of fuzzy logic is given by its inventor Lotfi Zadeh; “Fuzzy logic means of representing problems to computers in a way akin to the way human solve them and the essence of fuzzy logic is that everything is a matter of degree.”

The meaning of solving problems with computers akin to the way human solve can easily be explained with a simple example from a basketball game; if a player wants to guard another player firstly he should consider how tall he is and how his playing skills are. Simply if the player that he wants to guard is tall and plays very slow relative to him then he will use his instinct to determine to consider if he should guard that player as there is an uncertainty for him. In this example the important point is the properties are relative to the player and there is a degree for the height and playing skill for the rival player. Fuzzy logic provides a deterministic way for this uncertain situation.

There are some steps to process the fuzzy logic (Figure-1). These steps are; firstly fuzzification where crisp inputs get converted to fuzzy inputs secondly these inputs get processed with fuzzy rules to create fuzzy output and lastly defuzzification which results with degree of result as in fuzzy logic there can be more than one result with different degrees.

Figure 1 – Fuzzy Process Steps (David M. Bourg P.192)

To exemplify the fuzzy procee steps, the previous basketball game situation could be used. As mentioned in the example the rival player is tall with 1.87 meters which is quite tall relative to our player and can dripple with 3 m/s which is slow relative to our player. Addition to these data some rules are needed to consider which are called fuzzy rules such as;

• if player is short but not  fast then guard,
• if player is fast but not short then don’t guard
• If player is tall then don’t guard
• If player is average tall and average fast guard

Figure 2 – how tall

Figure 3- how fast

According to the rules and the input data an output will be created by fuzzy system such as; the degree for guard is 0.7, degree for sometimes guard is 0.4 and never guard is 0.2.

Figure 4-output fuzzy sets

On the last step, defuzzication, is using for creating a crisp output which is a number which may determine the energy that we should use to guard the player during game. The centre of mass is a common method to create the output. On this phase the weights to calculate the mean point is totally depends on the implementation. On this application it is considered to give high weight to guard or not guard but low weight given to sometimes guard.  (David M. Bourg, 2004)

Figure 5- fuzzy output (David M. Bourg P.204)

Output = [0.7 * (-10) + 0.4 * 1 + 0.2 * 10] / (0.7 + 0.4 + 0.2) ≈ -3.5

As a result fuzzy logic is using under uncertainty to make a decision and to find out the degree of decision. The problem of fuzzy logic is as the number of inputs increase the number of rules increase exponential.

2.     Definition of Artificial Neural Networks

Although there is no agreed definition the most common definition is “it involves a network of simple processing elements (neurons), which can exhibit complex global behaviour, determined by the connections between the processing elements and element parameters” (Wikipedia, Artificial neural network).

The structure of ANN (Artificial Neural Networks) is same with human’s neural network’s structure as in human neural system there are receptors to collect the data named dendrites, a processing unit named cell body and an output unit named axon. Similar to this structure in ANN there is an input layer, a processing unit which sums the inputs after multiplying them with weights and an output layer which gives a result for a neuron (Figure 6).

Figure 6- structure of a simple neuron (Mat Buckland, 2002, p.241)

As seen in the figure the output will depend on the inputs and the weights as well. This is the main point of the ANN as if we can find the correct weights for the inputs than we can create a correct result for ever situation. On reverse order if we know  have the inputs and if we know the results for every input pair than we can calculate the weights therefore after finding weights if we give input values to our neural network it can calculate the correct output for certain input pairs. This method which is called back propagation is a supervised learning. The problems of that method are; training sets for feeding the ANN may not have all the possibilities or these sets may contain novel data (Dr Emma Hart, 2005).

Although the logic sounds pretty easy for ANNs with supervised learning, there is a mathematical background which makes this logic work for computers. On this part of this paper no mathematical background will be explained since it is not the main goal of this paper. (To get more information about ANNs please check further reading)

Figure 7- structure of a neural network with one input layer, one hidden layer and one output layer (Mat Buckland, 2002, p.242)

As seen in the figure above there is a hidden layer which is necessary to make complex calculations. A neural network without hidden layer can make logical calculations such as ‘AND’ and ‘OR’ but cannot calculate XOR operation. This means that for complex calculations the hidden layer is obviously necessary. The number of neurons in the hidden layer is up to implementation. Increasing number of hidden layers increases the precision and complexity of the calculations which results with accurate calculations with using more CPU time and more memory that is the trade off. To find the best number of hidden layers the only way is experimenting different values, however, there are different estimations for that problem such as square root of number of input layers multiply by number of output layer.

In brief, ANNs are using for complicated or imprecise data and also for recognising the patterns (Christos Stergiou, NEURAL NETWORKS). Recognition of patterns requires very complicated methods and data structures with classic methods; however ANNs are very simple structures which make the implementation easier as ANNs are self-organising structures as well. The cons of ANNs with back propagation supervised learning are; high consumption of resources, choosing correct propagation sets, choosing correct number of hidden layers and hard debugging.

3.     Real Time Strategy Games (RTS Games)

A RTS game is a strategic game which runs in real time distinguish it from turn based strategy games. Although the high number of RTS games there are only a few scenarios are used for these games. The most popular scenario starts with a couple of productive units such as villagers, and a few resources. The player should use the villagers to collect resources and create new buildings or units who can be soldiers or villagers. The buildings usually enable new unit tapes. To give an example the player can create knights by using stable and to build a stable a barracks should have been built. On the other hand to upgrade the unit strength usually players should achieve new technologies which are usually much costly than unit production. As seen in this scenario usually in RTS games there is a hierarchical order which the player should follow to defeat the enemy.

Although there is a static hierarchical order for creation of units or achievement of technologies, always complex algorithms running for RTS games AI since the number of probabilities and parameters are very high for making decisions. Moreover the only problem in RTS games is not decision making but also path finding or this kind of AI algorithms but in this paper only decision making will be discussed.

In the history of RTS games, game AIs usually statically programmed like finite state machines for decision making. To give an example AOE-2 (Age of Empires II) has a rule based AI with priority queues which means nearly static development for NPC (non player character). If the player can find a way to defeat the NPC than the player will never lose again as the NPC cannot evolve itself and cannot find a way to overcome the bottlenecks of its management strategy. Besides these long term decision failures in AOE-2 consideration mechanism for choosing the correct the enemy to attack is not working at all as the troops always choose and attack an enemy until they die. This makes the game AI dummy since if the human player creates a trap with a wall such as builds a wall around an archer, all the knights try to attack this archer but they are not able to attack because of the wall but they lose lots of energy or time. Even if the AI programmer can find out this bug and tries to prevent this situation, he has to write lots of scripts as there is no rule to generalise in this situation. That makes the AI very complicated.

Thankfully in new generation games such as Command Conqueror Tiberium Wars, the troops behave reasonable as they can choose the correct troop to attack according to its health. To give an example all the troops choose the enemy with lowest health in their range but this approach still involves some problems. Moreover this game has obviously problem while making long term decisions because NPC obviously cheats by having lots of recourses and by producing them very fast. These approaches make strategy games very boring for game players as if the game player can defeat NPC for once then computer nearly has no chance to win again. That’s because of their static structure and behaviours.

To find alternative solutions for these problems in this paper the decisions will be divided into two parts according to their importance. If the decision is unit wise such as choosing the correct troop in the range to attack will be called as short term decision. On the other hand the decisions, like choosing correct technologies to achieve, which are not individual decision for a troop but about a certain group of all troops in the game or the decisions which will affect the destiny of the game for long term will be called long term decisions.

Not only the importance but also the complexity of the decisions is another reason for this differentiation. The short term decisions such as considering the correct troop to build is a less complex decision compare to resource management and it is only uncertain which means can be found out by analysing some other parameters.  However the resource management cannot be done by analysing some parameters because of the huge amount of the parameters and complex relations between these parameters.

a.      Short Term Decisions in Real Time Strategy Games

As mentioned before, short term decision means instantaneous decisions made by individual troops or the controlling mechanism such as; decision of attacking or fleeing or creating the correct troops according to immediate data. Although these kind of short term decisions are unpredictable, they can be made by deterministic calculations such as Fuzzy Logic which provides a high resolution for output with crisp input data. In the next parts of this paper, these short term decisions will be discussed in terms of their input data, output data and the way to make these decisions with the explanations.

The sections below based on a hypothetical RTS game scenario. In this hypothetical RTS game the units have the properties showed on the list below with explanations.

Health	The health of the troop, in fuzzy set it will be normalized range between [0,100], 100 represents the healthiest and 0 is death.
Armour	Unlike some of the RTS games, in this example there is only one type of armour for the troop which reduces the attack power of enemy against the troop.
Attack Power	Attack power of troop against the enemy.
Attack Speed	Attack speed is the variable defines how many time can the troop attack to the enemy in a constant time interval
Movement Speed	Differs from attack speed, this is the parameter to define how fast the troop move or escape, it is constant for every type of the troops.

And the parameters are found by some calculations using the variable above. The main reason for doing that is having more realistic information. To give an example, let’s assume there is two kinds of troops first one with attack power of 50 and attack speed of 1, the other troop has the attack power of 15 with the speed of 4. This means that while the first troop causes 50×1 = 50 units of damage in T seconds but the second troop causes 15×4 = 60 units of damage which means the second troop is stronger than second one.

Relative Attack Power		OAS = Own Attack Speed EAS = Enemy’s Attack Speed = Sum of the powers of the troops attacking to the target troop = The target troop’s armour = Sum of the powers of the enemy troops attacking to the troop = Enemy troop’s armour in the aim of the troop = The troop’s Armour. This variable normalized by multiplying with 50. The result of 50 means equality.
Relative Speed		= Own Movement Speed = Enemy’s Movement Speed = The mean of the speeds of the attacking troop’s to the target troop. = The mean of the speeds of the enemy troops attacking to the troop. This variable normalized by multiplying with 50. The result of 50 means equality.

The usage of  symbol adds the group behaviour for the decision making. To give an example, if the Team A has only one troop with good condition in terms of its health, on the other side Team B has five troops and all of them heavily injured but has the opportunity to destroy the enemy troop. On this condition if the consideration algorithm runs on only the troops one by one than all the Team B troops should escape from the clash but actually they shouldn’t since they can destroy the enemy.

Another example for better understanding can be given from the figure below. The troop with number 7 from the Square Team is attacking to the troop with number 2 from the Circle Team and the number 2 is being attacked by number 8 as well. On the other hand number 7 is being attacked by number 3. On this condition RAP (Relative Attack Power) of number 7 will be calculated by sum of powers of number 7 and number 8 divide by the attacking power of number 3. So the equation for RAP is;

Equation 1 – RAP Equation for Troop 7

Figure 8 – A clash example between two forces

i.      Attack or Flee

In a RTS game one of the most usual decisions to be made is the individual decision of a troop to continue attacking or start fleeing. In most of the RTS games like; warcraft, dune 2000, command conqueror as well as age of empires these kinds of decisions made by FSM (finite state machines). The main reason for these games to use finite state machines can be predict as reducing the CPU time for game AI as FSM is a static structure which needs minimum amount of resources but the problem of this structure is the number of states and unrealistic behaviour.

Apart from FSMs another approach can be the Fuzzy logic which provides realistic behaviour but consumes more CPU time but this is a trade off and acceptable due to the increasing CPU/GPU power.

The first stage to setup the fuzzy logic is determination of Fuzzy Input data Sets (FIS) and membership functions. However it’s better to define the Fuzzy Output Set (FOS) and its membership functions as we have only two FOSs; first one is Attack, second one is Flee (figure-9).

Figure 9 – Fuzzy output sets and member functions

The FISs are Own Health, Enemy’s Health, Relative Attack Power (RAP), Relative Movement Speed (RMS). With the help of these variables the troop will show the group behaviour and make realistic decisions.

The other step is determination of membership function. On this paper only linear membership functions such as triangle and trapezoid used for easy understanding as well as easier calculation, however it is better to use logarithmic curves for RAP and RMS since they are the results of division of two forces and two speeds. For both of these data sets 50 represents the Equality.

Own Health	Enemy’s Health	RAP	RMS
Near dead	Near dead	Weak	Slow
Injured	Injured	Equal	Equal
Normal	Normal	Strong	Fast

Table 1 – Fuzzy Input Sets

And the third step is defining these FIS’s membership functions.

Figure 10 – Membership function for Own Health / Enemy’s Health which are normalized to 100

The figure above shows the membership function for Own Health as well as Enemy’s Health. As seen in figure, the health under 10 is definitely named like near dead and over 30 counts like Normal. The main reason for doing that is to prevent all the troops escape from the battle as all the troops should definitely fight if they are over 30 and they should try to escape if they are near dead.

The figure below shows the RAP membership function. In this MF (membership function), equality is 50 and definitely stronger means 1.2 times stronger than the troop which is equal to 60 and definitely weaker means 1.2 time weaker than the troop which is equal to 46.6.

Figure 11 – RAP Membership function

Figure 12 – RMS Membership function

On the other hand 1.1 times faster means definitely faster and 1.1 times slower means definitely slower. The tolerance is 10% to prevent the errors. If the tolerance is kept 20% than the flee consideration may not work as fifteen percent slower means no way to escape so it is better to keep the tolerance at 10%.

And the last part is determination of rules. This part is the most important and most open part to make mistakes. In addition, the numbers of all combinations are. However it is not necessary to create all the rules for every possibility. In our system the rules are; 3×3x3×3=81

INPUTS				OUTPUT
Own Health	Enemy’s Health	RAP	RMS	Attack / Flee
Not near dead	All possibilities	All possibilities	All possibilities	Attack
near dead	All possibilities	All possibilities	Slower	Attack
near dead	injured	equal	Not faster	Attack
near dead	near dead	Weaker	Not slower	Flee
near dead	injured	Weaker	Not slower	Flee
near dead	normal	Weaker	Not slower	Flee
near dead	normal	equal	faster	Flee
near dead	normal	stronger	faster	Flee
near dead	injured	equal	faster	Flee

Table 2 – Fuzzy Conditions for Attack or Flee

If the troop is not heavily injured (not near dead) then it should continue attacking and shouldn’t escape. The problem is if it is near dead since all the flee possibilities needs this condition first. But even if the troop is near dead and slower than the enemy it should attack because there is no way to escape and the rest of the rules are easy to understand with the help of the table above.

And the very last part is testing the fuzzy system with different data. All the data should start with the near dead condition or injured for Own Health since for the condition of normal, troop will definitely attack.

One of the best examples is giving these values;

Own Health	Enemy’s Health	RAP	RMS	Result
9.38(near dead)	35.6(normal)	56.8(stronger)	53.1(faster)	Attack
9.38(near dead)	36.9(normal)	56.8(stronger)	53.1(faster)	Flee
8.38(near dead)	35.6(normal)	56.8(stronger)	53.1(faster)	Flee

Table 3 – Test decisions

Although the only difference is enemy’s health with the difference of 1.3 the result changes and seems to be a good decision since if the enemy is weaker than there is a possibility to kill because it is stronger. And the next turn if the troop got closer to death than will definitely escape showed at the last line of the table above.

Figure 13 – Decision is attack with the value of 51.1

Figure 14 -  Decision is flee with the value of 49.6

ii.      Choosing Correct Troop to Attack

Another short term decision for individual troops in the RTS games is choosing correct troop to attack. This decision also is a short term decision since it is only related with an individual troop also in this part of this essay; Fuzzy Logic will be used to answer this question. The main reason for using Fuzzy Logic is the problem involves uncertainty and the solution should be deterministic. To give an example during a clash between two enemy groups let’s assume that Team 1 has one troop and Team 2 has two troops and the troop in team 1 trying to determine which enemy troop to attack also let’s assume that for each troop number of properties are three such as; health attack, power as well as armour. In this situation this troop should choose it according to degree of weakness of the enemy and this weakness should be calculated with the help of these troops’ properties. This explanation seems best fit to Fuzzy Logic to be used for determination.

The FIS terms and the explanations for the hypothetical game scenario of this section are;

Own Effective Attack Power (OEAP)		OAP = Own Attack Power OAS = Own Attack Speed EAR = Enemy’s Armour EFH = Enemy’s Full Health
Enemy’s Effective Attack Power (EEAP)		EAP = Enemy’s Attack Power EAS = Enemy’s Attack Speed OAR = Own Armour OFH = Own Full Health
Enemy’s Health (EH)		ECH = Enemy’s Current Health EFH = Enemy’s Full Health This variable normalized between [0-100] where 100 is healthiest.

These equations differ from the equations in `Attack or Flee` section of this essay since while determining the correct troop to attack should be done according to individual attack power not group attack power. The main reason for doing that is, if the group power is calculated instead of individual power against an enemy troop we cannot really choose a weak enemy for our attacking skills.

Another interesting point for this approach is this part doesn’t include Own Health parameter since this section is not determining our attacking or fleeing decision this means that the decision of the troop should be best fitting decision for the weakness degree not our own health.

After choosing FISs second part is creating the MF for these FISs.

Figure 15 – Membership functions for Choosing Correct Troop to Attack

Different from the ‘Attack or Flee’ section, there are four MFs where healthy MF is new. The mean reason of putting the healthy parameter is increasing the precise for enemy’s health MF since it is really important for the decision calculation.

The next step for creating our fuzzy logic is creating the output (FOS). The output membership graph consists of three possibilities as we need a precise output for this short term decision.

And the third step is creating rules. In this section, there are 36 possible combinations however it is reduced by using not operations as a result there are 15 combinations to implement, these are;

Inputs			Output
OEAP	EH	EEAP
strong	not healthy	All conditions	attack
strong	healthy	All conditions	neutral
not weak	near dead	not weak	attack
weak	near dead	weak	neutral
not weak	injured	All conditions	attack
weak	injured	All conditions	attack
normal	healthy	All conditions	neutral
weak	healthy	All conditions	don’t attack
normal	normal	strong	attack
normal	normal	not strong	neutral
weak	normal	All conditions	don’t attack
weak	not near dead	All conditions	neutral
not weak	All conditions	strong	attack
weak	All conditions	strong	neutral

Table 4- Fuzzy rules for choosing correct troop to attack

The main goal of these rules is attacking to the weakest in terms of its health and our attack power against this troop as well as the strongest enemy in terms of its attack power. After creating out fuzzy system the troop can choose the weakest enemy after the defuzzication operation.

Figure 16- Example for which troop to attack decision

To give an example, let’s suppose that there are three troops, first and second are from Circle Team and the third one is from Square Team (Figure 16). And the variables for the troop 3 against 2 and one are;

Troop 1			Troop 2
OEAP	EH	EEAP	OEAP	EH	EEAP
88	19	88	55	7	95

Under these conditions, the troop is powerful against the troop 1 and health of troop 1 is bad but better than troop 2, however, troops 2’s effective attack power is better than troop 1. So it is hard to determine since troop 2 is closer to dead but our attack power is not as strong as against troop1. But the fuzzy logic has an answer for both of the troops;

	Troop 1	Troop 2
Output (result of defuzzication)	66.7	67

Also the results of the fuzzy logic are very close to each other as expected but there is a numerical result that we can decide according to. So the troop will attack to troop 2 as the output is bigger than troop 1’s output.

Figure 17- Outputs for troop 1

Figure 18- Outputs for troop 2

b.     Long Term Decisions in RTS Games

The easiest explanation for long term decisions in RTS games can be described as the decisions which affect all or a group of troops or buildings for a long time as well as can be considered as the management strategy of the game player. The best example for long term decisions is resource management during game play which means how many resource collectors (e.g. villagers, harvesters) should be created and ordered to collect certain types of resources. In addition to resource managements other examples for long term decisions are; determining correct buildings to build, achieving correct upgrades/technologies, choosing correct time to attack with full power etc.

Not only for RTS games but also for any kind of game long term decision making always a big problem as the number of possibilities is may not be a certain value. Even if the numbers of possibilities are certain then possibilities and parameters are too much to make an easy decision. Moreover the difference between a short term decision and a long term decision is simply unreasonable short term decisions can be fixed in a short time however long term decision cannot be fixed so easily.

For RTS games long term decisions are vital since the CCP (computer controlled player) should behave realistic, intelligent as well as shouldn’t to not to frustrate the human player. The most common method for making these kinds of decisions are again the finite state machines which cause nearly a static game play. Every time the game player experiences the same game for the same map. To give an example in AOE2, at the end of the fourth minute, the CCP sends a scout to explore the map and creates a static number of villagers, orders them in a static way such as; four villagers hunt deer, ten of them collect food from trees etc., at the end of the seventh minute CCP attacks with a constant number of troops. This causes very static game scenario since the human player finds a way to repel the CCP and after the human player cannot be defeated by the CCP because the CCP cannot create any new way. The game producers of these RTS games usually create an AI with a static behaviour by using scripting languages. Therefore to create a strong opponent against the human player they prefer using cheating methods such as gifting new troops as well as resources which frustrates the human player.

To create a realistic behaviour for CCPs, which means intelligent and learnable behaviour, some other methods should be used such as ANN since it provides learning for CCPs. In addition to this ability of ANNs, ANNs may also provide un-deterministic behaviour by integrating GA (Genetic Algorithm). However, this method consumes lots of CPU time and provides a very slow learning.

This section of this paper will recommend using ANN (artificial neural networks) for long term decision making by giving some basic ideas such as defining input layer parameters for ANN and the outputs as well. However, all the recommendations are hypothetical since none of the ideas are based on any calculations or experiment. For these hypothetical recommendations the famous RTS game AOE2 (Age of Empires 2) will be used as the RTS game environment.

i.      ANN Structure and Hypothetic Feeding Mechanism of the ANN

As mentioned before on this paper, an ANN structure consists of three fundamental elements; inputs, weights and outputs. In our system inputs are the numerical values of a relevant system such as; for resource management system first input can be the recourse (e.g. gold, wood) the player has and the outputs are actions such as; create villager or send a villager to collect gold.

Figure 19 – An ANN Module

As mentioned before, the training method for the RTS AI is back propagation which is a supervised learning method and the weight calculations are not dynamic. This means the calculation of weights can be done after the game play.

To reduce the calculations and to separate behaviours, the module approach is preferred to be used.  To give an example from our conceptual game AOE2, behaviour of the CCP during dark-age in terms of resource collection should be different from castle age. Instead of using the current age as an input value for the AI, creating different ANN modules can be another approach. This also provides parallel calculations for server-side such as calculation of dark- age could be made separately from the calculation of castle age by using threads. Another benefit of this approach is the success of each module can be send to server as training data if it fits the necessary criteria such if the player achieves castle age before computer this data can be send to the server even if the player losses the game. However, determination of the criteria should be search carefully to not to train the AI with novel data.

Figure 20 – Before games start all weights getting from AI server

To load the weights of the game AI, the client should connect to the AI server and should get the trained ANNs weights to provide a better and dynamic game play. This method can be another approach for reducing piracy in games like, if the player uses a pirate version of the game cannot download the last AI weights since to download the AI data server can request for the player ID and password first.

Also the game AI server should contain trained and tested data before release of the game AI since, no one wants to play with a dummy AI. So the game company should be well trained the game AI modules which can be a problem for the game development.

Figure 21 – At the end of each game all input and outputs sending to AI server for calculation of weights if the result of the game is successful

Briefly the steps of this system are;

For AI loading;

1. Connect to AI server
   1. Check for ID, Password
   2. Send weights if successfully connected
   3. Create the local AI with new weights
   4. If cannot connect
      1. Use the previous ANN weights.

For data collection (collecting the training data);

1. For every action
   1. Collect input variables of the relevant AI module
   2. Collect the action
   3. If the current ANN module reaches the criteria to be changed
      1. Change the ANN module
      2. If the ANN training data reaches the criteria, mark to be sent to server
      3. Else don’t mark and don’t discard the ANN training data set
      4. At the end of the game
         1. Send all ANN training data sets of the winner’s.
         2. Send all the marked ANN training data sets.

For Server Side Weight calculations;

1. If there is new data,
   1. join all the data of relevant ANN modules including new data and old data
      1. i.      Create threads for every ANN module
      2. ii.      Calculate until reaching a threshold for every ANN module
      3. If there is no new data,
         1. Define threshold values lower than the old threshold values
            1. i.      Create threads for every ANN module
            2. ii.      Calculate until reaching a threshold for every ANN module

ii.      Example ANN Structure – 1: Resource Management in Dark Age

To create the ANN structure first of all the input values should be determined. In a RTS game for resource management can be the number of resources, number of troops collecting the resources.

For our example game AOE2, the input values for dark-age can be chosen like;

1. Number of workers

2. Number of workers collecting food

3. Number of workers collecting wood

4. Number of workers collecting gold

5. Number of workers collecting stone

6. Amount of wood we have

7. Amount of wood we have

8. Amount of gold we have

9. Amount of stone we have

10. Race of the side (such as Aztecs, Goths, Turks etc.)

And the outputs of the ANN module are decisions such as;

1. Create villager

2. Send a villager from food to wood

3. Send a villager from food to gold

4. Send a villager from food to stone

5. Send a villager from wood to food

6. Send a villager from wood to gold

7. Send a villager from wood to stone

8. Send a villager from gold to food

9. Send a villager from gold to wood

10. Send a villager from gold to stone

11. Send a villager from stone to food

12. Send a villager from stone to wood

13. Send a villager from stone to gold

The data collection in the game should be triggered by the action such as, if the player choose a villager from stone collection and send him to wood collection than all the input data and should be collected after the worker starts working.

At the same time another module is working for achieving technologies and if the resources are enough to pass the next age than all the data should be recorded according to the algorithm mentioned before.

iii.      Example ANN Structure – 2: Deciding the Correct Troops to Create

This part can be a either fuzzy logic or ANN since it seems like a spontaneous action, however the number of variable makes the fuzzy logic nearly impossible to implement this part and also actually this decision is not only spontaneous but also effects the game in long term as creating of Paladin is as expensive as achieving some technologies.

The input variables can be;

1. Number of own archers
2. Number of own knights
3. Number of own swordsmen
4. Number of own skirmishers
5. Number of own special troops
6. Number of own other troops
7. Number of own defensive buildings
8. Number of enemy’s archers
9. Number of enemy’s knights

10.  Number of enemy’s swordsmen

11.  Number of enemy’s skirmishers

12.  Number of enemy’s special troops

13.  Number of enemy’s other troops

14.  Number of enemy’s defensive buildings

15.  Score of the enemy (this variable provided by the AOE2 game environment)

16.  Own score

17.  Own race

18.  Enemy’s race

19.  Amount of wood we have

20.  Amount of wood we have

21.  Amount of gold we have

22.  Amount of stone we have

Outputs are decisions again;

1. Create archer
2. Create knight
3. Create swordsman
4. Create skirmisher
5. Create special troop

As seen above, the numbers of inputs are pretty much since it is really hard to determine even for human instinct which troop to create that makes the game interesting and keeps players’ passions on the game. Therefore if this system really can be implemented and reaches a degree of success than game’s lifetime for players definitely will be longer as most of the strategy game players play strategy games for the game AI not only for graphics.

Conclusion

Artificial intelligence is the most important for strategy games and especially for real time strategy games since the computer controlled opponents should make reasonable decisions spontaneously. For most of the strategy games, game producers preferred the easy way; cheating in different ways such as giving new troops, increasing income rate providing from resources and other similar ways, however results of these methods never satisfied the game players. The most important reason for that is; when the human player finds a way to defeat the computer, by using the same patterns (tactics), the human player never lose again. Briefly human player can learn but the computer cannot. Actually they can but this method never tried in commercial strategy games. There are several reasons behind that; first of all the CPU intensity of the learning structure, secondly the success rate is not known as the learning method hardly used in games. Not only machine learning but also spontaneous decision making, in most of the strategy games, is poor. Actually these short term decisions can be made with fuzzy logic and in some of the new strategy games it seems to be used (Command Conqueror Tiberium Wars) for choosing correct troop to attack, however it is not definite since no explanation have been made about the game AI for this game.

At first sight the implementation of fuzzy logic could be seem like simple especially for the short term decision making, however, the exponentially increasing number of possibilities makes hard the fuzzy logic to be implemented. On the other hand, in this paper, the tested short term decisions seem working. Especially choosing correct troop in the range to attack seems to be a successful application of fuzzy logic according to test results but without a real time application it is not reasonable to judge as a successful application of fuzzy logic since CPU usage is not known.

The conceptual distributed server based ANN application hasn’t tried yet by any game. Besides this kind of distributed approach haven’t tried for any kind of game AI yet. The CPU usage problem of the ANN is a known weak point of this structure. Another weak point of ANNs is the requirement for huge number of feed data (training). This conceptual distributed system may be a key for solution of these problems. Additionally, this trained AI structures can have a commercial value since they can decrease the number of pirate usage as the strategy game players always want to play with a more intelligent opponent and most of them will want to be registered to the AI service provider this means preventing piracy.

As a result, the problems about short term decision making and also long term decision making tried to be solved by using ANNs and Fuzzy Logic but none of the approaches has been tried in a real time game. However, the results for fuzzy logic seem to fit like a glove the given problem. On the other hand the conceptual ANN distributed system can be a new approach for solving the problems of ANNs in RTS games.

References

[1] David M. Bourg & Glen Seemann, AI for Game Developers, July 2004, First Edition, O’Reilly

[2] Mat Buckland & Andre LaMothe, AI Techniques for Game Programming, 2002, Premier Press

[3] Mat Buckland, Programming Game AI by Example, 2005, Wordware Publishing Inc.

[4] Todd Barron, Strategy Game Programming with DirectX 9.0, 2003, Wordware Publishing Inc.

[5] Mark A. DeLoura (edited by),  Game Programming Gems, 2000, Charles River Media

[6] Mark A. DeLoura (edited by),  Game Programming Gems 2, 2001, Charles River Media

[7] Steve Rabin (edited by), AI Game Programming Wisdom, 2002, Charles River Media

[8] T.J.A. Jansen, Player Adaptive Cooperative Arti_cial Intelligence for RTS Games, June 13, 2007, http://www.cs.unimaas.nl/~uiterwyk/Theses/BSc/Jansen_BSc-paper.pdf, last visit date: 30-11-2007

[9] Michael Chung, Michael Buro, and Jonathan Schaeffer, Monte Carlo Planning in RTS Games, http://www.cs.ualberta.ca/~mburo/ps/mcplan.pdf, visit date: 30-11-2007

[10] F.C. Schadd, June 13 2007, Hierarchical Opponent Models for Real-Time Strategy Games http://www.cs.unimaas.nl/~uiterwyk/Theses/BSc/Schadd_BSc-paper.pdf, visit date: 30-11-2007

[11] Daniel Johnson and Janet Wiles, Computer Games with Intelligence, http://www.itee.uq.edu.au/~uqdjohns/publications/AI/AJIIPS_AI.pdf, visit date: 30-11-2007

[12] Wikipedia, Artificial_neural_network, http://en.wikipedia.org/wiki/Artificial_neural_network, last visit date: 30-11-2007

[13] Wikipedia, Fuzzy Logic, http://en.wikipedia.org/wiki/Fuzzy_logic, last visit date: 30-11-2007

[14] Dr Emma Hart, Lecture Notes; Introduction to Neural Networks, 13-04-2005,  http://www.soc.napier.ac.uk/module.php3?op=getlecture&cloaking=no&lectureid=365774, last visit date: 30-11-2007

[15] Christos Stergiou and Dimitrios Siganos, NEURAL NETWORKS, http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html#Why%20use%20neural%20networks, last visit date: 30-11-2007

[16] Stuart James Kelly, Applying Artificial Intelligence, Search Algorithms and Neural Networks to Games, 2003, http://www.generation5.org/content/2003/KellyMiniPaper.asp, last visit date: 30-11-2007

[17] Artificial Neural Networks, 1999,  http://www.psych.utoronto.ca/users/reingold/courses/ai/nn.html, last visit date: 30-11-2007

[18] Artificial Intelligence in Games , 17 Jul 2006, http://www.codeproject.com/KB/architecture/aigame.aspx, last visit date: 30-11-2007