Artificial Inteligence Tuning Howto
The computer opponents in this computer poker game are completely self taught.
They start out totaly clueless bumbling around randomly folding or raising
without a care. They end up as extremely capable opponents to any human
player with the guts to try their luck. In fact over time they become
better and better. How is this possible? Its simple: they evolve.
Genetic programming 101
In order to train anybody to do anything you have to allow them to make
choices. You then punish them when they choose incorrectly and reward
them when they are right. Training a computer is no different. You
simply setup the critieria for which it will be judged and then allow it
to make choices.
Life on this planet has come up with a pretty nifty trick for training,
it is called evolution. You start off with a population of creatures
that are capable of: 1. procreating and 2. dying. Now in every population
of creatures there will be differences in individuals. Those individuals
that do better than other individuals have a better chance of staying alive.
Those 'better' individuals then have a better chance of procreating
and passing whatever traits they have along to their offspring. Those
offspring with the 'better' traits will then live an pass those traits onto
their offspring and so on.
To do this with a computer you need two things:
- A way of passing traits from one generation to the next.
- A way of deciding which individuals are 'better' than others.
To pass traits along one simply encodes them into a 'genotype'
which in our particular case is a three dimensional array consisting of 128,000
cells. When individuals are then mated to produce offspring their genotypes
are then combined in totally random way. This means that the offspring
will share many of their parents traits and will more than likely also exhibit
totally new behaviours which may or may not be better. Along with the
random combination of genotypes there is also a very very small amount of
purely random point mutations (usually on the order of 0.5%). These
random point mutations insure that the population will not grow stale through
too much inbreeding.
Deciding which individuals are better than others is
actually fairly easy in a game like poker where money is involved. Simply
put: the more money you have the greater your chance of producing offspring,
and the less chance you have of being exterminated. You can somewhat
control the strategies played by the individuals by tinkering around with
factors such as how often they mate (once per round or once per hundred rounds?),
and if they have the chance to play the same cards in all positions (how often
the deck is shuffled).
The human factor
So how does all this affect you? If you allow the
computer opponents to learn from you (by turning on that 'learn from human'
feature) then you will begin to affect who lives and who dies in the computer
opponent population. Over time those players who are better at attacking
the weaknesses of their competitors (including you) will survive. They
will turn into better and better competitors who will hone in any weaknesses
you have in your game. The best part is that as you change so will the
oponents so you can try various strategies and see if they have any weaknesses.
Great things come to those who wait
Since your computer opponents are grown more than forged
they will need to take time to learn. Fortunatly you can have them learn
against each other. Simply choose a brain set from the AI options dialog
and then choose 'Run simulation' from the File menu. On a 800 mhz box
you can expect a fairly decent batch of players in just a few hours. This
is recommended any time you change any of the important game options (such
as number of players, or maximum number of bets).
Learn what it feels like to play god
If at any time you feel that a set of opponents is no
longer worthy of your time then smote them via the 'Initialize' button located
in the AI options dialog box. Don't worry no perminant harm will be
done. You will however probably want to simulate a few hours of gameplay
with this newly initialzed brain set if you want to play against them again
(unless you just want some easy opponents!).