ccenol@rivm05.rivm.nl (Arnold Reinders) (03/04/91)
A World full of Vehicles
Invitation to join a Neural Network competition
In this document a world will be described, in which vehicles,
driven by small brains of neural networks drive around. Some
die, others mate and produce offspring and still others clone.
After a while some species will survive and others are extinct.
This is a good test environment to test neural networks.
Therefore we ask everybody to join us in a competition of
neural networks. You are requested to send in your best
performing networks according to guidelines put down in this
document. The networks will be implemented in vehicles and
we will see which neural network survives best. The winner will
be announced in this newsgroup and in an article to appear in a
major neural network magazine.
Next month the software will be ready in a library form.
Anyone who wants to test his own networks can request a copy
of this library. It will be a library of Turbo C++ routines.
If enough people are interested a copy could be provided in
Zortech C++. Send in your home address as well, because The software
contains almost half a megabyte of information, so I'm not
sure whether I am allowed to use the net for that. In that case I will
mail a floppy.
Contact:
Arnold Reinders
RIVM (state institute for public health and environment)
Dept. ISC
P.O. Box 1
3720 BA Bilthoven
the Netherlands
email: nollie@rivm.nl
Introduction
============
Vehicles is a program that is strongly influenced by a book with
the same name from Valentino Braitenberg (1986). In his book
Braitenberg explains how neural networks work by means of
simple vehicles that inhabit an artificial world. When I once had
to deliver a lecture about neural networks it seemed to me this
was a good metaphor, so I wrote a simple program that
simulated his idea's. In this program a simple world is created in
which vehicles move around to eat mushrooms and to avoid
cactuses.
The basic machinery that makes a vehicle move around is its
brain. This can be a neural network of any kind (back
propagation, ART or a Boltzmann machine). A representation of
the external world and the internal states are mapped upon this
brain and as a response the brain returns some action like
moving a wheel from the vehicle.
A friend of mine and myself started to write neural networks for
these vehicles to see who could write the best network. We had
a lot of fun and as a result of this we ask everybody to join us in
a competition of neural networks. The networks will be im-
plemented in vehicles and we will see which neural network
survives best.
Warning:
Any algorithm will be considered public domain and will be
published in this newsgroup. If you send in an algorithm, no
copyright can be claimed, neither by the author, myself or
anybody else. I oblige myself to publish any article in this
newsgroup that is sent to me. In my own publications the
names of the authors will be attached to it.
Note:
This world was and still is intended for fun. That means that
a lot of physical laws are interpreted in such a way that
computation on normal PC's is minimised. Reality is quite
irrealistic in Vehicles.
Users Guide
===========
The world in which the vehicles move is a rectangle surrounded
by impenetrable walls. In this world vehicles, mushrooms and
cactuses are living. As a vehicle moves around it diminishes and
when it becomes too small it dies and becomes a mushroom.
Whenever a vehicle stumbles upon a mushroom, this mushroom
will be eaten. When a cactus stands in the way, the vehicle is
wounded and declines while the cactus grows proportional in
size.
Every type of object emits light that are characteristic for this
object and by which it can be distinguished from other objects.
Every object is considered to be round (they may be drawn in
any other shape however) and its mass is pi * r * r. The different
kind of objects will be described below.
The Vehicle
===========
A vehicle is the only moving object in this world. It contains
sensors, two wheels and a neural network that interprets the
world. It is perfectly round and it emits light at a wavelength of
at least 75 (nanometers if you wish, but keep in mind: it needn't
be realistic). Before we go into more detail of all aspects of a
vehicle, I'll first describe how it behaves in general. In figure 2.1
a picture is shown.
As said before, a vehicle contains two wheels, and sensors with
which it can monitor the outside world and its internal state. The
information that the sensors register are sent to the vehicles
controlling brain (its neural network). The brain interprets and
handles all signals and as a result it will send signals to, for
example, its wheels. When it meets another vehicle, it can
decide to mate in order to produce offspring.
In the above mentioned vehicles, two survival strategies can be
distinguished: at first the survival of each individual vehicle. It
tries to take decisions in such a way that it lives as long as
possible. Another survival strategy has to do with the survival of
the species. By choosing a useful way of reproduction (a so
called genetic algorithm) it is possible to create a species that
survives for a long period of time. Below we will describe
some of the characteristics of a vehicle.
This document describes a competition. Now it is time to
describe clearly what part of the world and vehicle is given, i.e.
cannot be changed by the competitor and which part must be
defined by the competitors. The world and the vehicle are pre-
defined. The vehicle is governed by physical laws which are
defined below and which cannot be changed by anyone. The
behaviour of the vehicle on the other side is determined by its
brain, and this is fully defined by the competitor . Any neural
network is allowed, like backprop, Boltzmann, ART or a self
defined network. This network receives input from the outside
world via sensors, processes this information and transmits the
results to effectors. These are interpreted by the vehicle so it can
move along. Reproduction of the network is also fully
determined by the competitor.
Species of vehicles
===================
Each neural network that is sent in, is considered as a separate
species, that means that it is assigned to a vehicle of a certain
species. Vehicles of the same species can produce offspring,
while vehicles of different species cannot. Species can be recog-
nized by two features. At first it is assigned a unique picture.
In this way it can be
distinguished from other vehicles. At second it is assigned its
own wavelength to distinguish it from other species. In this way
the vehicles can use its photo sensors to distinguish vehicles of
its own species from vehicles from other species.
In summary we can say that the vehicle is the carrier of the
neural network. The vehicle is subject to predefined physical
laws which cannot be influenced by the competitor. The neural
network is defined by the competitor. The vehicle and the neural
network communicate with each other by means of an interface.
All these aspects will be described in detail below.
Sensors
=======
There are three types of sensors in the vehicle:
1. photosensors that detect light of a certain wavelength,
2. pressoreceptors that indicate whether a vehicle bumped
against another objects,
3. internal receptors that reflect the state of the vehicle.
Photosensors
============
The purpose of photosensors is:
- to discriminate between intensities of light,
- to discriminate between different kind of objects and
- to determine the direction of the light
The first object is reached because objects radiate light,
depending upon size and distance of the object to the vehicle.
The photosensors sum all values and transmit them to the neural
network.
To discriminate between objects can be accomplished by letting
each object emit its own wavelength of light that is unique for
that type of objects. In table 2.1 this has been summarized:
Object type Wave length
Mushroom 25
Cactus 50
Vehicle >=75
Table 2.1, the wavelengths of various objects.
Each species of vehicles will be assigned its unique wavelength
so that it can be distinguished from other vehicles. So a
competitor has to keep in mind that his vehicle will be assigned
a wavelength that is not known beforehand. He can distinguish
other vehicles because they have another wavelength that is also
not known before.
The photosensors can discriminate because there exists a row of
sensors, each with a specific sensitivity for a certain wavelength.
The number of photosensors is not known beforehand. A sensor
does not sense exactly one wavelength, but senses a wavelength
that is normally distributed around its defined wavelength and a
standard deviation. The sensitivity of the sensors is computed as
follows: suppose that there is a maximum bandwidth w of light
for which photosensors can be sensitive. Furthermore there are
n sensors. The sensors are sensitive at equidistant point in the
bandwidth, so the sensitivities of sensor si can be computed by
(2.1):
(2.1) i * b
si = -----
n + 1
Suppose we have a bandwidth of 100, and 3 sensors, then the
sensitivity of the first, second and third sensor is 25, 50 and 75
respectively. The standard deviation sd. for each sensitivity can
be found in 2.2:
(2.2) 1 b
sd = - * -----
2 n + 1.
That means that when light of a certain wavelength falls upon a
sensor that is sensitive for another wavelength, a portion of that
light still filters through.
The direction of light (the third objective we would like to fulfil
with our photosensors) can be computed by setting up two
groups of sensors at different locations so that differences in
radiation can be attributed to distance. The photosensors are lo-
cated at the wheel.
This all boils down to the following: the intensity of the light is
proportional to the mass of the object, so the heavier the object,
the more light it emits. The sensor of the vehicle receives only
some of it, that is the mass of the emitting object divided by the
distance to that object. The intensity of light a sensor s
registrates from object o can be put into formula (2.3) as:
(2.3) m
s = -------- , where d (o, s) is the Cartesian distance of
d (o, s) o and s.
In (2.3) is m the mass of the emitting object which is equal to
pi * r * r. Of this amount only a part is filtered that fits to the
wavelength y (lambda). This amount f is described by formula
(2.4) as:
(2.4) normal (signal; y, sd)
f = ----------------------
normal (y; y, sd)
where b is the bandwidth (typically a number between 100 and
1000) and n is the number of sensors (which usually varies
between 3 and 10).
The total input of a sensor is determined by f * s. In this way an
object can be perceived by different sensors, but each sensor
receives another image from the perceived object.
A last remark must be made about how light travels in this
program, because this is most peculiar. It is not disturbed by any
object, so for photosensors other objects are 100% transparent.
At second, a vehicle can see at an angle of at most 90x at each
side of the direction in which it is moving. Because it has no
eyes in its back, it can't see what is happening behind its back.
Pressoreceptors
===============
Pressoreceptors transmit information of an impact against the
wall, against other vehicles and cactuses to the neural network.
The aim is to tell the brain that an impact took place and the
speed of the impact.
The physical laws of the world do not know elasticity. What a
vehicle should do after it has encountered a wall should be
completely determined by its brain. It can continue to bang its
head against the wall, or it can start to turn around to do some
more useful things.
Pressoreceptors are located at four positions: at both wheels, the
front and the back. Pressoreceptors cannot distinguish between
the type of objects, only the energy of the impact is transmitted,
which is equal to the speed of the vehicle.
Mostly the impact will not be located at exactly one
pressoreceptor, but will be somewhere between two receptors.
The result of the impact is proportional distributed over the two
nearest receptors.
Internal Receptors
==================
The so called fatness indicator reflects the mass of the vehicle.
If the vehicle has a mass that is too small , it will die and
become a mushroom with the same mass. Whenever a vehicle
becomes too fat, it will clone, that means create an identical
copy of itself. Typical radiuses (masses) are in the range of (5
(79) to 20 (1257), while a vehicle with radius 50 (7854) may
sometimes occur. The value transmitted is the current mass
divided by its maximum allowed mass, a number between 0 and 1.
Another internal receptor is the mass change receptor. It
transmits positive values if the vehicle has grown (i.e. by eating)
or negative values when it has declined since the last move. The
value transmitted is the current mass minus the mass of the
former move, divided by the maximum allowed mass. Again a
number between 0 and 1.
The pleasure receptor records whether an event was pleasant or
not for the vehicle during the last move. It is positive (+1) when
and event was considered pleasant and negative (-1) if an event
was experienced as unpleasant. Mating and eating are con-
sidered as pleasant events, while touching a cactus or cloning
are considered as unpleasant.
Effectors
=========
Effectors are devices by which neural energy is transformed into
some action, i.e. the movement of a wheel, or reflects some state
or ability of the vehicle (for example sexual arousal).
Speed
The two most important effectors affect the vehicles wheels. By
means of this the vehicle moves around and because it has two
wheels, one on each side, it can turn or even move backwards.
Moving around costs energy. This energy is proportional to a
vehicles mass and speed. How do we relate this energy to the
mass lost by a movement. When we multiply this by a constant,
and determine this constant Ce empirically , we have a formula
of how to determine the mass md that is lost by making a move
with a certain speed:
(2.5) md = Ce * m * (1 + |v|)
As we can see it takes proportional more mass the faster we
move. The vehicle can now determine how to trade in mass for
speed. A vehicle could decide to do so when it detects a
mushroom and sees another vehicle speeding towards it. Notice
that if a vehicle decides to do nothing, it still will lose some
weight.
Arousal
=======
There is one state variable and that is sexual arousal. If a vehicle
is aroused it can mate with other aroused vehicles and in doing
so offspring can be produced. Mating costs energy and will
inevitable take some of the vehicles mass away. Small vehicles
could decide to search for mushrooms in stead of mating. This
decision can be based upon the fatness indicator.
Reproduction
============
This chapter deals with reproduction by means of genetic
algorithms. If you do not know genetic algorithms, don't worry,
in the next paragraph is described what to do if you want to
work without genetic algorithms. Maybe gives this introduction
enough hints to experiment some yourself.
The aim of reproduction is to produce more out of two. Well...
not exactly. It provides us with a means to provide variation into
the world and thus to create vehicles with properties that did not
exist before.
When a vehicle bumps into another vehicle, and it has its sexual
arousal indicator switched to on (as well as the vehicle bumped
into) and they belong to the same species, they reproduce. This
reproduction results in one child is produced, of which its brain
is the result of a genetic algorithm. The sum of the mass of the
parents and their child is exactly the same as that of both parents
before.
After this the three vehicles go their own way. The crucial point
here is that there is presumably an algorithm that creates a better
performing neural network as output when it is given two neural
networks as input. Such an algorithm is a so called genetic
algorithm.
There are many genetic algorithms and I have chosen just one
for the purpose of illustration. Suppose that all routes between
one input neuron and one output neuron are known and that they
must be transmitted to the child. The child will get randomly
this connection from the one or the other parent. In this way we
transmit the experience of one generation to the next, while
preserving sufficient variation. We introduce some mutation,
that means that in some cases the connection will randomly
chosen, and not from one of the parents. In this way some new
variation can be introduced.
When a vehicle clones, the reproduction routine is called with
the same vehicle as both parents, so a new brain has to be
created out of two the same brains. In practice this means that a
same brain is created with some mutations. It is up to the
competitor to make something out of this.
No genetic algorithm
It can be that you are not interested in the genetic algorithm
part. In that case a network can be generated in the standard way
(to be used for example when a vehicle is created) and transmit
this as a result of the reproduction scheme. In this way there will
be no learning from one generation to another. If a competitor
does not know much about genetic algorithms, but wants to
profit from the experience learned from a vehicle, he could copy
the network of one of the parents chosen at random to the child.
The Mushroom
============
The mushroom is an object of the world and it emits at a
wavelength of 25. It can be eaten by a vehicle, with the result
that the mass of the mushroom is added to the mass of the
vehicle. The mushroom then disappears from sight and out of
the world.
Each mushroom grows slowly, and as it gets too heavy it
clones. They grow randomly, or as vehicles die, at the spot of
the deceased vehicle appears a mushroom with the same mass of
the deceased.
The Cactus
==========
A cactus is an object in the world that emits light at a
wavelength of 50. A cactus is dangerous for vehicles while it
causes them to lose mass. The amount of mass lost is randomly
determined. The mass a vehicle loses is added to that of the
cactus. However, contrary to mushrooms they gradually decline,
so they depend for their living upon vehicles to bump against
them. If a cactus becomes too small, it will die and become a
mushroom, when it becomes too big, it will clone.
The competition
===============
This competition should also know a winner. The winner is that network
which species has the biggest amount of mass. For each species the
sum of the mass of each vehicle is determined, and the species that
has the most mass accumulated is the winner. In some pilot trials
we will look what is the amount of days for one simulation, we think
of 1000 to 10000 days.
The competition is organised according to a system of elimination.
We organise pools of species (the pool size must be determined, but
will probably be a number between 3 and 10) where each species will
have the same amount of members as the others (as well to be determined,
and probably a number between 3 and 10).
Each simulation lasts the same amount of days.
Technical documentation
=======================
The world has been written in C++, while this language allows
for modular, well documented programs and it is rather easy to
learn for C'ists. Each object in this world (i.e. a vehicle,
mushroom, etc...) is a class with some public and virtual
routines (these routines will be called methods for the rest of the
document). Because the vehicle and the physical aspects of the
world are predefined the user has no way to change them.
Because in this competition a neural network and a genetic
algorithm should be delivered, in this section I will describe
how the interface should look like.
Each competitor is free to define his own neural network, as
long as it obeys some rules. Neural networks that do not obey
these rules will not be regarded in the competition. In order to
understand these rules a short outline of how the vehicle com-
municates with the network will be given.
The time in the world is discretised, and each time step the
vehicle determines its external and internal states, transmits
them to the neural network, requests the network to process
these data and reads out the output results. Based upon this
output the vehicle makes a move. When a sexually excited
vehicle bounces against another vehicle of the same species , the
reproduction routine is called. Notice that the competitor cannot
determine when the reproduction method is called, that can only
be done by the vehicle.
The neural network must be a C++ class. It should have a public
constructor and destructor, and the methods cycle and
reproduce. Cycle processes the data that the vehicle itself
places in the neural network by means of so called parameter
blocks. The results are put into a shared piece of memory, the so
called output parameter block, which at the end of each cycle is
read by the vehicle to determine what its next move will be. The
method reproduce is called to create a new network out of
itself and its partners network.
Parameter blocks
================
Input and output is communicated to the neural network via
parameter blocks. A parameter block is such defined that it can
be easily extended in future. It consists of groups of parameters.
Each parameter group is identified by its type, the amount, to be
followed by the values of the parameter. The values that TYPE
can assume are described in table 3.1:
Type Meaning
PHOTO_1 Photosensor at right wheel
PHOTO_2 Photosensor at left wheel
PRESSO_1 Pressoreceptor at front
PRESSO_2 Pressoreceptor at right wheel
PRESSO_3 Pressoreceptor at back
PRESSO_4 Pressoreceptor at left wheel
FATNESS Fatness indicator
W_CHANGED Amount of weight changed
PLEASURE Pleasure indicator
WHEEL_1 Right wheel
WHEEL_2 Left wheel
SEX Sexual arousal indicator
Table 3.1, the different values of TYPE
Parameter groups cannot be accessed directly. This must be
done via methods of the parameterblock in order to avoid errors.
A parameter block exists out of several parameter groups. The
total number of parameter groups is determined by the method
size of parameter block. The number of parameter groups in a
parameter block is 1..size.
Parameter groups are strictly topographically organised
whenever possible, that means clock wise, starting from front.
In this way the competitor has the possibility to compare results
from sensors at the same location. A prototype of class
parameter block is defined in figure 3.1.
class Parameter_Block
{
int parameters ();
int size ();
int group_size (int index);
int set_parameter (int index, int par, float value);
int get_parameter (int index, int par, float& value);
};/*** Class: Parameter_Block ***/
Figure 3.1, Prototype of class Parameter_Block
The input parameter block
=========================
The input parameter block is a piece of memory that the vehicle
and the neural network share. The vehicle puts values into it
while the neural network reads them. It is passed to the network
during set-up, that means as the first parameter to the construc-
tor. In this input parameter block the following parameter
groups are currently defined (see table 3.2):
Size Type Location Meaning
1 PRESSO_1 front Pressoreceptor at front
n PHOTO_1 right wheel Photosensor at right wheel
1 PRESSO_2 right wheel Pressoreceptor at right wheel
1 PRESSO_3 back Pressoreceptor at back
n PHOTO_2 left wheel Photosensor at left wheel
1 PRESSO_4 left wheel Pressoreceptor at left wheel
1 FATNESS internal Fatness indicator
1 W_CHANGED internal Amount of weight changed
1 PLEASURE internal Pleasure indicator
Table 3.2, the layout of the input parameter block
By means of the get_parameter method, values can be read
from the input parameter block. Suppose we got the following
parameter block transmitted through the constructor of the
neural network:
Parameter_Block* in;
and suppose that we want to read out the values of both photo
sensors. At first we have to determine the size of the sensors.
This can be done by the method group_size. It returns the
size of the requested parameter. We can now read the values of
the photo sensors:
int n = in->group_size (PHOTO_1); // Size of sensor
float* foto1 [n + 1]; // Declare array to store values
for (int i = 1; i <= n; i++) // Loop to read out values
{ // of right wheel
in->get_parameter (PHOTO_1, i, foto1 [i]);
}
Fig 3.2, Reading the input parameter block
In the above mentioned example we have copied the values of
the photo sensors to an array in our neural network. Because we
use the built-in methods for requesting parameters and their
values this piece of routine is insensitive for changes in the
general build-up of the vehicle. So if the amount of photo
sensors changes, and that will certainly happen, this network
piece of code will not suffer because of it.
The output parameter block
==========================
Like the input parameter block, the output is a piece of memory
that is shared by vehicle and network alike. The network sets
values in this block and it is read by the vehicle in order to
determine its next move. To avoid disruption of the output
parameter, they are hidden. Output parameters can be set by the
following method:
int Set_Parameter (int index, int par, float value)
A zero is returned if no problems were encountered, a one if the
denoted parameter did not exist. The parameter group accessed
by index indicates the parameter group to be accessed, pa-
rameter indicates the index in the group and value is the
value the indicated parameter is to be set to. The following
output parameters do exist:
Size Type Location Meaning
1 WHEEL_1 Right wheel Distance wheel turns
1 WHEEL_2 Left wheel " " "
1 SEX Internal If the value is
positive the vehicle
is aroused, else not
Table 3.3 The definition of the output parameter block
Suppose we have computed in our cycle routine new positions
for the wheels, and have stored these values in wheel1 and
wheel2, than we can transmit these values to the output
parameter block as follows:
float wheel1, wheel2;
// Let the network compute values for wheel1 and wheel2.
out->set_parameter (WHEEL_1, 1, wheel1);
out->set_parameter (WHEEL_2, 1, wheel2);
Fig 3.3, Setting the output parameters
The network and the vehicle share the same copy of the
parameter blocks. When defining a neural network, these two
parameter blocks are the only parameters that are transmitted to
the network via the constructor. The vehicle sets the input
parameter block, let the network cycle and requests values of
from the output parameter block.
In figure 3.4 a prototype of class network is given. In this
network there are only three public routines: the constructor and
the methods cycle and reproduce. These methods will be
discussed in detail in the next paragraphs.
class Neural_Network
{
public:
Neural_Network (Parameter_Block*, Parameter_Block*);
~Neural_Network ();
virtual void cycle ();
virtual Neural_Network* reproduce
(Neural_Network* partner, // The other parent
Parameter_Block* in, // Input parameters
Parameter_Block* out); // Output parameters
};/*** Class: Neural_Network ***/
Figure 3.4, Prototype of class Neural_Network
The Constructor
===============
Prototype: network (Parameter_Block* input, Parameter_Block* output)
Parameters:
input The parameter block through which the input is
transmitted from the vehicle to the network.
output The parameter block through which the output of
the network is transmitted to the vehicle. It is set
by the network.
Description:
The constructor sets up the network. Relevant
information as to the size of the network needed
and the construction of the vehicle should be
derived from the parameter block.
The Cycle Method
================
Prototype: void cycle ()
Description:
This method reads the input parameter block,
does an update of the neural network and
transmits the results to the output parameter
block.
The Reproduction
================
Prototype: network* reproduce (network* other,
Parameter_Block* in,
Parameter_Block* out)
Parameters:
other Other is a pointer to a network of the vehicle the
owner of this network bounced into.
in Pointer to input parameter. Will be provided by
the vehicle.
out Pointer to output parameter. Will be provided by
the vehicle.
Results:
A pointer of a newly produced network that will
be used for the child of both networks.
Description:
The reproduce method is part of a network and
will be called when a reproduction should take
place. The first parameter is a pointer to another
network that together with the calling network
provided the input for the reproduction. The other
two parameters are pointers to the input and
output parameter block that should be transmitted
to the constructor of the neural network.
How to reproduce is completely free to the user
as long as it results in a new network. A pointer
to this new network must be returned.
Whenever you are not interested in this part, you
could simply state the following in your routine:
return new Neural_Network (in, out);
An Example
==========
As an example for a neural network I have chosen for a simple
feed forward network with a Hebbian learning rule. Below first
these two things will be explained, and next to that an outline of
the data and method structure will be given. Finally,
I will give the complete listing of the class as will be
accepted by the program.
The Neural Network
==================
The network consists out of an input layer, a hidden layer and
an output layer. Input is mapped from the input parameter block
upon the input layer. The input is processed by the units if the
input layer and transmitted to the hidden layer. The hidden layer
transmits the information to the output layer where it is mapped
upon the output parameter block.
The Input Layer
===============
The input layer consists out of a number of units that is equal to
the amount of input parameters and output parameters. This is
done in order to create some feedback because not only the
input sensors will be mapped upon the input layer, but the
output effects are mapped as well. In this way the network is
able to associate some input with former behaviour. Each unit
transmits input to the hidden layer via a sigmoid function. That
means that the activation of each unit is in the interval [0..1].
The Hidden Layer
================
The hidden layer consists of half of the units of the input layer.
It transmits information to the output layer via the sigmoid
function.
The Output Layer
================
The output layer consists of a number of units that is determined
by the amount of effectors needed. The output receives the
signals from the hidden layer. The sum of these signals is not
affected by some transfer function but immediately mapped
upon the output parameter block. However, for comparing the
values of the hidden with the output the activity computed by
the sigmoid function is used.
Weights and Learning
====================
Weights in this demonstration model lie in the interval [-1..+1].
When they are adjusted by the learning rule they will never
exceed these values.
Learning is some special case of the Hebbian learning rule. It is
assumed that when units contain the same activation, that their
weights should not be adjusted. That means that the bigger the
difference in activations, the bigger the adjustment to the
weights should be. The weight will be changed in the direction
determined by the difference in activations, while the amount of
change is governed by this change as well. This amount
however, is multiplied with a small value, epsilon, in order to
keep down the change. Let wij be the weight between unit ui
and uj and ai and aj be the activation of ui and uj respectively.
We can then determine the direction of change by ai - aj. This is
immediately the amount of change. In order to avoid overflow
of the weight, we must look to the amount of weight available,
that is 1 - |wij|. Second, we want to have a slow change, so we
multiply this with a small number e(psilon). This result in
formula (4.1):
(4.1) delta wij = e * (1 - |wij|) * (ai - aj)
The Genetic Algorithm
=====================
When a vehicle meets another vehicle and they are both sexual
aroused, the two will generate a neural network for their child.
This network is generated by the method reproduce
(Neural_Network*). There are many ways of creating such
a network. The method we have described before will now be
algorithmically explained.
Between each unit in the input layer and each unit in the output
layer exist a lot of paths. Such a path will be randomly inherited
from either parent. In about 10% of the cases the weight will be
randomly generated, in order to create some mutation.
Outline of Class Neural_Network
===============================
Bases upon the former discussions and upon the requirements of
the competition, we can make a prototype the neural network as
follows:
#include <stdio.h>
#include "intface.hpp" // Must be included: will
described below
class Neural_Network
{
protected:
float* F1; // Sensor layer
float* F2; // Hidden layer
float* F3; // Effector layer
float** W12; // Weight matrix from 1 to 2
float** W23; // Weight matrix from 2 to 3
int n1, n2, n3; // Size of F1, F2 and F3
Parameter_Block* IN; // Pointer to input parameter
Parameter_Block* OUT; // Pointer to output parameter
float sigmoid (float);
virtual void get_input ();
virtual void set_output ();
virtual void learn (float);
public:
Neural_Network (Parameter_Block*, Parameter_Block*);
~Neural_Network ();
virtual void cycle ();
virtual Neural_Network* reproduce (Neural_Network*,
Parameter_Block*,
Parameter_Block*);
};/*** Class: Neural_Network ***/
Neural_Network::Neural_Network (Parameter_Block* in,
Parameter_Block* out)
// Initialisation of the network
{
int i, j;
IN = in;
OUT = out;
n1 = IN->size () + OUT->size ();
n2 = n1 / 2;
n3 = OUT->size ();
F1 = new float [n1];
F2 = new float [n2];
F3 = new float [n3];
W12 = new float* [n1];
W23 = new float* [n2];
for (i = 0; i < n1; i++)
{
W12 [i] = new float [n2];
for (j = 0; j < n2; j++) W12 [i] [j] = uniform (-0.1, 0.1);
}
for (i = 0; i < n2; i++)
{
W23 [i] = new float [n3];
for (j = 0; j < n3; j++)
W23 [i] [j] = uniform (-0.1, 0.1);
}
}/*** Neural_Network ***/
Neural_Network::~Neural_Network ()
{
int i;
delete F1;
delete F2;
delete F3;
for (i = 0; i < n1; i++) delete W12 [i];
delete W12;
for (i = 0; i < n2; i++) delete W23 [i];
delete W23;
}/*** ~Neural_Network ***/
float Neural_Network::sigmoid (float x)
{
return 2.0 / (1.0 + exp (x)) - 1.0;
}/*** sigmoid ***/
void Neural_Network::connect12 (int i, int j, float weight)
{
if (i >= 0 && i < n1 && j >= 0 && j < n2)
W12 [i] [j] = weight;
}/*** connect ***/
void Neural_Network::connect23 (int i, int j, float weight)
{
if (i >= 0 && i < n2 && j >= 0 && j < n3)
W12 [i] [j] = weight;
}/*** connect ***/
void Neural_Network::get_input ()
{
float value;
int j, par;
int count = 0;
// Map input parameters upon first layer of the network
for (par = 0; par < IN->parameters (); par++)
{
for (j = 0; j < IN->group_size (par); j++)
{
if (IN->get_parameter (par, j, value) != OK)
{
// Gets here when a parameter was not recognized by the
// parameter block.
printf ("Get_parameter () not OK");
}
F1 [count] = sigmoid (value);
count++;
}
}
// Map the output parameters upon the F1 layer as a
// way of feedback.
for (par = 0; par < OUT->parameters (); par++)
{
for (j = 0; j < OUT->group_size (par); j++)
{
if (OUT->get_parameter (par,j,F1 [count]) != OK)
{
// Gets here when a parameter was not recognized by the
// parameter block.
printf ("Get_parameter didn't return OK");
}
count++;
}
}
// Adjust some values to get a higher discrimination.
// Add first at some
// noise so that always some behaviour is performed.
for (j = 0; j < n1; j++)
F1 [j] += uniform (-0.25, 0.25);
}/*** get_input ***/
void Neural_Network::set_output ()
{
int j, par;
int count = 0;
for (par = 0; par < OUT->parameters (); par++)
{
for (j = 0; j < OUT->group_size (par); j++)
{
if (OUT->set_parameter (par,j,F3 [count]) != OK)
{
printf ("set_parameter didn't return OK");
}
count++;
}
}
}/*** set_output ***/
void Neural_Network::learn (float epsilon)
{
int i, j;
for (i = 0; i < n1; i++)
for (j = 0; j < n2; j++)
{
W12 [i] [j] += epsilon * (F1 [i] - F2 [j])
* (1.0 - fabs (W12 [i] [j]));
}
for (i = 0; i < n2; i++)
for (j = 0; j < n3; j++)
{
W23 [i] [j] += epsilon * (F2 [i] - F3 [j])
* (1.0 - fabs (W23 [i] [j]));
}
}/*** learn ***/
void Neural_Network::cycle ()
{
float sum;
int i, j;
get_input ();
for (j = 0; j < n2; j++)
{
sum = 0.0;
for (i = 0; i < n1; i++)
{
sum += F1 [i] * W12 [i] [j];
}
F2 [j] = sigmoid (sum);
}
for (j = 0; j < n3; j++)
{
sum = 0.0;
for (i = 0; i < n2; i++)
{
sum += F2 [i] * W23 [i] [j];
}
F3 [j] = sigmoid (sum);
}
set_output ();
learn (0.01);
}/*** cycle ***/
Neural_Network* Neural_Network::reproduce
(
Neural_Network* partner,// Other network as parent
Parameter_Block* in, // Input parameter block from new vehicle
Parameter_Block* out // Output parameter block from new vehicle
)
// Generates a new neural network from the
// current one and the supplied
// partner network. The parameter blocks are
// transmitted as they are supplied
// from the new vehicle.
{
Neural_Network* NN = new Neural_Network (in, out);
float mutation = 0.1; // Mutation of about 10%
int i, j, k;
for (i = 0; i < n1; i++)
for (j = 0; j < n3; j++)
for (k = 0; k < n2; k++)
if (uniform (0.0, 1.0) < mutation)
{ // Mutation of the weights
NN->connect12 (i, k, uniform (-1.0, 1.0));
NN->connect23 (k, j, uniform (-1.0, 1.0));
} else
{
if (uniform (0.0, 1.0) < 0.5)
{ // Weights from parent one in half of the cases
NN->connect12 (i, k, W12 [i] [k]);
NN->connect23 (k, j, W23 [k] [j]);
} else
{ // Else the weights of parent two
NN->connect12 (i, k, partner->W12 [i] [k]);
NN->connect23 (k, j, partner->W23 [k] [j]);
}
}
return NN; // Return pointer to new neural network
}/*** reproduce ***/
Intface.hpp
===========
The intface header file provides a lot of definitions and
prototypes which must be used. You should include it in your
own source. In order to make it easier, the complete source is
given below.
const int OK = 0;
const int ILLEGAL_INDEX = 1;
const int ILLEGAL_PARAMETER = 2;
const int PHOTO_1 = 0;
const int PHOTO_2 = 1;
const int PRESSO_1 = 2;
const int PRESSO_2 = 3;
const int PRESSO_3 = 4;
const int PRESSO_4 = 5;
const int W_CHANGED = 6;
const int FATNESS = 7;
const int PLEASURE = 8;
const int SEX = 0;
const int WHEEL_1 = 1;
const int WHEEL_2 = 2;
class Parameter_Block;
class Parameter_Group
{
friend class Parameter_Block;
private:
int SIZE;
float* VALUE;
Parameter_Group (int size);
~Parameter_Group ();
};/*** Class: Parameter_Group ***/
class Parameter_Block
{
private:
int PARAMETERS;
Parameter_Group** PARAMETER;
public:
Parameter_Block (int*, int);
~Parameter_Block ();
int parameters ();
int size ();
int group_size (int index);
int set_parameter (int index, int par, float value);
int get_parameter (int index, int par, float& value);
};/*** Class: Parameter_Block ***/