st@gmdzi.gmd.de (Sebastian Thrun) (09/17/90)
The following might be interesting for everybody who works with the PDP
backpropagation simulator and has access to a Connection Machine:
********************************************************
** **
** PDP-Backpropagation on the Connection Machine **
** **
********************************************************
For testing our new Connection Machine CM/2 I extended the PDP
backpropagation simulator by Rumelhart, McClelland et al. with a parallel
training procedure for the Connection Machine (Interface C/Paris, Version 5).
Following some ideas by R.M. Faber and A. Singer I simply made use of the
inherent parallelism of the training set: Each processor on the connection
machine (there are at most 65536) evaluates the forward and backward
propagation phase for one training pattern only. Thus the whole training set
is evaluated in parallel and the training time does not depend on the size of
this set any longer. Especially at large training sets this reduces the
training time greatly. For example:
I trained a network with 28 nodes, 133 links and 23 biases to approximate the
differential equations for the pole balancing task adopted from Anderson's
dissertation. With a training set of 16384 patterns, using the conventional
"strain" command, one learning epoch took about 110.6 seconds on a SUN 4/110 -
the connection machine with this SUN on the frontend managed the same in
0.076 seconds.
--> This reduces one week exhaustive training to approximately seven minutes!
(By parallelizing the networks themselves similar acceleration can be
achieved also with smaller training sets.)
--------------
The source is written in C (Interface to Connection Machine: PARIS) and can
easily be embedded into the PDP software package. All origin functions of the
simulator are not touched - it is also still possible to use the extended
version without a Connection Machine. If you want to have the source, please
mail me!
Sebastian Thrun, st@gmdzi.uucp
You can also obtain the source via ftp:
ftp 129.26.1.90
Name: anonymous
Password: <transmit your full e-mail address, e.g. st@gmdzi.uucp>
ftp> cd pub
ftp> cd gmd
ftp> get pdp-cm.c
ftp> bye