Russell Leighton <russ@dash.mitre.org> (04/08/91)
Archive-name: ai/neural-nets/asprin-for-migraines/1991-03-06
Archive: polaris.cognet.ucla.edu:/alexis/am4.tar.Z [128.97.50.3]
Original-posting-by: Russell Leighton <russ@dash.mitre.org>
Original-subject: simulator announcement
Reposted-by: emv@msen.com (Edward Vielmetti, MSEN)
The following describes a neural network simulation environment made
available free from the MITRE Corporation. The software contains a neural
network simulation code generator which generates high performance C code
implementations for backpropagation networks. Also included is a
graphical interface for visualization.
PUBLIC DOMAIN
NEURAL NETWORK SIMULATOR
AND GRAPHICS ENVIRONMENT
AVAILABLE
Aspirin for MIGRAINES
Version 4.0
The Mitre Corporation is making available free to the public a neural
network simulation environment called Aspirin for MIGRAINES. The
software consists of a code generator that builds neural network
simulations by reading a network description (written in a language
called "Aspirin") and generates a C simulation. A graphical interface
(called "MIGRAINES") is provided for platforms that support the Sun
window system NeWS1.1. For platforms that do not support NeWS1.1 no
graphics are currently available.
The system has been ported to a number of platforms:
Sun3/4
Silicon Graphics Iris
IBM RS/6000
DecStation
Cray YMP
Included with the software are "config" files for these platforms.
Porting to other platforms may be done by choosing the "closest" platform
currently supported and adapting the config files.
Aspirin 4.15
=------------
The software that we are releasing now is principally for creating, and
evaluating, feed-forward networks such as those used with the
backpropagation learning algorithm. The software is aimed both at the
expert programmer/neural network researcher who may wish to tailor
significant portions of the system to his/her precise needs, as well as
at casual users who will wish to use the system with an absolute minimum
of effort.
Aspirin was originally conceived as ``a way of dealing with MIGRAINES.''
Our goal was to create an underlying system that would exist behind the
graphics and provide the network modeling facilities. The system had to
be flexible enough to allow research, that is, make it easy for a user to
make frequent, possibly substantial, changes to network designs and
learning algorithms. At the same time it had to be efficient enough to
allow large ``real-world'' neural network systems to be developed.
Aspirin uses a front-end parser and code generators to realize this goal.
A high level declarative language has been developed to describe a
network. This language was designed to make commonly used network
constructs simple to describe, but to allow any network to be described.
The Aspirin file defines the type of network, the size and topology of
the network, and descriptions of the network's input and output. This
file may also include information such as initial values of weights,
names of user defined functions, and hints for the MIGRAINES graphics
system.
The Aspirin language is based around the concept of a "black box". A
black box is a module that (optionally) receives input and (necessarily)
produces output. Black boxes are autonomous units that are used to
construct neural network systems. Black boxes may be connected
arbitrarily to create large possibly heterogeneous network systems. As a
simple example, pre or post-processing stages of a neural network can be
considered black boxes that do not learn.
The output of the Aspirin parser is sent to the appropriate code
generator that implements the desired neural network paradigm. The goal
of Aspirin is to provide a common extendible front-end language and
parser for different network paradigms. The publicly available software
will include a backpropagation code generator that supports several
variations of the backpropagation learning algorithm. For
backpropagation networks and their variations, Aspirin supports a wide
variety of capabilities:
1. feed-forward layered networks with arbitrary connections
2. ``skip level'' connections
3. one and two-dimensional tessellations
4. a few node transfer functions (as well as user defined)
5. connections to layers/inputs at arbitrary delays,
also "Waibel style" time-delay neural networks
The file describing a network is processed by the Aspirin parser and
files containing C functions to implement that network are generated.
This code can then be linked with an application which uses these
routines to control the network. Optionally, a complete simulation may be
automatically generated which is integrated with the graphics and can
read data in a variety of file formats. Currently supported file formats
are:
Ascii
Type1, Type2, Type3 (simple floating point file formats)
ProMatlab
Examples
=--------
A set of examples comes with the distribution:
xor: from RumelHart and McClelland, et al,
"Parallel Distributed Processing, Vol 1: Foundations",
MIT Press, 1986, pp. 330-334.
encode: from RumelHart and McClelland, et al,
"Parallel Distributed Processing, Vol 1: Foundations",
MIT Press, 1986, pp. 335-339.
detect: Detecting a sine wave in noise.
characters: Learing to recognize 4 characters independent
of rotation.
sonar: from Gorman, R. P., and Sejnowski, T. J. (1988).
"Analysis of Hidden Units in a Layered Network Trained to
Classify Sonar Targets" in Neural Networks, Vol. 1, pp. 75-89.
spiral: from Kevin J. Lang and Michael J, Witbrock, "Learning
to Tell Two Spirals Apart", in Proceedings of the 1988 Connectionist
Models Summer School, Morgan Kaufmann, 1988.
ntalk: from Sejnowski, T.J., and Rosenberg, C.R. (1987).
"Parallel networks that learn to pronounce English text" in
Complex Systems, 1, 145-168.
perf: A large network used only for performance testing.
Performance of Aspirin simulations
=----------------------------------
The backpropagation code generator produces simulations that run very
efficiently. Aspirin simulations do best on vector machines when the
networks are large, as exemplified by the Cray's performance. All
simulations were done using the Unix "time" function and include all
simulation overhead. The connections per second rating was calculated by
multiplying the number of iterations by the total number of connections
in the network and dividing by the "user" time provided by the Unix time
function. Two tests were performed. In the first, the network was simply
run "forward" 100,000 times and timed. In the second, the network was
timed in learning mode and run until convergence. Under both tests the
"user" time included the time to read in the data and initialize the
network.
Sonar:
This network is a two layer fully connected network
with 60 inputs: 2-34-60.
Millions of Connections per Second
Forward:
SparcStation1: 1
IBM RS/6000 320: 2.8
Cray YMP: 15.7
Backward:
SparcStation1: 0.3
IBM RS/6000 320: 0.8
Cray YMP: 7
Gorman, R. P., and Sejnowski, T. J. (1988). "Analysis of Hidden Units
in a Layered Network Trained to Classify Sonar Targets" in Neural Networks,
Vol. 1, pp. 75-89.
Nettalk:
This network is a two layer fully connected network
with [29 x 7] inputs: 26-[15 x 8]-[29 x 7]
Millions of Connections per Second
Forward:
SparcStation1: 1
IBM RS/6000 320: 3.5
Cray YMP: 64
Backward:
SparcStation1: 0.4
IBM RS/6000 320: 1.3
Cray YMP: 24.8
Sejnowski, T.J., and Rosenberg, C.R. (1987). "Parallel networks that
learn to pronounce English text" in Complex Systems, 1, 145-168.
Perf:
This network was only run on the Cray. It is very large with very long
vectors. The performance on this network is in some sense a peak
performance for a machine.
This network is a two layer fully connected network
with 2048 inputs: 128-512-2048
Millions of Connections per Second
Forward:
Cray YMP: 96.3
Backward:
Cray YMP: 18.9
Note: The cray benchmarks are courtesy of the Center for High Performance
Computing at the University of Texas.
Aspirin 5.0
=-----------
The next release of the software *may* include:
1. 2nd order (quadratic) connections
2. Auto-regressive nodes (this a form
of limited recurrence)
3. Code generators for other (not backprop) neural network
learning algorithms.
4. More supported file formats
5. More config files for other platforms.
MIGRAINES 4.0
=-------------
MIGRAINES is a graphics system for visualizing neural network systems.
The graphics that are currently being released are exclusively for
feed-forward networks. They provide the ability to display networks, arc
weights, node values, network inputs, network outputs, and target outputs
in a wide variety of formats.
There are many different representations that may be used to display arcs
weights and node values, including pseudo-color (or grayscale) arrays
(with user modifiable colors and value-to-color mappings), various plots,
bar charts and other pictorial representations.
MIGRAINES is not necessary for the execution of the Aspirin system.
Networks may be designed, executed, tested, and saved entirely apart from
any graphic interface. The more exotic the network being considered, the
smaller the amount of graphics that will be useful. However, the
graphics offer such a degree of creative and analytic power for neural
network research that even the most jaded researcher will find them
useful. Although the graphics were developed for the NeWS1.1 window
system, it can be run under Sun's OpenWindows which supports NeWS
applications. Note: OpenWindows is not 100% backward compatible with
NeWS1.1 so some features of the graphics may not work well.
MIGRAINES 5.0
=-------------
The next release will replace the NeWS1.1 graphics with an X based system
as well extending the graphical capabilities. An interface to the
scientific visualization system apE2.0 *may* be available.
How to get the software
=-----------------------
The software is available from two FTP sites, CMU's simulator collection
and UCLA's cognitive science machines. The compressed tar file is a
little more than 2 megabytes. The software is currently only available
via anonymous FTP.
> To get the software from CMU's simulator collection:
1. Create an FTP connection from wherever you are to machine "pt.cs.cmu.edu"
(128.2.254.155).
2. Log in as user "anonymous" with password your username.
3. Change remote directory to "/afs/cs/project/connect/code". Any
subdirectories of this one should also be accessible. Parent directories
should not be.
4. At this point FTP should be able to get a listing of files in this
directory and fetch the ones you want.
Problems? - contact us at "connectionists-request@cs.cmu.edu".
5. Set binary mode by typing the command "binary" ** THIS IS IMPORTANT **
6. Get the file "am4.tar.Z"
> To get the software from UCLA's cognitive science machines:
1. Create an FTP connection to "polaris.cognet.ucla.edu" (128.97.50.3)
(typically with the command "ftp 128.97.50.3")
2. Log in as user "anonymous" with password your username.
3. Change remote directory to "alexis", by typing the command "cd alexis"
4. Set binary mode by typing the command "binary" ** THIS IS IMPORTANT **
5. Get the file by typing the command "get am4.tar.Z"
How to unpack the software
=--------------------------
After ftp'ing the file make the directory you wish to install the
software. Go to that directory and type:
zcat am4.tar.Z | tar xvf -
How to print the manual
=-----------------------
The user manual is located in ./doc in a few compressed PostScript files.
To print each file on a PostScript printer type:
zcat <file.ps.Z> | lpr
Thanks
-=-----
Thanks to the folks at CMU and UCLA for the ftp site.
Thanks to the folks at the Center for High Performance Computing at the
University of Texas for the use of their computers.
Copyright and license agreement
=-------------------------------
Since the Aspirin/MIGRAINES system is licensed free of charge, the MITRE
Corporation provides absolutely no warranty. Should the Aspirin/MIGRAINES
system prove defective, you must assume the cost of all necessary
servicing, repair or correction. In no way will the MITRE Corporation be
liable to you for damages, including any lost profits, lost monies, or
other special, incidental or consequential damages arising out of the use
or in ability to use the Aspirin/MIGRAINES system.
This software is the copyright of The MITRE Corporation. It may be
freely used and modified for research and development purposes. We
require a brief acknowledgement in any research paper or other
publication where this software has made a significant contribution. If
you wish to use it for commercial gain you must contact The MITRE
Corporation for conditions of use. The MITRE Corporation provides
absolutely NO WARRANTY for this software.
February, 1991
Russell Leighton
Alexis Wieland
The MITRE Corporation
7525 Colshire Dr.
McLean, Va. 22102-3481