neuron-request@HPLMS2.HPL.HP.COM ("Neuron-Digest Moderator Peter Marvit") (05/14/91)
Neuron Digest Monday, 13 May 1991
Volume 7 : Issue 26
Today's Topics:
2 TRs: Categorical Perception and Neural Nets
2 TRs - Speaker Independent Vowel Recognition + Optimal Pruning
proceedings 3rd NN & PDP + Air Mail Postage
preprints and reports
Journal of Ideas, Vol 2 #1 Abstracts
Technical Report available: High-Level Perception
TR - Investigating Fault Tolerance in Artificial Neural Networks
Send submissions, questions, address maintenance and requests for old issues to
"neuron-request@hplabs.hp.com" or "{any backbone,uunet}!hplabs!neuron-request"
Use "ftp" to get old issues from hplpm.hpl.hp.com (15.255.176.205).
------------------------------------------------------------
Subject: 2 TRs: Categorical Perception and Neural Nets
From: Stevan Harnad <harnad@Princeton.EDU>
Date: Tue, 23 Apr 91 16:31:25 -0400
The following two tech reports are available by anonymous ftp from
directory /pub/harnad on princeton.edu. Full ftp instructions follow the
abstracts.
-------------------------------------------------
(1) Categorical Perception and the Evolution
of Supervised Learning in Neural Nets
S Harnad*, SJ Hanson*,** & J Lubin*
*Princeton University
**Siemens Research Center
[Presented at 1991 AAAI Symposium on Symbol Grounding: Problem and
Practice]
ABSTRACT: Some of the features of animal and human categorical perception
(CP) for color, pitch and speech are exhibited by neural net simulations
of CP with one-dimensional inputs: When a backprop net is trained to
discriminate and then categorize a set of stimuli, the second task is
accomplished by "warping" the similarity space (compressing
within-category distances and expanding between-category distances). This
natural side-effect also occurs in humans and animals. Such CP
categories, consisting of named, bounded regions of similarity space, may
be the ground level out of which higher-order categories are constructed;
nets are one possible candidate for the mechanism that learns the
sensorimotor invariants that connect arbitrary names (elementary
symbols?) to the nonarbitrary shapes of objects. This paper examines how
and why such compression/expansion effects occur in neural nets.
[Retrieve by anonymous ftp in binary mode as (compressed) file
harnad91.cpnets.Z from directory /pub/harnad on princeton.edu,
instructions below]
-----------------------------------------------------------------
(2) Connecting Object to Symbol in Modeling Cognition
Stevan Harnad
Department of Psychology
Princeton University
Princeton NJ 08544
[To appear in Clark, A. & Lutz, R. (Eds) (1992) "CONNECTIONISM IN
CONTEXT," Springer-Verlag]
Connectionism and computationalism are currently vying for hegemony in
cognitive modeling. At first glance the opposition seems incoherent,
because connectionism is itself computational, but the form of
computationalism that has been the prime candidate for encoding the
"language of thought" has been symbolic computationalism, whereas
connectionism is nonsymbolic. This paper examines what is and is not a
symbol system. A hybrid nonsymbolic/symbolic system will be sketched in
which the meanings of the symbols are grounded bottom-up in the system's
capacity to discriminate and identify the objects they refer to. Neural
nets are one possible mechanism for learning the invariants in the analog
sensory projection on which successful categorization is based.
"Categorical perception," in which similarity space is "warped" in the
service of categorization, turns out to be exhibited by both people and
nets, and may mediate the constraints exerted by the analog world of
objects on the formal world of symbols.
[Retrieve by anonymous ftp in binary mode as (compressed) file
harnad92.symbol.object.Z from directory /pub/harnad on princeton.edu]
To retrieve a file by ftp from a Unix/Internet site, type:
ftp princeton.edu
When you are asked for your login, type:
anonymous
For your password, type your full name
then change directories with:
cd pub/harnad
Then type:
binary
(This is for retrieving compressed files.)
To show the available files, type:
ls
Next, retrieve the file you want with (for example):
get filename.Z
When you have the file(s) you want, type:
quit
Next uncompress the file with:
uncompress filename.Z
Now the file will be called, simply,
filename
*************
The above cannot be done from Bitnet directly, but there is a fileserver
called bitftp@pucc.bitnet that will do it for you. Send it the one line
message
help
for instructions (which will be similar to the above, but will be in the
form of a series of lines in an email message that bitftp will then
execute for you).
------------------------------
Subject: 2 TRs - Speaker Independent Vowel Recognition + Optimal Pruning
From: sankar@bach.RUTGERS.EDU
Date: Tue, 23 Apr 91 18:49:46 -0400
The following two papers are now available via FTP from the neuroprose
archives. Both will be presented at IJCNN-Seattle, 1991. These papers
describe a new approach that combines Neural Networks and Decision Trees
to form a classifier that grows the neurons as it learns.
****************************************************************************
SPEAKER INDEPENDENT VOWEL RECOGNITION USING NEURAL TREE NETWORKS
Ananth Sankar and Richard Mammone
CAIP Center and Dept. of Electrical Engg.
Rutgers University, P.O. Box 1390
Piscataway, NJ 08855-1390
Speaker independent vowel recognition is a difficult pattern recognition
problem. Recently there has been much research using Multi-Layer
Perceptrons (MLP) and Decision Trees for this task. This paper presents a
new approach to this problem. A new neural architecture and learning
algorithm called Neural Tree Networks (NTN) are developed. This network
uses a tree structure with a neural network at each tree node. The NTN
architecture offers a very efficient hardware implementation as compared
to MLPs. The NTN algorithm grows the neurons while learning as opposed to
backpropagation, for which the number of neurons must be known before
learning can begin. The new algorithm is guaranteed to converge on the
training set whereas backpropagation can get stuck in local minima. A
gradient descent technique is used to grow the NTN. This approach is more
efficient than the exhaustive search techniques used in standard decision
tree algorithms. We present simulation results on a speaker independent
vowel recognition task. These results show that the new method is
superior to both MLP and decision tree methods.
*****************************************************************************
OPTIMAL PRUNING OF NEURAL TREE NETWORKS FOR IMPROVED GENERALIZATION
Ananth Sankar and Richard Mammone
CAIP Center and Dept. of Electrical Engg.
Rutgers University, P.O. Box 1390
Piscataway, NJ 08855-1390
An optimal pruning algorithm for a Neural Network recently developed
called Neural Tree Networks (NTN) is presented. The NTN is grown by a
constructive learning algorithm that decreases the classification error
on the training data recursively. The optimal pruning algorithm is then
used to improve generalization. The pruning algorithm is shown to be
computationally inexpensive. Simulation results on a speaker independent
vowel recognition task are presented to show the improved generalization
using the pruning algorithm.
***************************************************************************
To retrieve:
unix> ftp cheops.cis.ohio-state.edu (or 128.146.8.62)
Name: anonymous
Password: neuron
ftp> cd pub/neuroprose
ftp> binary
ftp> get sankar.ijcnn91_1.ps.Z
ftp> get sankar.ijcnn91_2.ps.Z
ftp> quit
unix> uncompress sankar.ijcnn*.ps
unix> lpr sankar.ijcnn91_1.ps sankar.ijcnn91_2.ps
Thanks to Jordan Pollack for making this service available!
------------------------------
Subject: proceedings 3rd NN & PDP + Air Mail Postage
From: SAYEGH@CVAX.IPFW.INDIANA.EDU
Date: Thu, 25 Apr 91 22:54:32 -0400
In the announcement of the proceedings of the third conference on Neural
Networks and Parallel Distributed Processing at Indiana-Purdue
University, there was a minor mix-up in the list of papers. We also
received a number of requests from Europe and Japan inquiring about Air
Mail costs.
Here is the info:
Proceedings can be obtained by writing to:
Ms. Sandra Fisher
Physics Department
Indiana University-Purdue University
Ft Wayne, IN 46805
and including $5 + mailing and handling costs as follows
$1 for the US,
$2 for Canada and Mexico,
$8.50 for all others by Air Mail or $3.95 by surface mail.
Checks should be made payable to The Indiana-Purdue Foundation.
The 109 page proceedings contain the following papers:
INTEGRATED AUTONOMOUS NAVIGATION BY ADAPTIVE
NEURAL NETWORKS
Dean A. Pomerleau
Department of Computer Science
Carnegie Mellon University
APPLYING A HOPFIELD-STYLE NETWORK TO DEGRADED
PRINTED TEXT RESTORATION
Arun Jagota
Department of Computer Science
State University of New York at Buffalo
RECENT STUDIES WITH PARALLEL, SELF-ORGANIZ-
ING, HIERARCHICAL NEURAL NETWORKS
O.K. Ersoy & D. Hong
School of Electrical Engineering
Purdue University
INEQUALITIES, PERCEPTRONS AND ROBOTIC PATH-
PLANNING
Samir I. Sayegh
Department of Physics
Indiana University-Purdue University
GENETIC ALGORITHMS FOR FEATURE SELECTION FOR
COUNTERPROPAGATION NETWORKS
F.Z. Brill & W.N. Martin
Department of Computer Science
University of Virginia
MULTI-SCALE VISION-BASED NAVIGATION ON DIS-
TRIBUTED-MEMORY MIMD COMPUTERS
A.W. Ho & G.C. Fox
Caltech Concurrent Computation Program
California Institute of Technology
A NEURAL NETWORK WHICH ENABLES SPECIFICATION
OF PRODUCTION RULES
N. Liu & K.J. Cios
The University of Toledo
PIECE-WISE LINEAR ESTIMATION OF MECHANICAL
PROPERTIES OF MATERIALS WITH NEURAL NETWORKS
I.H. Shin, K.J. Cios, A. Vary* & H.E. Kautz*
The University of Toledo & NASA Lewis Re-
search Center*
INFLUENCE OF THE COLUMN STRUCTURE ON INTRA-
CORTICAL LONG RANGE INTERACTIONS
E. Niebur & F. Worgotter
California Institute of Technology
LEARNING BY GRADIENT DESCENT IN FUNCTION
SPACE
Ganesh Mani
University of Wisconsin-Madison
SUCCESSIVE REFINEMENT OF MULTI-RESOLUTION REPRESENTATIONS OF THE
ENVIRONMENT IN CONNECTIONIST NETWORKS
Vasant Honovar and Leonard Uhr
Computer Sciences Department
University of Wisconsin-Madison
A NEURAL ARCHITECTURE FOR COGNITIVE MAPS
Martin Sonntag
Cognitive Science & Machine Intelligence Lab
University of Michigan
------------------------------
Subject: preprints and reports
From: Juergen Schmidhuber <schmidhu@informatik.tu-muenchen.dbp.de>
Date: 30 Apr 91 09:17:00 +0200
Recent preprints and technical reports are available via ftp:
------------------------------------------------------------------
ADAPTIVE DECOMPOSITION OF TIME
Juergen Schmidhuber, TUM
(Talk at ICANN'91, Helsinki, June 24-28, 1991)
In this paper we introduce design principles for unsupervised detection
of regularities (like causal relationships) in temporal sequences. One
basic idea is to train an adaptive predictor module to predict future
events from past events, and to train an additional confidence module to
model the reliability of the predictor's predictions. We select system
states at those points in time where there are changes in prediction
reliability, and use them recursively as inputs for higher-level
predictors. This can be beneficial for `adaptive sub-goal generation' as
well as for `conventional' goal-directed (supervised and reinforcement)
learning: Systems based on these design principles were successfully
tested on tasks where conventional training algorithms for recurrent nets
fail. Finally we describe the principles of the first neural sequence
`chunker' which collapses a self-organizing multi-level predictor
hierarchy into a single recurrent network.
LEARNING TO GENERATE SUBGOALS FOR ACTION SEQUENCES
Juergen Schmidhuber, TUM
(Talk at ICANN'91)
This paper extends the technical report FKI-129-90 (`Toward compositional
learning with neural networks').
USING ADAPTIVE SEQUENTIAL NEUROCONTROL FOR EFFICIENT LEARNING
OF TRANSLATION AND ROTATION INVARIANCE
Juergen Schmidhuber and Rudolf Huber, TUM
(Talk at ICANN'91)
This paper is based on FKI-128-90 (announced earlier).
-------------------------------------------------------------------
LEARNING TO CONTROL FAST-WEIGHT MEMORIES: AN ALTERNATIVE TO
DYNAMIC RECURRENT NETWORKS
Juergen Schmidhuber, TUM
Technical report FKI-147-91, March 26, 1991
Previous algorithms for supervised sequence learning are based on dynamic
recurrent networks. This paper describes alternative gradient-based
systems consisting of two feed-forward nets which learn to deal with
temporal sequences by using fast weights: The first net learns to produce
context dependent weight changes for the second net whose weights may
vary very quickly. One advantage of the method over the more conventional
recurrent net algorithms is the following: It does not necessarily occupy
full-fledged units (experiencing some sort of feedback) for storing
information over time. A simple weight may be sufficient for storing
temporal information. Since with most networks there are many more
weights than units, this property represents a potential for storage
efficiency. Various learning methods are derived. Two experiments with
unknown time delays illustrate the approach. One experiment shows how the
system can be used for adaptive temporary variable binding.
NEURAL SEQUENCE CHUNKERS
Juergen Schmidhuber, TUM
Technical report FKI-148-91, April 26, 1991
This paper addresses the problem of meaningful hierarchical adaptive
decomposition of temporal sequences. This problem is relevant for
time-series analysis as well as for goal-directed learning. The first
neural systems for recursively chunking sequences are described. These
systems are based on a principle called the `principle of history
compression'. This principle essentially says: As long as a predictor is
able to predict future environmental inputs from previous ones, no
additional knowledge can be obtained by observing these inputs in
reality. Only unpredicted inputs deserve attention. The focus is on a
2-network system which tries to collapse a self-organizing multi-level
predictor hierarchy into a single recurrent network (the automatizer).
The basic idea is to feed everything that was not expected by the
automatizer into a `higher-level' recurrent net (the chunker). Since the
expected things can be derived from the unexpected things by the
automatizer, the chunker is fed with a reduced description of the input
history. The chunker has a comparatively easy job in finding
possibilities for additional reductions, since it works on a slower time
scale and receives less inputs than the automatizer. Useful internal
representations of the chunker in turn are taught to the automatizer.
This leads to even more reduced input descriptions for the chunker, and
so on. Experimentally it is shown that the system can be superior to
conventional training algorithms for recurrent nets: It may require fewer
computations per time step, and in addition it may require fewer training
sequences. A possible extension for reinforcement learning and adaptive
control is mentioned. An analogy is drawn between the behavior of the
chunking system and the apparent behavior of humans.
ADAPTIVE CONFIDENCE AND ADAPTIVE CURIOSITY
Juergen Schmidhuber
Technical Report FKI-149-91, April, 26, 1991
Much of the recent research on adaptive neuro-control and reinforcement
learning focusses on systems with adaptive `world models'. Previous
approaches, however, do not address the problem of modelling the
reliability of the world model's predictions in uncertain environments.
Furthermore, with previous approaches usually some ad-hoc method (like
random search) is used to train the world model to predict future
environmental inputs from previous inputs and control outputs of the
system. This paper introduces ways for modelling the reliability of the
outputs of adaptive world models, and it describes more sophisticated and
sometimes much more efficient methods for their adaptive construction by
on-line state space exploration: For instance, a 4-network reinforcement
learning system is described which tries to maximize the future
expectation of the temporal derivative of the adaptive assumed
reliability of future predictions. The system is `curious' in the sense
that it actively tries to provoke situations for which it {\em learned to
expect to learn} something about the environment. In a very limited
sense the system learns how to learn. An experiment with a simple
non-deterministic environment demonstrates that the method can be clearly
faster than the conventional model-building strategy.
--------------------------------------------------------
To obtain copies of the papers, do:
unix> ftp 131.159.8.35
Name: anonymous
Password: your name, please
ftp> binary
ftp> cd pub/fki
ftp> get <file>.ps.Z
ftp> bye
unix> uncompress <file>.ps.Z
unix> lpr <file>.ps
Here <file> stands for any of the following six possibilities:
icanndec (Adaptive Decomposition of Time)
icannsub (Subgoal-Generator). This paper contains 5 partly
hand-drawn figures which are not retrievable. Sorry.
icanninv (Sequential Neuro-Control).
fki147 (Fast Weights)
fki148 (Sequence Chunkers)
fki149 (Adaptive Curiosity)
Please do not forget to leave your name. This will allow us to save
paper if you are on our hardcopy mailing list.
NOTE: icanninv.ps, fki148.ps, and fki149.ps are designed
for European A4 paper format (20.9cm x 29.6cm).
-----------------------------------------------------------
In case of ftp-problems contact
Juergen Schmidhuber
Institut fuer Informatik,
Technische Universitaet Muenchen
Arcisstr. 21
8000 Muenchen 2
GERMANY
or send email to
schmidhu@informatik.tu-muenchen.de
DO NOT USE REPLY!
------------------------------
Subject: Journal of Ideas, Vol 2 #1 Abstracts
From: Elan Moritz <well!moritz@apple.com>
Date: Tue, 30 Apr 91 18:22:58 -0700
+=++=++=++=++=++=++=++=++=++=++=++=++=++=++=+
please post & circulate
Announcement
.........
Abstracts of papers appearing in
Volume 2 # 1 of the Journal of Ideas
THOUGHT CONTAGION AS ABSTRACT EVOLUTION
Aaron Lynch
Abstract: Memory abstractions, or mnemons, form the basis of a
memetic evolution theory where generalized self-replicating ideas
give rise to thought contagion. A framework is presented for
describing mnemon propagation, combination, and competition. It is
observed that the transition from individual level considerations to
population level considerations can act to cancel individual
variations and may result in population behaviors. Equations for
population memetics are presented for the case of two-idea
interactions. It is argued that creativity via innovation of ideas is
a population phenomena. Keywords: mnemon, meme, evolution,
replication, idea, psychology, equation.
...................
CULTURE AS A SEMANTIC FRACTAL:
Sociobiology and Thick Description
Charles J. Lumsden
Department of Medicine, University of Toronto
Toronto, Ontario, Canada M5S 1A8
Abstract: This report considers the problem of modeling culture as a
thick symbolic system: a system of reference and association
possessing multiple levels of meaning and interpretation. I suggest
that thickness, in the sense intended by symbolic anthropologists
like Geertz, can be treated mathematically by bringing together two
lines of formal development, that of semantic networks, and that of
fractal mathematics. The resulting semantic fractals offer many
advantages for modeling human culture. The properties of semantic
fractals as a class are described, and their role within sociobiology
and symbolic anthropology considered. Provisional empirical evidence
for the hypothesis of a semantic fractal organization for culture is
discussed, together with the prospects for further testing of the
fractal hypothesis. Keywords: culture, culturgen, meme, fractal,
semantic network.
...................
MODELING THE DISTRIBUTION OF A "MEME" IN A SIMPLE AGE DISTRIBUTION
POPULATION: I. A KINETICS APPROACH AND SOME ALTERNATIVE MODELS
Matthew Witten
Center for High Performance Computing
University of Texas System, Austin, TX 78758-4497
Abstract. Although there is a growing historical body of literature
relating to the mathematical modeling of social and historical
processes, little effort has been placed upon modeling the spread of
an idea element "meme" in such a population. In this paper we review
some of the literature and we then consider a simple kinetics
approach, drawn from demography, to model the distribution of a
hypothetical "meme" in a population consisting of three major age
groups. KEYWORDS: Meme, idea, age-structure, compartment,
sociobiology, kinetics model.
...................
THE PRINCIPIA CYBERNETICA PROJECT
Francis Heylighen, Cliff Joslyn, and Valentin Turchin
The Principia Cybernetica Project[dagger]
Abstract: This note describes an effort underway by a group of
researchers to build a complete and consistent system of philosophy.
The system will address, issues of general philosophical concern,
including epistemology, metaphysics, and ethics, or the supreme human
values. The aim of the project is to move towards conceptual
unification of the relatively fragmented fields of Systems and
Cybernetics through consensually-based philosophical development.
Keywords: cybernetics, culture, evolution, system transition,
networks, hypermedia, ethics, epistemology.
...................
Brain and Mind: The Ultimate Grand Challenge
Elan Moritz
The Institute for Memetic Research
P. O. Box 16327, Panama City, Florida 32406
Abstract: Questions about the nature of brain and mind are raised. It
is argued that the fundamental understanding of the functions and
operation of the brain and its relationship to mind must be regarded
as the Ultimate Grand Challenge problem of science. National research
initiatives such as the Decade of the Brain are discussed. Keywords:
brain, mind, awareness, consciousness, computers, artificial
intelligence, meme, evolution, mental health, virtual reality,
cyberspace, supercomputers.
+=++=++=++=++=++=++=++=++=++=++=++=++=++=++=+
The Journal of Ides an archival forum for discussion of 1)
evolution and spread of ideas, 2) the creative process, and 3)
biological and electronic implementations of idea/knowledge
generation and processing.
The Journal of Ideas, ISSN 1049-6335, is published quarterly by the
Institute for Memetic Research, Inc. P. O. Box 16327, Panama City
Florida 32406-1327.
>----------- FOR MORE INFORMATION ------->
E-mail requests to Elan Moritz, Editor, at moritz@well.sf.ca.us.
+=++=++=++=++=++=++=++=++=++=++=++=++=++=++=+
------------------------------
Subject: Technical Report available: High-Level Perception
From: David Chalmers <dave@cogsci.indiana.edu>
Date: Wed, 01 May 91 23:05:31 -0500
The following paper is available electronically from the Center for
Research on Concepts and Cognition at Indiana University.
HIGH-LEVEL PERCEPTION, REPRESENTATION, AND ANALOGY:
A CRITIQUE OF ARTIFICIAL INTELLIGENCE METHODOLOGY
David J. Chalmers, Robert M. French, and Douglas R. Hofstadter
Center for Research on Concepts and Cognition
Indiana University
CRCC-TR-49
High-level perception -- the process of making sense of complex data at
an abstract, conceptual level -- is fundamental to human cognition. Via
high-level perception, chaotic environmental stimuli are organized into
mental representations which are used throughout cognitive processing.
Much work in traditional artificial intelligence has ignored the process
of high-level perception completely, by starting with hand-coded
representations. In this paper, we argue that this dismissal of
perceptual processes leads to distorted models of human cognition. We
examine some existing artificial-intelligence models -- notably BACON, a
model of scientific discovery, and the Structure-Mapping Engine, a model
of analogical thought -- and argue that these are flawed precisely
because they downplay the role of high-level perception. Further, we
argue that perceptual processes cannot be separated from other cognitive
processes even in principle, and therefore that such
artificial-intelligence models cannot be defended by supposing the
existence of a "representation module" that supplies representations
ready-made. Finally, we describe a model of high-level perception and
analogical thought in which perceptual processing is integrated with
analogical mapping, leading to the flexible build-up of representations
appropriate to a given context.
N.B. This is not a connectionist paper in the narrowest sense, but the
representational issues discussed are very relevant to connectionism, and
the advocated integration of perception and cognition is a key feature of
many connectionist models. Also, philosophical motivation for the
"quasi-connectionist" Copycat architecture is provided.
---------------------------------------------------
This paper may be retrieved by anonymous ftp from cogsci.indiana.edu
(129.79.238.6). The file is cfh.perception.ps.Z, in the directory pub.
To retrieve, follow the procedure below.
unix> ftp cogsci.indiana.edu # (or ftp 129.79.238.6)
ftp> Name: anonymous
ftp> Password: [identification]
ftp> cd pub
ftp> binary
ftp> get cfh.perception.ps.Z
ftp> quit
unix> uncompress cfh.perception.ps.Z
unix> lpr -P(your_local_postscript_printer) cfh.perception.ps
If you do not have access to ftp, hardcopies may be obtained by sending
e-mail to dave@cogsci.indiana.edu.
------------------------------
Subject: TR - Investigating Fault Tolerance in Artificial Neural Networks
From: george@minster.york.ac.uk
Date: 02 May 91 14:17:33 +0000
[[ Editor's Note: I assume non-UK folks need to use eitehr international
coupons or translate their own currency into pounds. Hmmm, maybe it is
time to use the ecu? -PM ]]
The following technical report is now available. For a copy, email
"ilp@uk.ac.york.minster" and include your ordinary mail address. To
cover postage, a charge of 50p is made to academic institutions else
1.50. Cheques should be made payable to "The University of York".
Alternatively, email me (george@uk.ac.york.minster) and I can send you a
photocopy, though you may have to wait a little longer.
YCS 154: Investigating Fault Tolerance in
Artificial Neural Networks
G Bolt, University of York, UK
Abstract
A review of fault tolerance in neural networks is presented. Work
relating to the various issues of redundancy, reliability, complexity and
capacity are considered, as well as covering both empirical results and a
general treatment of theoretical work. It is shown that in the majority
of the work, few sound theoretical methods have been applied, and that
conventional fault tolerant techniques cannot straightforwardly be
transferred to neural networks. It is concluded that although neural
networks are often cited as being fault tolerant, little substantial
evidence is available to support this claim.
A proposal for a framework which can be used to assess fault
tolerance and robustness in neural networks, and also to guide work in
this field is given. Various factors which might influence the
reliability of such a system are discussed.
To support this framework, two fundamental prerequisite stages are
described in sections 4 and 5 which can act as a base for research into
the fault tolerance of neural networks. Section 4 describes how fault
models can be developed for neural networks visualised at the abstract
level. This involves first locating where faults can occur, and then
defining the attributes of the faults. Section 5 uses this fault model to
develop various methods which can be used to measure the reliability of
the neural network. Fault Injection and Mean-Time-Between-Failure methods
are examined, and from these a more appropriate Service Degradation
Method is developed. Two critical issues of how to measure the degree of
failure within a neural network and how to choose a suitable timescale
are discussed. The multi-layer perceptron network model is used in
examples which illustrate how ideas described here can be applied.
____________________________________________________________
George Bolt, Advanced Computer Architecture Group,
Dept. of Computer Science, University of York, Heslington,
YORK. YO1 5DD. UK. Tel: [044] (0904) 432771
george@uk.ac.york.minster JANET
george%minster.york.ac.uk@nsfnet-relay.ac.uk ARPA
george!mcsun!ukc!minster!george UUCP
____________________________________________________________
------------------------------
End of Neuron Digest [Volume 7 Issue 26]
****************************************