neuron-request@HPLABS.HP.COM ("Neuron-Digest Moderator Peter Marvit") (03/27/89)
Neuron Digest Sunday, 26 Mar 1989
Volume 5 : Issue 15
Today's Topics:
Abracts from JETAI
Summer course in computational neurobiology
Efficiency (was Re: Loan applications...)
Help with TSP
Re: Help with TSP
Unsupervised Learning Algorithms, Can you send me your comments again? [I lost
Neural net topology
Re: Data Compression
Re: Data Compression
Re: Data Compression
Re: Data Compression
light relief
Neural Network Programs
Research Opportunity for Undergraduates
genetic algorithms vs. backpropagation
noise cancellation
Send submissions, questions, address maintenance and requests for old issues to
"neuron-request@hplabs.hp.com" or "{any backbone,uunet}!hplabs!neuron-request"
ARPANET users can get old issues via ftp from hplpm.hpl.hp.com (15.255.16.205).
------------------------------------------------------------
Subject: Abracts from JETAI
From: cfields@NMSU.Edu
Date: Fri, 03 Mar 89 15:16:53 -0700
The following are abstracts of papers appearing in the inaugural issue of
the Journal of Experimental and Theoretical Artificial Intelligence. JETAI
1, 1 was published 1 January, 1989.
For submission information, please contact either of the editors:
Eric Dietrich Chris Fields
PACSS - Department of Philosophy Box 30001/3CRL
SUNY Binghamton New Mexico State University
Binghamton, NY 13901 Las Cruces, NM 88003-0001
dietrich@bingvaxu.cc.binghamton.edu cfields@nmsu.edu
JETAI is published by Taylor & Francis, Ltd., London, New York, Philadelphia
_________________________________________________________________________
Minds, machines and Searle
Stevan Harnad
Behavioral & Brain Sciences, 20 Nassau Street, Princeton NJ 08542, USA
Searle's celebrated Chinese Room Argument has shaken the foundations of
Artificial Intelligence. Many refutations have been attempted, but none
seem convincing. This paper is an attempt to sort out explicitly the
assumptions and the logical, methodological and empirical points of
disagreement. Searle is shown to have underestimated some features of
computer modeling, but the heart of the issue turns out to be an empirical
question about the scope and limits of the purely symbolic (computational)
model of the mind. Nonsymbolic modeling turns out to be immune to the
Chinese Room Argument. The issues discussed include the Total Turing Test,
modularity, neural modeling, robotics, causality and the symbol-grounding
problem.
_________________________________________________________________________
Explanation-based learning: its role in problem solving
Brent J. Krawchuck and Ian H. Witten
Knowledge Sciences Laboratory, Department of Computer Science, University of
Calgary, 2500 University Drive, NW, Calgary, Alta, Canada, T2N 1N4.
`Explanation-based' learning is a semantically-driven, knowledge-intensive
paradigm for machine learning which contrasts sharply with syntactic or
`similarity-based' approaches. This paper redevelops the foundations of EBL
from the perspective of problem-solving. Viewed in this light, the
technique is revealed as a simple modification to an inference engine which
gives it the ability to generalize the conditions under which the solution
to a particular problem holds. We show how to embed generalization
invisibly within the problem solver, so that it is accomplished as inference
proceeds rather than as a separate step. The approach is also extended to
the more complex domain of planning to illustrate that it is applicable to a
variety of logic-based problem-solvers and is by no means restricted to only
simple ones. We argue against the current trend to isolate learning from
other activity and study it separately, preferred instead to integrate it
into the very heart of problem solving.
- ----------------------------------------------------------------------------
The recognition and classification of concepts in understanding
scientific texts
Fernando Gomez and Carlos Segami
Department of Computer Science, University of Central Florida,
Orlando, FL 32816, USA.
In understanding a novel scientific text, we may distinguish the following
processes. First, concepts are built from the logical form of the sentence
into the final knowledge structures. This is called concept formation.
While these concepts are being formed, they are also being recognized by
checking whether they are already in long-term memory (LTM). Then, those
concepts which are unrecognized are integrated in LTM. In this paper,
algorithms for the recognition and integration of concepts in understanding
scientific texts are presented. It is shown that the integration of
concepts in scientific texts is essentially a classification task, which
determines how and where to integrate them in LTM. In some cases, the
integration of concepts results in a reclassification of some of the
concepts already stored in LTM. All the algorithms described here have been
implemented and are part of SNOWY, a program which reads short scientific
paragraphs and answer questions.
- ---------------------------------------------------------------------------
Exploring the No-Function-In-Structure principle
Anne Keuneke and Dean Allemang
Laboratory for Artificial Intelligence Research, Department of Computer and
Information Science, The Ohio State University, 2036 Neil Avenue Mall,
Columbus, OH 43210-1277, USA.
Although much of past work in AI has focused on compiled knowledge systems,
recent research shows renewed interest and advanced efforts both in
model-based reasoning and in the integration of this deep knowledge with
compiled problem solving structures. Device-based reasoning can only be as
good as the model used; if the needed knowledge, correct detail, or proper
theoretical background is not accessible, performance deteriorates. Much of
the work on model-based reasoning references the `no-function-in-structure'
principle, which was introduced be de Kleer and Brown. Although they were
all well motivated in establishing the guideline, this paper explores the
applicability and workability of the concept as a universal principle for
model representation. This paper first describes the principle, its intent
and the concerns it addresses. It then questions the feasibility and the
practicality of the principle as a universal guideline for model
representation.
------------------------------
Subject: Summer course in computational neurobiology
From: Jim Bower <jbower@bek-mc.caltech.edu>
Date: Sun, 05 Mar 89 18:09:10 -0800
Course announcement:
Methods in Computational Neuroscience
The Marine Biological Laboratory
Woods Hole, Massachusetts
August 6 - September 2,1989
General Description
The Marine Biological Laboratory (MBL) in Woods Hole Massachusetts is
a world famous marine biological laboratory that has been in existence for
over 100 years. In addition to providing research facilities for a large
number of biologists during the summer, the MBL also sponsors a number of
outstanding courses on different topics in Biology.
This summer will be the second year in which the MBL has offered a
course in "Methods in Computational Neuroscience". This course is designed
as a survey of the use of computer modeling techniques in studying the
information processing capabilities of the nervous system and covers models
at all levels from biologically realistic single cells and networks of cells
to biologically relevant abstract models. The principle aim of the course
is to provide participants with the tools to simulate the functional
properties of those neural systems of interest to them as well as to
understand the general advantages and pitfalls of this experimental
approach.
The Specific Structure of the Course
The course itself includes both a lecture series and a computer
laboratory. The lectures are given by invited faculty whose work represents
the state of the art in computational neuroscience (see list below). The
course lecture notes have been incorporated into a book published by MIT
press (" Methods in Neuronal Modeling: From Synapses to Networks" C. Koch
and I. Segev, editors. MIT Press, Cambridge, MA.,1989).
The computer laboratory is designed to give students hands-on
experience with the simulation techniques considered in the lecture. It
also provides students with the opportunity to actually begin simulations of
neural systems of interest to them. The students are guided in this effort
by the visiting lecturers and course directors, but also by several students
from the Computational Neural Systems (CNS) graduate program at Caltech who
serve as Laboratory TAs. The lab itself consists of state of the art
graphics workstations running a GEneral NEtwork SImulation System (GENESIS)
that Dr. Bower and his colleagues at Caltech have constructed over the last
several years. Students return to their home institutions with the GENESIS
system to continue their work.
The Students
The course is designed for advanced graduate students and
postdoctoral fellows in biology, computer science, electrical engineering,
physics, or psychology with an interest in computational neuroscience.
Because of the heavy computer orientation of the Lab section, a good
computer background is required (UNIX, C or PASCAL). In addition, students
are expected to have a basic background in neurobiology. Course enrollment
is limited to 20 so as to assure the highest quality educational experience.
Course Directors
James M. Bower and Christof Koch
Computation and Neural Systems Program
California Institute of Technology
The Faculty
Paul Adams (Stony Brook)
Dan Alkon (NIH)
Richard Anderson (MIT)
John Hildebrand (Arizona)
John Hopfield (Caltech)
Rudolfo Llinas (NYU)
David Rumelhart (Stanford)
Idan Segev (Jerusalem)
Terrence Sejnowski (Salk/UCSD)
David Van Essen (Caltech)
Christoph Von der Malsburg (USC)
For further information and application materials contact:
Admissions Coordinator
Marine Biological Laboratory
Woods Hole, MA 02543
(508) 548-3705 extension 216
Application Deadline May 15, 1989
Acceptance notification in early June.
------------------------------
Subject: Efficiency (was Re: Loan applications...)
From: bph@buengc.BU.EDU (Blair P. Houghton)
Organization: Boston Univ. Col. of Eng.
Date: Sat, 11 Mar 89 19:31:24 +0000
In article <37300001@m.cs.uiuc.edu> kadie@m.cs.uiuc.edu writes:
>
>In article joe@amos.ling.ucsd.edu (Fellow Sufferer) writes:
>>
>> Hecht-Nielsen Corp of San Diego, Ca is doing just such research.
>> Their real problem was explaining why an applicant was refused [...]
>> That's not quite as easy as it sounds.
>>
> There is another potential problem, even if an explanation is found, it may
> be illegal.
>
> For example, the ANN may be very sensitive to zipcode. This is called
> redlining; in many places it is illegal.
This is cured easily by doing what humans should (in effect) do:
Don't allow the net to process irrelevant information. ZIP code has nil to
do with whether an applicant will repay. There can be no _causal_
relationship between ZIP and credit rating. (There is a large body of
evidence supporting a positive correlation, but it's got nothing to do with
the number.) To use it as input for a NN is to make the job _more_
difficult and _less_ accurate, whether it results in a "better class" of
clientele or not.
This raises the question of efficiency metrics for Neural Networks. In our
example, it is bad business to lend money to deadbeats, and it is worse
business to label potentially profitable debtors as deadbeats for erroneous
reasons. There are only so many of Donald Trump out there, thank Napoleon.
The network used to make this decision would have to be tuned to optimize
the return on the lent money.
So, like, how do you tell beforehand that it's doing its job, and that it's
not _missing_ some people who were just never allowed to have a loan before?
How do you know if a neural net is being overselective? How do you even
define the point of overselectiveness? It's easy in dollar-based problems:
the net with the most at the end of the game wins. What do you use for
dollars in other situations?
--Blair
------------------------------
Subject: Help with TSP
From: jal@wsu-cs.uucp (Jason Leigh)
Organization: Computer Science Department, Wayne State University
Date: Tue, 14 Mar 89 17:02:25 +0000
I have a problem with trying to understand how to correctly implement the
Travelling Salesman Problem (TSP) using the Boltmann machine.
I have read the pertinent parts of the PDP book and compared numerous papers
from the IEEE Conference 87 but there still seems to be something boggling
me.
>From what I understand, the objective is to minimize an energy function
that consists of components pertinent to forming a minimal permutation
matrix.
This energy function E' must be recast to a form similar to that of the
Boltzmann machine E. This means that the weights must be designed to
present hypotheses described by E'. What I have read seems to suggest that
the rate of E' is to be used as the weights for E. But the question is when
we use the Boltzmann algorithm, (flipping some state Si etc..) do the
weights need to be readjusted since the Si that pertains to E should also
pertain to E' and that would cause a change in E' and hence a change in the
weight.
Am I missing something fundamentally simple?
If any one can give me some assistance on this, perhaps elaborate on how
this is correctly done/interpreted, I would appreciate it. Any additional
references would also be helpful.
Thanks in anticipation.
Jason Leigh
------------------------------
Subject: Re: Help with TSP
From: marcoz@MARCOZ.BOLTZ.CS.CMU.EDU (Marco Zagha)
Organization: Carnegie-Mellon University, CS/RI
Date: Thu, 16 Mar 89 15:01:50 +0000
See J.J. Hopfield and D.W. Tank, "Neural Computation of Decisions in
Optimization Problems," Biological Cybernetics 52(1985) p. 141-152.
> [...] But the question is
> when we use the Boltzmann algorithm, (flipping some state Si etc..)
> do the weights need to be readjusted since the Si that pertains to E
> should also pertain to E' and that would cause a change in E' and
> hence a change in the weight.
The weights are fixed. Only activations are adjusted. The Hopfield & Tank
paper explains their TSP energy function.
== Marco (marcoz@cs.cmu.edu)
------------------------------
Subject: Unsupervised Learning Algorithms, Can you send me your comments again? [I lost all replies]
From: Francisco Camargo <camargo@cs.columbia.edu>
Date: Thu, 16 Mar 89 14:03:10 -0500
[Due to a mis-spelled command, I erase all replies that I had receive so far.
Would all of you who had already sent me messages kindly do it again ? I
appologize for the inconvenience, and appreciate the help that I'm
getting.]
> I'm currently doing a survey article on various aspects of
> Unsupervised Learning Algorithms and would like to receive
> any references related to that area. I'm already familiar
> with most of the work done by Grossberg, Kohonen, Fukushima,
> Linsker, Ziesper, etc. I'd like pointers to comparative work
> in the area, mainly if to more traditional (non-NN) classifiers.
> In any case, any pointers are greatly appreciated.
>
> I will summarize the replies and return it to anyone who requests it.
> Please, send mail to:
>
> e-mail: camargo@cs.columbia.edu
>
> US-mail: Francisco A. Camargo
> 511 Computer Science Department
> Columbia University, New York, NY, 10027
>
>
> Tnx.
------------------------------
Subject: Neural net topology
From: eghbalni@spectra.COM (Hamid Eghbalnia)
Organization: Spectragraphics, Corp., San Diego, CA
Date: Fri, 17 Mar 89 19:51:40 +0000
This is to all of you who requested a summary if I got info.
on topological study of neural nets. There are too many of
you for me to reply individually (thanks for all the interest).
I hate to dissapoint everyone, but amazing as it may seem, there
was only one reply with a reference.
Eugene M. Norris, 'Maximal rectangular relations', Lecture notes
in computer science, 56, pp.475-481.
...!nosc!spectra!eghbalni
------------------------------
Subject: Re: Data Compression
From: fozzard@boulder.Colorado.EDU (Richard Fozzard)
Organization: University of Colorado, Boulder
Date: Sat, 18 Mar 89 15:56:17 +0000
In article <10199@nsc.nsc.com> andrew@nsc.nsc.com (andrew) writes:
with regard to:
3b) Terence D. Sanger, "An Optimality Principle for Unsupervised Learning"
> (I find it mysterious that random noise training produces orientation-
> selective receptive fields spontaneously; what's the connection between
> eigenvectors of an input autocorrelation and straight lines?
> Not only are these fields similar to those found in retinal cells of
> cat and monkey, but, as one goes down the list in order of decreasing
> eigenvalue, resemble somewhat eigenstates of wave-functions of atoms
> from quantum mechanics - perhaps a coincidental isomorphism!).
>
Well, I dont have the solution to the mystery, but Lehky and Sejnowski
report a similar learning of line segment receptive fields under equally
unexpected circumstances - learning surface curvature from shading. This
work used standard back-prop instead of the Hebb rule, though.
Here's the reference:
"Neural Network Model for the Cortical Representation
of Surface Curvature from Images of Shaded Surfaces"
Sidney R. Lehky and Terrence Sejnowski
Department of Biophysics
Johns Hopkins University
In: Lund, J.S. (Ed.) Sensory Processing, Oxford (1988)
PS: If you talked to a certain Dr. Mandelbrot, he would insist that your
"coincidental isomorphism" was hardly that - remember fractals?
Richard Fozzard
University of Colorado
fozzard@boulder.colorado.edu
------------------------------
Subject: Re: Data Compression
From: aboulang@bbn.com (Albert Boulanger)
Date: Sat, 18 Mar 89 15:56:47 +0000
Here is another one from my collection since I am interested in this subject:
"Dimensionality-Reduction Using Connectionist Networks"
Eric Saud, MIT AI Memo 941 (January 1987)
Also the so-called "encoder" networks using backprop where the desired
output is set to be the input (dimensionality is reduced at the hidden layer
and the hidden layer activity can serve as the desired "real" output) and
Hinton's & McClelland's recirculating networks generalization of encoder
nets (see "Learning Representations by Recirculation" Heoffrey Hinton &
James McClelland, NIPS Proceedings AIP Press 1988) can reduce
dimensionality. In general the class of learning algorithms called
"unsupervised" learning can potentially reduce dimensionality. There is
however a spectrum of characteristics among the different unsupervised
learning procedures:
Do the reduced dimensions span the space?
Are the reduced dimensions orthogonal?
Terry Sanger's algorithm does both. It would be interesting to work out what
his learning rule does with sigmoid transfer functions for the neurons.
Albert Boulanger
BBN Systems & Technologies Corporation
aboulanger@bbn.com
------------------------------
Subject: Re: Data Compression
From: kortge@Portia.Stanford.EDU (Chris Kortge)
Organization: Stanford University
Date: Sat, 18 Mar 89 20:04:30 +0000
In article <10199@nsc.nsc.com> andrew@nsc.nsc.com (andrew) writes:
> [...]
>3a) Terence D. Sanger, "Optimal Unsupervised Learning in a Single-Layer
> Linear Feedforward Neural Network", MIT AI Lab., NE43-743, Cambridge,
> MA 02139. TDS@wheaties.ai.mit.edu
>
>The Sanger 3a) paper is highly germane; he seems to have defined a method
>whereby maximal information preservation occurs across one layer, using
>only linear elements, and a purely local update structure. The learned
>matrix of weights becomes (row-wise) the eigenvectors of the input
>autocorrelation. [...]
>Highly relevant is his comparitive data with respect to (cf. #2) self-supervised
>backprop, where numerous criteria show GHA ("Generalised Hebbian Algorithm")
>to be superior. These criteria include:
>- training time
> [and several other things]
It wasn't clear to me from reading Sanger's thesis that the GHA is obviously
faster than self-supervised backprop. He says that, for backprop, "training
time still seems to be an exponential function of the number of units in the
network." It seems like this would be problem-dependent, though, and
principal components is not that tough as typical backprop problems go.
Does anyone know of an actual scaling study on this? (E.g., where
n-dimensional random Gaussian vectors with m known principal components are
used as inputs, and n & m are varied, keeping percent variance explainable
constant, say.)
Another problem with the claim is that the fuzzy term "training time" hides
something important. Namely, Sanger's algorithm trains output units
one-by-one during the presentation of each pattern, and to my knowledge this
sequentiality is inherent. Thus it could be that the GHA is superior to
backprop when measured in "pattern time", but not when measured in real time
(i.e. operations of an ideal parallel device). Here again, I don't know the
answer; I would be interested in whatever info people have on this.
Chris Kortge
------------------------------
Subject: Re: Data Compression
From: "Erik J. Fretheim" <efrethei@BLACKBIRD.AFIT.AF.MIL>
Organization: Air Force Institute of Technology; WPAFB, OH
Date: 19 Mar 89 00:09:49 +0000
One more to check out is a paper by Daugman (I think) in the July 88 (or was
that August) issue of ASSP. In it the aouthor discussed using a nueral net
to find the Gabor coefficients for an image. Using these they were able to
represent an image in less than 1 bit per pixel. Gabor transforms are nice
in that they have a close relationship to biological image processors.
Although the information given here is somewhat sketchy, the article
shouldn't be hard to find for anyone interested. ejf
------------------------------
Subject: light relief
From: andrew@nsc.nsc.com (andrew)
Organization: National Semiconductor, Santa Clara
Date: Sun, 19 Mar 89 08:47:04 +0000
"deep variables" of physical theory. By suitably training a (proprietary)
net with constraints given by our particular version of spacetime, he finds
that the energy minima correspond to the values of many physical constants
..pi, c, h, and so on. He calls his net "god's brain". o well.
============================================================================
DOMAIN: andrew@logic.sc.nsc.com
ARPA: nsc!logic!andrew@sun.com
USENET: ...{amdahl,decwrl,hplabs,pyramid,sun}!nsc!logic!andrew
Andrew Palfreyman 408-721-4788 work
National Semiconductor MS D3969 408-247-0145 home
2900 Semiconductor Dr.
P.O. Box 58090 there's many a slip
Santa Clara, CA 95052-8090 'twixt cup and lip
------------------------------
Subject: Neural Network Programs
From: David Kanecki <kanecki@vacs.uwp.wisc.edu>
Date: Tue, 21 Mar 89 17:10:55 -0600
Neural Network Programs for PC's
--------------------------------------
I have developed a neural network simulator/ programmer for PC's or CP/M
machines. The system has a capacity of 2700 neurons, A and B can have a 128
neuron pattern but the multiple of the A and B neuron needs to be less than
2700. It comes in two versions:
KANN21 - Binary Neuron 0 or 1
2 transfer functions
Uses DELTA learning rule
KANN30 - Continous Neuron 0<x<1
1 transfer function
Uses DELTA learning rule
Accurate to 4 significant digits
For information please write:
David Kanecki, Bio. Sci./ACS
P.O. Box 93
Kenosha, WI 53141
E-Mail: kanecki@vacs.uwp.wisc.edu
------------------------------
Subject: Research Opportunity for Undergraduates
From: masud cader <GJTAMUSC%AUVM2.BITNET@CUNYVM.CUNY.EDU>
Date: Thu, 23 Mar 89 16:46:10 -0400
ANNOUNCEMENT
UNDERGRADUATE COMPUTER SCIENCE/INFORMATION SYSTEMS STUDENTS
To participate in: National Science Foundation-funded "Research Experience
for Undergraduates" at the American University, Washington, D.C.
When: Mid-May til mid-July, 1989.
Why?: To be a part of a team made up of faculty and undergraduate student
researchers working in the area of Expert Systems. The team will experiment
with software development tools, design and implement software, and develop
expert systems applications.
Students may also have the opportunity to work in other areas such as
artificial neural networks and parallel processing.
How will students be compensated?: Student participants will receive a
stipend of $2,000, plus free room and board (students MUST live on A.U.'s
campus), and be awarded three undergraduate credits. In addition, there will
be opportunities to publish papers about the research carried on in this
project.
Who is eligible?: Undergraduates with at least Junior standing (i.e., going
into Junior year or already enrolled in Junior courses) who are American
citizens are qualified. Participants must be knowledgeable about Pascal -
some 'C' experience would be nice, but not necessary.
How to apply: Send for (Part I) of application, at address given
below, or FJ4DMUSC@AUVM2.BITNET.
and a personal Profile (Part II).
The Profile should describe: your career goals, projects you have worked on,
subjects you enjoyed and why, and whether you are interested in
teaching/research as a career. The Profile need not be longer than one page.
Send the application and Profile to:
Dr. Larry Medsker
Computer Science and Information Systems Department
The American University
4400 Massachusetts Avenue N.W.
Washington, D.C. 20016
For additional information, contact any of the following at
(202) 885-1470:
Dr. Larry Medsker, Dr. Anita La Salle, Dr. Carolyn Mc Creary The American
Univ Research Experience for Undergraduates
Application - Part II
Profile
Guidelines: Describe: where you are currently enrolled and your major; what
you think your career goals are; the kinds of computing projects you have
worked on; subjects you most enjoyed and why; what you think your strengths
and weaknesses are; when you expect to graduate; and, whether you are
interested in teaching and/or research as a career and why. The Profile can
be brief (one page or less).
------------------------------
Subject: genetic algorithms vs. backpropagation
From: DMONTANA%COOPER@rcca.bbn.com
Date: Thu, 23 Mar 89 16:37:00 -0500
In the last edition, Arnfried Ossen wrote that "Our results indicate that
genetic algorithms cannot outperform backpropagation in feedforward networks
in general." This is a strong statement that is almost certainly based on
misinformation about genetic algorithms. I would like to set the record
straight and point out some important concepts for people to remember when
trying to evaluate the performance of genetic algorithms in the future.
First, all genetic algorithms are not equal. The performance of a genetic
algorithm on a particular problem is influenced by many factors including
the method of representing individuals on chromosomes and the genetic
operators used to create new individuals. The difference between genetic
algorithms is often many orders of magnitude. The generic genetic algorithm
uses a binary representation and bit-based mutation and crossover operators.
It is not particularly suited to any one problem. In our paper "Training
Feedforward Networks Using Genetic Algorithms", I and Dave Davis describe a
genetic algorithm we have developed specifically for the problem of training
neural networks which has given very good preliminary results. Unless Ossen
actually used our genetic algorithm (which is extremely unlikely), then his
results do not reflect at all on our results.
Second, when evaluating genetic algorithms with respect to backprop, it is
important to do it on a sufficiently complex problem. To evaluate genetic
algorithms on XOR or other toy problems is like evaluating a marathoner by
clocking him at 40 meters. An intuitive explanation of why genetic
algorithms should be better than standard backpropagation for complex
problems but not simple ones is provided by the following analogy. Consider
a video game where the player navigates through layers of landmines to reach
a target. Hitting a landmine puts the player back to the beginning. The
object is to minimize the time to the target. Consider two strategies: (1)
to go quickly but hit a landmine in each layer with a fixed probability or
(2) to go slowly but never hit a landmine. For a small number of layers of
landmines, strategy 1 wins. However, the expected time of strategy 1 goes
up exponentially with the number of layers while that of strategy 2 goes up
linearly. Now substitute backprop for strategy 1, genetic algorithms for
strategy 2, local minima for landmines, and complexity for the number of
layers of landmines, and you get the picture. (Note that when backprop hits
a local minima, it either restarts or pops out of the local minima in a way
which loses a lot of the information which got it there.)
Third, make sure that the evaluation function (i.e. training optimality
criterion) of the genetic algorithm matches the criterion on which it will
ultimately be judged. In particular, don't use a least-squares evaluation
function if you will judge performance on number classified correctly.
Instead, use number classified correctly as your evaluation function.
Backprop requires differentiable optimality criteria and therefore any
non-differentiable criterion must be distorted to let it work. Genetic
algorithms should not be handicapped because of this deficiency of backprop.
David Montana
dmontana@bbn.com
------------------------------
Subject: noise cancellation
From: andrew <amdahl!nsc!andrew@APPLE.COM>
Organization: National Semiconductor, Santa Clara
Date: 25 Mar 89 04:58:30 +0000
Has anybody experimented with neural nets vis a vis adaptive noise
cancellation?
============================================================================
DOMAIN: andrew@logic.sc.nsc.com
ARPA: nsc!logic!andrew@sun.com
USENET: ...{amdahl,decwrl,hplabs,pyramid,sun}!nsc!logic!andrew
Andrew Palfreyman 408-721-4788 work
National Semiconductor MS D3969 408-247-0145 home
2900 Semiconductor Dr.
P.O. Box 58090 there's many a slip
Santa Clara, CA 95052-8090 'twixt cup and lip
------------------------------
End of Neurons Digest
*********************